Usually in linux, you can add a static route using the route-eth0 that would contain a shema like
192.168.2.0/24 via 192.168.1.1

for an additional static route.

I've been told that route-xenbr0 won't work right.

How can I add a persistent static route to my xenserver box?

xe network-param-set uuid=<Network UUID> other-config:static-routes=<network>/<bitmask>/<gateway IP>

You maybe have to to an xe-toolstack-restart or reboot the system.

Could you give me an actual example of the syntax?

Thanks,

Using the 1st posters network the command would look something like this:

xe network-param-set uuid=<Network UUID> other-config:static-routes=192.168.2.0/24/192.168.1.1

You can get the network uuid by running:

xe network-list

and using the UUID of the bridge that has access to the gateway.

Hi Joseph,

Thanks for the info.

However, I have three servers, one storage VLAN and all servers have different /30 subnets :(
If I issue the command, the route only applies to one of the three servers because the route is applied to the
XenServer network the VLAN is assigned to. This route is ofcourse not correct for two of the three subnets.

Is there anyway around this?

Regards,
Vincent

Can you give me more details? I'm a little confused on your setup.

I have the same issue. Was there ever an answer?

I have a 2 server pool. It appears that both servers get the same UUID for both Pool and Network xe network-list command.

I do not want NFS traffic going out the Primay interface so I created a secondary. No problem
The path to NFS is viable (routable) through either interface so I put a static route in. No problem
When I get the uuid for the bond from server 1 and compare to server 2 they are the same. Okay
When i apply the xe network-param command it works for the first server, which has ip address of xxx.xxx.xxx.246,
When i go to the second server the management ip is xxx.xxx.xxx.247, the route statement is already applied as 246 so it does not work. I was very surprised to see this.

I broke the bond and applied to the network, but its the same. The uuid's are the same on server 1 as on server 2.
In short can i apply a static route to the server network or server bond, that is not the pool network or bond. (IE unique uuid). (The nics i am trying to setup a static route on.)

The whole thing works great for 1 server, but not for 2 or more. So i am stuck as to how to use a static route statement in a pooled environment.

a) use hardware routers
b) create a guest for the routing purpose (ie. Vyatta)

That way around, your XenServers stay clean of any services they are not designed for.

BfN, Konrad

A hardware/software router would not solve the problem. It would not help Xen determine which Network to use to get to the NFS mount. I would still need a static route to tell Xen that I wanted to use the router default gateway to get to the SR.

I need/want to be able to apply a static route to the IP address of the secondary management interface on both servers. They both seem to have the same UUID. Applying the xe network-param-set..... command works great on only the first server. If applied to the other servers it overwrites the POOL UUID for all servers. So only one can see the SR.

An alternative way is to edit directly the /etc/sysconfig/static-routes file and put in the entry:

any net 192.168.2.0 netmask 255.255.255.0 gw 192.168.1.1 metric 1 INTERFACE

where INTERFACE is whatever the designated network interface is that handles this traffic.
Of course, a "service network restart" is then needed.

--Tobias

Matt, static routes are nothing else than routing functionality. If you have a system attached to two IP segments, it will always be able to route if it is configured to do so.

Tobias, that file does not exist on my XS's - have you tested if it will survive a system restart? Many config files are being rewritten during the startup...

Konrad, I am not sure you understand the question. I thank you for trying. You are correct that static-routes are a function of routing. I will give you that. I do not think you understand Pooled server configurations.

Starting with 1 server. The NFS mount is available via both Primary and Secondary interfaces. I can tracepath through either gateway to get to the storage. I do not want Xen to use the primary gateway for storage, only management. So I apply a xe network-param-set other-config:static-routes uuid="x" destination/mask/gateway and it works great. Tracepath shows destination only uses the 2nd interface. NFS allows the connection because its using the correct IP.

When I use a Pool with the same server it works great.

When I add the 2nd server to the Pool it applies the same xe network-param-set other-config:static-routes to all servers in the Pool. The second server does not have that gateway IP on its interface, so I has no route to the NFS mount. As you know you have to use a unique IP on each interface. I cannot change the IP on the Secondary interface of all server to be identical as it would cause a conflict.

If you look further into it a bond or network interface in a pool appears to have the same UUID across all servers in the pool. So when I apply the static route for the unique IP address on Server1, ending in 246. It works great. That IP is on Server1. When I go to console of Server2 and apply the static route, xe network-param-list other-config, it shows the route for 246 has been applied. This server however has 247 for the last octet of its IP of the bond. So it fails to find the NFS mount.

Try a test of your own on one of your pooled servers. Run an xe network-list on your pooled servers. You should see that they are all the same. Then try to apply a static route to it and you will find that all servers in the pool use the route.

In that case I would go with separate IP networks - that would ideally separate the SR from guest traffic.

Vincent, did you ever get this static-routes to work in your pool?

Thank you.

Reboot your system. Networking for the dom0, for the most part, happens on 2 occasions. Pif reconfigures and startup. My suggestion, apply the static route as I've posted previously. The systems that do not have their routing table updated, reboot.

yes i have many many pools with static routes and they work fine.

No, it never exists by default, Konrad, but if you look in the /etc/rc.d/init.d/network script it always looks for this file as a standard part of bringing up networking. I use this on many Linux boxes and it is stable as part of the boot process:{code}
.
.
.
# Add non interface-specific static-routes.
if [ -f /etc/sysconfig/static-routes ]; then
grep "^any" /etc/sysconfig/static-routes | while read ignore args ; do
/sbin/route add -$args
done
fi
.
.
.{code}

--Tobias
{quote}
Tobias, that file does not exist on my XS's - have you tested if it will survive a system restart? Many config files are being rewritten during the startup... {quote}

Edited by: Tobias K on Jul 1, 2010 11:56 AM

Joseph, I think that is it. I must have my reboots out of sync. It seems to work fine now. I will tear it down and build again. To verify. Thank you for the suggestion.

I was able to get static routing working on a bonded interface after Citrix Support told me they don't support this and it can't be done. Here were the steps I performed that worked on several XenServer's.

Verify which XAPI interface is used for your management bond

add the following command:

route add -net "network/netmask" gw "gateway" dev xapi#

(ex. "route add -net 192.168.0.0/16 gw 192.168.10.1 dev xapi1")

This route will not persist after a reboot. I would recommend adding adding the exact command into your /etc/rc.local file so the route is added as part of the reboot process.

Hope this solves everyone issues as it solved mine!

Interesting. You could also create the file /etc/sysconfig/static-routes and
insert a line like this:{code}
any net 192.168.0.0 netmask 255.255.0.0 gw 192.168.10.1 metric 1 xapi1{code}
and it should do the same, without the need to add an extra local init script entry.

As mentioned before, this file does not exist by default, but if it does, it will be picked up in the network configuration process (and definitely survives reboots).

--Tobias