Navigation

Assarbad · #1 19th February 2009, 05:27

Hi,

on my Debian Lenny (freshly upgraded from Etch), I have a weird issue that didn't happen on Etch.
(Linux yggdrasil 2.6.26-1-xen-amd64 #1 SMP Sat Jan 10 20:39:26 UTC 2009 x86_64 GNU/Linux)

After figuring out why I couldn't boot into the new kernel after upgrade (Lenny uses Xen 3.2, but my old Xen 3.0 was retained ... so removing it and installing the 3.2 version fixed that issue), I needed to adjust the kernel and ramdisk parameters in the .cfg file of the domUs.

In addition, I had to change this line in /etc/xen/xend-config.sxp:

Code:

(network-script network-route)

to

Code:

(network-script 'network-route netdev=eth0')

Now

Code:

/etc/init.d/xend start

works as expected.

Only one huge issue left. Whenever I try to create a domU from any of the config files, it fails with the following error:

Code:

Error: Device 0 (vif) could not be connected. /etc/xen/scripts/vif-route failed; error detected.

The gist of my xend configuration file is this:

Code:

(xend-address localhost)
(network-script 'network-route netdev=eth0')
(vif-script     vif-route)
(dom0-min-mem 96)
(dom0-cpus 0)

Only the "network-script" line had to be changed after the upgrade, everything else is as it used to be.

I need to use the routing setup, but for some weird reason it's impossible to get the vif interfaces created. They also do not appear inside the output from ifconfig.

Any ideas?

Assarbad · #2 19th February 2009, 07:43

BTW: I've tried loading various modules, including tun. There's no difference.

Once I remove the "vif=" line from my domU config, everything works ... apart from the network, of course.

falko · #3 20th February 2009, 17:52

Quote:

Originally Posted by Assarbad

Only one huge issue left. Whenever I try to create a domU from any of the config files, it fails with the following error:

Code:

Error: Device 0 (vif) could not be connected. /etc/xen/scripts/vif-route failed; error detected.

What's in your logs when this happens?

Can you post your domU config?

Assarbad · #4 21st February 2009, 02:37

Problem solved. It turns out to be rather simple once you know it, but a few changes have to be made to the system(s). I do have a /29-subnet along with my dedicated server and needed to make use of as any IPs as possible.

Certainly everyone knows that there are only 6 host-IPs in such a subnet, plus one for the network and one for the broadcast address. Now, since the configuration "outside" my server is set up in a way that allows all of the IPs to be used, I wanted to do that.

Let's assume a sample set of IPs as follows:

192.168.1.179, the main IP of the dedicated server (this is the one that appears as the last hop before any of the subnet IPs in a traceroute)
192.168.1.161, the IP of the gateway for the network if we assume a /27-subnet on which the main IP sits
192.168.2.184/29, the subnet whose IPs we want to use. In this case, 192.168.2.184 is the network itself, 192.168.2.185 through 192.168.2.190 are the host IPs and 192.168.2.191 is the broadcast IP in a traditional sense. The IP 192.168.2.190 is the one we use for this example case (in the configs below)!

The /etc/network/interfaces on the dom0 needs to be configured this way for the eth0 entry (well, variations are possible, obviously ...):

Code:

auto eth0
iface eth0 inet static
   address   192.168.1.179
   netmask   255.255.255.224
   gateway   192.168.1.161
   pointopoint 192.168.1.161
   dns-search yourdomain.net
   dns-nameservers 192.168.0.1 192.168.0.5

The pointopoint (and its "strange" spelling is indeed required!) is the important point here.

The domUs get a "vif"-line like this:

Code:

vif  = [ 'mac=AA:BB:CC:DD:EE:FF,ip=192.168.2.190' ]

Please note that it is best to define a MAC address here, one way of doing so would be to start the domU without it defined, i.e. with the line looking like:

Code:

vif  = [ 'ip=192.168.2.190' ]

... and then using ifconfig inside the domU to find the MAC that was assigned.

Now in your domUs the /etc/network/interfaces has to look like this (staying with 192.168.2.190):

Code:

auto eth0
iface eth0 inet static
   address 192.168.2.190 # domU IP
   netmask 255.255.255.255 # netmask to restrict broadcasts to the IP itself
   gateway 192.168.1.179 # main IP acts as gateway
   pointopoint 192.168.1.179 # main IP again
   dns-search yourdomain.net
   dns-nameservers 192.168.0.1 192.168.0.5
   post-up ethtool -K eth0 tx off

The last line ensures that potential MTU differences between domU and dom0 won't cause trouble.

Now one last change is required in /etc/xen/scripts/vif-common.sh (take a backup of the file first

) inside the ip_of function:

Code:

ip_of()
{
# The next line was the original line
#  ip addr show "$1" | awk "/^.*inet.*$1\$/{print \$2}" | sed -n '1 s,/.*,,p'
# The next line is used with routed setup + point-to-point
  ip -4 -o addr show primary dev $1 | awk '$3 == "inet" {print $4; exit}' | sed 's#/.*##'
}

Also don't forget to use only the following network scripts in your /etc/xen/xend-config.sxp and comment out others (unless you have a more complicated custom setup):

Code:

(network-script 'network-route netdev=eth0')
(vif-script     vif-route)

Note: on Etch (and probably prior to it), the following lines would be sufficient instead:

Code:

(network-script network-route)
(vif-script     vif-route)

... on Lenny, however, you need to add the interface name in the netdev parameter. Make sure to adjust that to your system if you have more than one actual interface or your first one is not eth0.

Please note that all of this is very likely specific to Debian and Debian-derived distros! That's because of the use of pointopoint, mainly. However, similar facilities in other distros will obviously allow similar scenarios.

Additional notes:
My problem was an upgrade from Etch to Lenny. Both inside the domUs and in the dom0. However, due to it not working, I thought I'd use the downtime to move to a different (better) server. It needs to be added that in Lenny apart from the netdev-parameter in the xend configuration, one more change was needed in the configuration of the domUs, instead of the previous setting:

Code:

extra = 'xennet.rx_copy'

... I had to switch to this one:

Code:

extra = 'xennet.rx_copy, xencons=tty'

... in order to make "xm console <domU>" work (otherwise it hung).
Another thing happened in the domUs. Unlike before, the modules of the kernel from the dom0 had to be installed inside the domU(s). This could be circumvented by modifying the initrd appropriately instead. Also, I needed to install the udev package to get SSH to work again, because a connection attempt always failed to allocate a pty ...

References (in German):
http://wiki.hetzner.de/index.php/9_I...sen_mit_Ubuntu
http://wiki.hetzner.de/index.php/Ums...N_3_auf_routed

falko · #5 21st February 2009, 18:30

Quote:

Originally Posted by Assarbad

The /etc/network/interfaces on the domU needs to be configured this way for the eth0 entry (well, variations are possible, obviously ...):

Code:

auto eth0
iface eth0 inet static
   address   192.168.1.179
   netmask   255.255.255.224
   gateway   192.168.1.161
   pointopoint 192.168.1.161
   dns-search yourdomain.net
   dns-nameservers 192.168.0.1 192.168.0.5

The pointopoint (and its "strange" spelling is indeed required!) is the important point here.

You mean dom0 instead of domU, right?