How To Enable Networking In Xen Guests On Hetzner's DS Servers (Debian Etch)

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Follow me on Twitter
Last edited 10/23/2009

This tutorial shows how you can enable networking in Xen guests (domU) on Hetzner's DS servers. With the DS servers, you can get a subnet of eight additional IPs (or more) - usually that subnet is different from the subnet that the server's main IP is from. The problem is that these additional IPs are bound to the MAC address of the host system (dom0) - Hetzner's routers will dump IP packets if they come from an unknown MAC address. This means we cannot use Xen's bridged mode, but must switch to Xen's routed mode where the host system (dom0) acts as the gateway for the guests.

I do not issue any guarantee that this will work for you!

1 Preliminary Note

I've tested this on a Debian Etch Xen server with existing Xen guests where I had to switch from bridged to routed mode. I haven't tested this on Debian Lenny, but I guess it will not be much different.

I'm assuming that you set up Xen on the DS server (running Debian Etch) according to this tutorial: Debian Etch And Xen From The Debian Repository.

I have an existing Xen guest on the server, vm.example.com. Now I need to configure the routed mode.

• IP of the DS server (dom0): 88.198.70.47
• Gateway: 88.198.70.33
• Netmask: 255.255.255.224
• Additional subnet: 78.47.159.32/255.255.255.240 (usable IP addresses: 78.47.159.33 - 78.47.159.46)
• IP from the subnet that will be used for vm.example.com (domU): 78.47.159.44

2 Existing Network Configuration

dom0:

On the host system, the network configuration currently looks as follows:

cat /etc/network/interfaces

### Hetzner Online AG - installimage
# Loopback device:
auto lo
iface lo inet loopback

# device: eth0
auto eth0
iface eth0 inet static
  address 88.198.70.47
  broadcast 88.198.70.63
  netmask 255.255.255.224
  gateway 88.198.70.33

# default route to access subnet
up route add -net 88.198.70.32 netmask 255.255.255.224 gw 88.198.70.33 eth0

On the guest, the network configuration is as follows:

domU:

cat /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
 address 78.47.159.44
 gateway 78.47.159.33
 netmask 255.255.255.240

 # post-up  ethtool -K eth0 tx off

#
# The commented out line above will disable TCP checksumming which
# might resolve problems for some users.  It is disabled by default
#
up route add -net 78.47.159.32 netmask 255.255.255.240 gw 78.47.159.33 eth0

Before we continue, make sure that the ethtool package is installed on the guest:

apt-get install ethtool

3 Configure The Host System (dom0)

dom0:

First make a backup of /etc/network/interfaces:

cd /etc/network
cp interfaces interfaces.old

Then open /etc/network/interfaces and make it look as follows:

vi /etc/network/interfaces

### Hetzner Online AG - installimage
# Loopback device:
auto lo
iface lo inet loopback

# device: eth0
auto eth0
iface eth0 inet static
  address 88.198.70.47
  netmask 255.255.255.255
  gateway 88.198.70.33
  pointopoint 88.198.70.33

It is important that you remove the broadcast and up route add -net lines; netmask must be changed to 255.255.255.255, and we add a pointopoint line with the same IP as the gateway (yes, it's pointopoint, not pointtopoint!).

Next we must modify the files /etc/xen/xend-config.sxp and /etc/xen/scripts/vif-common.sh, so we create backup copies of these two files first:

cd /etc/xen
cp xend-config.sxp xend-config.sxp.old
cd scripts
cp vif-common.sh vif-common.sh.old

Now open /etc/xen/xend-config.sxp and comment out the (network-script network-bridge) and (vif-script vif-bridge) lines and add (network-script network-route) and (vif-script vif-route) instead:

vi /etc/xen/xend-config.sxp

[...]
#(network-script network-bridge)
#(vif-script vif-bridge)
(network-script network-route)
(vif-script     vif-route)
[...]

Open /etc/sysctl.conf and make sure you have the following lines in it:

vi /etc/sysctl.conf

[...]
net.ipv4.conf.all.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1

net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1
net.ipv4.conf.default.proxy_arp=1
[...]

Run

sysctl -p

to read in the new configuration.

Open /etc/xen/scripts/vif-common.sh and modify the function ip_of():

vi /etc/xen/scripts/vif-common.sh

Comment out the ip addr show "$1" | awk "/^.*inet.*$1\$/{print \$2}" | sed -n '1 s,/.*,,p' line and add ip -4 -o addr show primary dev $1 | awk '$3 == "inet" {print $4; exit}' | sed 's#/.*##' instead:

[...]
##
# ip_of interface
#
# Print the IP address currently in use at the given interface, or nothing if
# the interface is not up.
#
function ip_of()
{
  #ip addr show "$1" | awk "/^.*inet.*$1\$/{print \$2}" | sed -n '1 s,/.*,,p'
  ip -4 -o addr show primary dev $1 | awk '$3 == "inet" {print $4; exit}' | sed 's#/.*##'
}
[...]

Then reboot the system:

reboot

4 Configure The Guest System

dom0:

After the reboot, please start the guest system, e.g. as follows:

xm create /etc/xen/vm.example.com.cfg

Then connect to its console:

xm console vm.example.com

domU:

Now in the guest system, make a backup copy of /etc/network/interfaces...

cd /etc/network
cp interfaces interfaces.old

... and edit /etc/network/interfaces so that it looks as follows:

vi /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
 address 78.47.159.44
 gateway 88.198.70.47
 netmask 255.255.255.255
 pointopoint 88.198.70.47
 post-up ethtool -K eth0 tx off

The address line contains the IP of the guest; the gateway line must contain the IP address of the host system (dom0) (88.198.70.47), not the host system's gateway (88.198.70.33)! The netmask must be 255.255.255.255. The pointopoint line must contain the IP address of the host system (dom0) again (88.198.70.47) (yes, it's pointopoint, not pointtopoint!).

That's it! now we can restart the guest from the host system:

dom0:

xm reboot vm.example.com

Afterwards you should be able to connect to the guest and to ping it, and you should also be able to ping other hosts from inside the guest.

5 Links

• Xen: http://www.xensource.com/xen/
• Hetzner's DS Servers: http://www.hetzner.de/en/hosting/produktmatrix/rootserver-produktmatrix-ds/