The Perfect Xen Setup For Debian And Ubuntu - Page 6

5 Create A Virtual Local Network From The Virtual Machines (Optional)

(This chapter is optional, and what is described here can only be done if you installed Xen from the sources and compiled a dom0 kernel with iptables and the dummy network driver as modules (see chapter 3.2).)

Now let's say you got a dedicated server in some data center that has one network card and only one IP address. Now you want to set up a web server (vm01) and a mail server (vm02) as virtual machines. If you would do it the way described above, you would need three public IP addresses (one for dom0, one for vm01, one for vm02), but you only got one. The solution is to set up a virtual local network on your server which means the dom0 has the public IP address and acts as a router (doing NAT, network address translation), and behind that router we have a local network (in this example it is the network

This is how you do it (all these steps have to be made on dom0!):

First, we need a second network interface; it is for the local network. Since we have only one real network card (eth0) which has the public IP address, we use the dummy network driver to set up the network interface dummy0.

echo dummy >> /etc/modules

Append the following part to /etc/network/interfaces:

auto dummy0
iface dummy0 inet static

This will give dummy0 the IP address

Then we have to tell Xen that it should bind the Xen bridge xen-br0 to dummy0. Therefore you have to edit /etc/xen/scripts/network. Change the line




Of course, we have to change the network settings in /etc/xen/vm01-config.sxp and /etc/xen/vm02-config.sxp. vm01 will have the IP address, so its configuration file looks like this:

name ="vm01"
kernel ="/boot/vmlinuz-"
root ="/dev/hda1"
memory =128
disk = ['file:/vserver/images/vm01.img,hda1,w','file:/vserver/images/vm01-swap.img,hda2,w']

# network
dhcp ="off"


Now we have to tell dom0 that it should do NAT so that the virtual machines have internet access. We also have to tell dom0 which ports it should forward to which IP address. Therefore we create the file /etc/network/if-up.d/iptables:


echo "1" > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s -j MASQUERADE

### Port Forwarding ###
iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 80 -j DNAT --to
iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 25 -j DNAT --to
iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 110 -j DNAT --to

The first two commands enable Nat'ing on dom0. In the section after ### Port Forwarding ### you put as many rules as you need. This tells dom0 to forward certain ports to certain destination ports on certain destination IP addresses. For example, the first rule tells dom0 to forward requests on port 80 (http) to port 80 on So if you have a web server running on vm01 (, then all requests on port 80 on dom0 will be forwarded to this web server. The last two rules forward ports 25 (smtp) and 110 (pop3) to our mail server vm02 (

Now we have to make that script executable:

chmod 755 /etc/network/if-up.d/iptables

Finally, we reboot the server:

shutdown -r now

After the reboot, you should have a virtual local network on your Xen system!

Whenever you need new port forwarding rules, put them at the end of /etc/network/if-up.d/iptables. And because you do not want to reboot your system whenever you need new port forwarding rules, you can run the same rule on the shell. For example, if you want to forward port 21 (ftp) to vm01, you put the rule

iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 21 -j DNAT --to

at the end of /etc/network/if-up.d/iptables. Plus, you run this rule on the shell so that it becomes valid immediately:

iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 21 -j DNAT --to





Share this page:

21 Comment(s)

Add comment


From: Anonymous at: 2006-01-30 15:17:16

This has been a very helpful howto. Thanks for creating it. A follow up for xen 3 would be appreciated!

From: Anonymous at: 2009-10-18 00:26:41


can you change this (on first page):


echo > /etc/hostname


to this:


echo server1 > /etc/hostname


Quote from `man hostname':

       /etc/hosts /etc/hostname This file should only contain domain name and not the full FQDN.

Thank you.
(i didn't want to create an account to be able to send an email...) ;-)

From: Anonymous at: 2006-03-11 15:56:39

In Ubuntu my ./ failed due to i missing the twisted framework.

The error was;

Installing Xen from './install' to '/'...
All done.
Checking to see whether prerequisite tools are installed...
FAILED check_twisted
Checks failed. See /usr/src/xen-2.0-install/check/.chkinstall for details.
All done.

I fixed this by 'apt-get install python-twisted'


From: Anonymous at: 2006-03-16 15:15:21

This is a great how-to and I only have a small addition:

If you want to run more than 4 virtual machines, you need to add a boot option for the dom0 kernel in order to permit enough file-backed virtual block devices. In the example above, the line

module /vmlinuz-2.6.11-xen0 root=/dev/hda6 ro console=tty0

in section 3.1.2 has to be modified to

module /vmlinuz-2.6.11-xen0 root=/dev/hda6 ro console=tty0 max_loop=X

where the X needs to be replaced by 2 times the number of virtual machines.

Of course, as the Xen manual points out, if you have heavy I/O taking places in your virtual machines, you may experience performace losses if you increase X much beyond the default value of 8.

Dirk Petry

From: Anonymous at: 2006-05-06 21:21:13

as well as the max_loop=16 (or 2 x however many VMs you want)

you might need to make the loop devices eg /dev/loop8 /dev/loop9 etc

This helped me:

for i in $(seq 8 63) ; do mknod /dev/loop$i b 7 $i ; done

chmod 660 /dev/loop* ; chown 0.disk /dev/loop*

HTH Simon Faulkner

From: Anonymous at: 2005-11-16 16:58:47

If you receive:

/root/xen-2.0/xen/include/asm/processor.h:175: error: array type has incomplete element type

You may need to specify the version of GCC you wish to use for the compile.

I found people recommending to use:
make CC=gcc-3.2 world

However, the best I could get was gcc-3.3 (ie. /usr/bin/gcc-3.3) so I did:
make CC=gcc-3.3 world

From: Anonymous at: 2005-11-16 17:33:12

My previous comment regarding make world also applies to make install

From: Anonymous at: 2005-12-02 03:52:19

When I compiled the kernel, I also needed to apt-get install patch to get the patch program.

From: Anonymous at: 2005-12-21 17:21:54

Steps that begin with "cd .." cause confusion. I suggest either changing to a specific folder "cd ~/xen" or at least commenting where you are changing to "#Change to the ~/xen folder" so the user can know they are in the correct path.

This is very useful. Thanks for taking the time to document the perfect xen setup!


From: admin at: 2005-12-22 09:00:21

As stated at the beginning of this tutorial, this is a "copy & paste" howto, i.e. you should simply follow the steps presented here without doing anything else in between (e.g. change directories or something like that) - then it will work without problems.

From: Anonymous at: 2006-01-11 20:17:03

Falko has done a brilliant job with this Howto. Apart from very minor things such as not mounting devpts after chrooting, it worked without a hitch. I've pasted a slightly modified version of section 3 below to show how I installed Xen 3.0.0 using Falko's Howto.



3.2 Installing From The Sources

Run the following commands:

apt-get remove exim4 exim4-base lpr nfs-common portmap pidentd pcmcia-cs pppoe pppoeconf ppp pppconfig
apt-get install iproute bridge-utils python-twisted gcc-3.3 binutils make libcurl3-dev zlib1g-dev python-dev transfig bzip2 screen ssh debootstrap libcurl-dev libncurses5-dev
(1 line!)
cd ~
mkdir xen
cd xen
mkdir archive
cd archive

3.2.1 Install Xen

Now execute these commands:


cd ..

tar -xvzf archive/xen-unstable.7-src.tgz
tar -xvjf archive/linux-2.6.12.tar.bz2

cd xen-unstable
make world

make install
sh ./

Now Xen is installed. In order to start the Xen services at boot time, do the following:

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20

3.2.2 Compile A New dom0 Kernel

Next we compile a new dom0 kernel with Xen-, iptables-, quota-, and dummy support. The kernel will be installed to ~/xen/install so that we can save it for other machines. From ~/xen/install we will install it to the real locations. Xen works with kernel 2.6.11, so I take the latest 2.6.12 kernel (2.6.12) instead of newer kernels, e.g. 2.6.14.

cd ..
mv /lib/tls /lib/tls.disabled
cp -al linux-2.6.12 linux-2.6.12-xen0
cd xen-unstable/linux-2.6-xen-sparse/
./mkbuildtree ../../linux-2.6.12-xen0/
cd ../..
echo "-xen0" > linux-2.6.12-xen0/localversion-xen
diff -Naur linux-2.6.12 linux-2.6.12-xen0 > archive/linux-2.6.12-xen0.patch

cd linux-2.6.12
patch -p1 < ../archive/linux-2.6.12-xen0.patch Apply additional patches for Xen 3.0 (unstable) for dom0 Kernel

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/smp-alts.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/rcu-nohz.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/pmd-shared.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/net-csum.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/cpu-hotplug-init.patch

cd ../
mkdir install
cd linux-2.6.12-xen0
cp arch/xen/configs/xen0_defconfig .config
ARCH=xen; INSTALL_PATH=../install; INSTALL_MOD_PATH=../install
make oldconfig
make menuconfig

In the kernel configuration menu that shows up we have to enable quota, iptables and the dummy network driver as modules. This is where you enable these modules:

File systems --> [*] Quota support
<M> Old quota format support
<M> Quota format v2 support

Device Drivers ---> Networking support ---> <M> Dummy net driver support

Device Drivers ---> Networking support ---> Networking options ---> [*] Network packet filtering (replaces ipchains) ---> IP: Netfilter Configuration ---> <M> IP tables support (required for filtering/masq/NAT)

[*] means: build into the kernel statically.
means: build as a kernel module.

Now we install the kernel to ~/xen/install:

make modules
make modules_install
make install

Finally, we copy the kernel to the "real" locations:

cd ../install/
cp boot/* /boot
cp -r lib/modules/2.6.12-xen0/ /lib/modules
cp -r usr/include/xen /usr/include

3.2.3 Configure The Bootloader And Reboot

I configured my kernel with SMP support, hence the name I have given my kernel. Make sure you change the other settings to suit as per falko’s howto.

title Xen 3.0.0 Debian SID dom0 2.6.12-6 SMP

root (hd0,0)

kernel /xen.gz dom0_mem=1024M vga=791

module /vmlinuz- root=/dev/hde2 ro console=tty0

From: Anonymous at: 2005-12-11 19:03:43

I had to make the ptmx device as well to get it up and running...

Issue the following command before mounting it:

mknod --mode=666 /dev/ptmx c 5 2

From: Anonymous at: 2005-12-05 22:47:08

Great document! Everything worked like a charm... except for one thing...

I recieved an error while running the base-config command. It constantly stopped with "Terminated" being output to the terminal. I ran strace against the command and I found that /dev/pts is not actually mounted in that file system, so it could not open the device (weird how that works). So, make sure to run the following command before running base-config to remedy this issue:

mount -t devpts -o rw,gid=5,mode=620 none /dev/pts

Also, FYI: Xen running a virtual machine inside of a VMWare GSX server does not work so well.

Thanks for the great write-up!

From: Anonymous at: 2006-02-24 15:07:16

Under Ubuntu 5.10 (breezy) instead of doing 'apt-get install localeconf' which will fail anyway unless you add the universe repository, just issue this command:

dpkg-reconfigure locales

Then you can select:

en_US ISO-8859-1


From: Anonymous at: 2006-02-27 22:21:40

Some annotation to the last sentence of this page:

if you get a warning like this: umount: /vserver/vm_base: device is busy it probably means that you accidentally started some daemon out of your chroot. At least it happened to me. I chrooted into the vserver disk, installed ssh and it immediately stopped my regulary ssh daemon and started that of the chroot instead. That was the cause of the error message.

Otherwise thanks for this document!

From: Anonymous at: 2005-12-18 18:10:01

I had to create the hda2 entry under /dev in order to have the swap (I use the Xen 3.0 version):

mknod --mode=666 /dev/hda2 b 3 2 (for example)

Anyway, thanks a lot for this wonderful work!!

From: Anonymous at: 2005-11-15 08:15:20

Thx for this nice in depth article. Fine work. I would like to point out 2 things. As of Nov 13. Intel releases its VT processors[1], so you can run even an unmodifyed windows within a Xen 3.0 domain. There is another Xen related debian project called Xenophilia that is worth a look.


the polarizer

From: Anonymous at: 2006-01-30 13:15:59

needed for iptables:

apt-get install modconf module-init-tools

From: falko at: 2006-01-18 17:46:06

Yes, as soon as anybody cares to answer this question:

It seems I'm having the same problem than you... :-(

From: Anonymous at: 2006-01-18 12:53:23

Hi there,

first of all, thanks for the really great setup tutorial. It worked out great for me.

Could you also provide help on how to set up the virtual local network in Xen 3.0, as this page seems not applicable to that version.

Kind regards


From: Anonymous at: 2006-01-27 11:42:06

Instead of dummy driver you could just use the IP alias feature (eth0:1)! correct?