This is a "copy & paste" HowTo! The easiest way to follow this tutorial is to use a command line client/SSH client (like PuTTY for Windows) and simply copy and paste the commands (except where you have to provide own information like IP addresses, hostnames, passwords,...). This helps to avoid typos.

The Perfect Xen Setup For Debian And Ubuntu

Version 1.1
Author: Falko Timme
Last edited 12/18/2005

This tutorial provides step-by-step instructions on how to install Xen (version 2; I have not tested this with version 3 yet) on a Debian Sarge (3.1) system. It should apply to Ubuntu systems with little or no modifications.

Xen lets you create guest operating systems (*nix operating systems like Linux and FreeBSD), so called "virtual machines" or domUs, under a host operating system (dom0). Using Xen you can separate your applications into different virtual machines that are totally independent from each other (e.g. a virtual machine for a mail server, a virtual machine for a high-traffic web site, another virtual machine that serves your customers' web sites, a virtual machine for DNS, etc.), but still use the same hardware. This saves money, and what is even more important, it's more secure. If the virtual machine of your DNS server gets hacked, it has no effect on your other virtual machines. Plus, you can move virtual machines from one Xen server to the next one.

I will use Debain Sarge for both the host OS (dom0) and the guest OS (domU). In an additional section at the end I will also show how to create a virtual local network with virtual machines, with dom0 being the router.

This howto is meant as a practical guide; it does not cover the theoretical backgrounds. They are treated in a lot of other documents in the web.

This document comes without warranty of any kind! I want to say that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Install The Debian Host System (dom0)

You can overall follow these instructions, but with a few changes:

However, it's important that you type linux26 at the boot prompt to install a kernel 2.6 system. dom0's FQDN in this example will be server1.example.com, so I specify server1 as Hostname and example.com as Domain name. server1.example.com's IP address will be 192.168.0.100 in this tutorial.

When it comes to the partitioning, I select Manually edit partition table. I create the following partitions:

  • /boot 100 MB (Primary) (Location for the new partition: Beginning) (ext3) (Bootable flag: on <-- important, otherwise your system will not boot!)
  • swap 1GB (Logical) (Location for the new partition: Beginning)
  • / 2GB (Logical) (Location for the new partition: Beginning) (ext3)
  • /vserver the rest (Logical) (Location for the new partition: Beginning) (ext3)

(Side note: You can also install everything in one big partition (as described here: http://www.howtoforge.com/perfect_setup_debian_sarge), but then you have to keep in mind that the Grub stanzas I describe in this howto are slightly different. For example, when I write that I add

title        Xen 2.0.6 / XenLinux 2.6.11.12-xen0
root (hd0,0)
kernel /xen.gz dom0_mem=65536
module /vmlinuz-2.6.11.12-xen0 root=/dev/hda6 ro console=tty0

to /boot/grub/menu.lst then you should probably use

title        Xen 2.0.7 / XenLinux 2.6.11.12-xen0
root (hd0,0)
kernel /boot/xen.gz dom0_mem=65536
module /boot/vmlinuz-2.6.11.12-xen0 root=/dev/hda1 ro console=tty0

in that file...)

When the Debian installer prompts Choose software to install: I make no selection and go on (dom0 should run as few software as possible in order not to be vulnerable to attacks. To the outside world it will be accessible only over SSH.).

2 Configure dom0's Network

Because the Debian Sarge installer has configured our system to get its network settings via DHCP, we have to change that now because a server should have a static IP address. Edit /etc/network/interfaces and adjust it to your needs (in this example setup I will use the IP address 192.168.0.100):

# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
auto lo
iface lo inet loopback

# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
auto eth0
iface eth0 inet static
address 192.168.0.100
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.1

Then restart your network:

/etc/init.d/networking restart

Edit /etc/resolv.conf and add some nameservers:

search server
nameserver 145.253.2.75
nameserver 193.174.32.18
nameserver 194.25.0.60

Then set dom0's hostname:

echo server1.example.com > /etc/hostname
/bin/hostname -F /etc/hostname

3 Install Xen

There are two ways to install Xen: install the binary package from the Xen website, or compile Xen and the Xen kernels from the sources.

The first way is easier, but it has the disadvantage that the domU kernel that comes with the binary package has no support for quota and iptables, both features that I need in my virtual machines (domU). Plus, the dom0 kernel has no support for the dummy network driver, which I need at the end in the optional chapter 5 where I describe how to set up a virtual local network with virtual machines.

In chapter 3.1 I describe how to install the Xen binary package which is recommended for beginners (skip chapter 3.2 and continue with chapter 4). If you need quota and iptables in your virtual machines, then skip chapter 3.1 and continue with chapter 3.2 where I show how to install Xen from the sources.

Share this page:

21 Comment(s)

Add comment

Comments

From: Anonymous at: 2006-01-30 15:17:16

This has been a very helpful howto. Thanks for creating it. A follow up for xen 3 would be appreciated!

From: Anonymous at: 2009-10-18 00:26:41

Hi,

can you change this (on first page):

 

echo server1.example.com > /etc/hostname

 

to this:

 

echo server1 > /etc/hostname

 

Quote from `man hostname':

FILES
       /etc/hosts /etc/hostname This file should only contain domain name and not the full FQDN.

Thank you.
(i didn't want to create an account to be able to send an email...) ;-)

From: Anonymous at: 2006-03-11 15:56:39

In Ubuntu my ./install.sh failed due to i missing the twisted framework.

The error was;

Installing Xen from './install' to '/'...
All done.
Checking to see whether prerequisite tools are installed...
FAILED check_twisted
Checks failed. See /usr/src/xen-2.0-install/check/.chkinstall for details.
All done.

I fixed this by 'apt-get install python-twisted'

k31th

From: Anonymous at: 2006-03-16 15:15:21

This is a great how-to and I only have a small addition:

If you want to run more than 4 virtual machines, you need to add a boot option for the dom0 kernel in order to permit enough file-backed virtual block devices. In the example above, the line

module /vmlinuz-2.6.11-xen0 root=/dev/hda6 ro console=tty0

in section 3.1.2 has to be modified to

module /vmlinuz-2.6.11-xen0 root=/dev/hda6 ro console=tty0 max_loop=X

where the X needs to be replaced by 2 times the number of virtual machines.

Of course, as the Xen manual points out, if you have heavy I/O taking places in your virtual machines, you may experience performace losses if you increase X much beyond the default value of 8.


Dirk Petry

From: Anonymous at: 2006-05-06 21:21:13

as well as the max_loop=16 (or 2 x however many VMs you want)

you might need to make the loop devices eg /dev/loop8 /dev/loop9 etc

This helped me:

for i in $(seq 8 63) ; do mknod /dev/loop$i b 7 $i ; done

chmod 660 /dev/loop* ; chown 0.disk /dev/loop*

HTH Simon Faulkner

From: Anonymous at: 2005-11-16 16:58:47

If you receive:

/root/xen-2.0/xen/include/asm/processor.h:175: error: array type has incomplete element type

You may need to specify the version of GCC you wish to use for the compile.

I found people recommending to use:
make CC=gcc-3.2 world

However, the best I could get was gcc-3.3 (ie. /usr/bin/gcc-3.3) so I did:
make CC=gcc-3.3 world


From: Anonymous at: 2005-11-16 17:33:12

My previous comment regarding make world also applies to make install

From: Anonymous at: 2005-12-02 03:52:19

When I compiled the kernel, I also needed to apt-get install patch to get the patch program.

From: Anonymous at: 2005-12-21 17:21:54

Steps that begin with "cd .." cause confusion. I suggest either changing to a specific folder "cd ~/xen" or at least commenting where you are changing to "#Change to the ~/xen folder" so the user can know they are in the correct path.

This is very useful. Thanks for taking the time to document the perfect xen setup!

Tim


From: admin at: 2005-12-22 09:00:21

As stated at the beginning of this tutorial, this is a "copy & paste" howto, i.e. you should simply follow the steps presented here without doing anything else in between (e.g. change directories or something like that) - then it will work without problems.

From: Anonymous at: 2006-01-11 20:17:03

Falko has done a brilliant job with this Howto. Apart from very minor things such as not mounting devpts after chrooting, it worked without a hitch. I've pasted a slightly modified version of section 3 below to show how I installed Xen 3.0.0 using Falko's Howto.

cheers,

Hizar

3.2 Installing From The Sources

Run the following commands:

apt-get remove exim4 exim4-base lpr nfs-common portmap pidentd pcmcia-cs pppoe pppoeconf ppp pppconfig
apt-get install iproute bridge-utils python-twisted gcc-3.3 binutils make libcurl3-dev zlib1g-dev python-dev transfig bzip2 screen ssh debootstrap libcurl-dev libncurses5-dev
(1 line!)
cd ~
mkdir xen
cd xen
mkdir archive
cd archive

3.2.1 Install Xen

Now execute these commands:

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.12.tar.bz2
wget http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-3.0.0-src.tgz

cd ..

tar -xvzf archive/xen-unstable.7-src.tgz
tar -xvjf archive/linux-2.6.12.tar.bz2

cd xen-unstable
make world

make install
sh ./install.sh

Now Xen is installed. In order to start the Xen services at boot time, do the following:

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20

3.2.2 Compile A New dom0 Kernel

Next we compile a new dom0 kernel with Xen-, iptables-, quota-, and dummy support. The kernel will be installed to ~/xen/install so that we can save it for other machines. From ~/xen/install we will install it to the real locations. Xen works with kernel 2.6.11, so I take the latest 2.6.12 kernel (2.6.12) instead of newer kernels, e.g. 2.6.14.

cd ..
mv /lib/tls /lib/tls.disabled
cp -al linux-2.6.12 linux-2.6.12-xen0
cd xen-unstable/linux-2.6-xen-sparse/
./mkbuildtree ../../linux-2.6.12-xen0/
cd ../..
echo "-xen0" > linux-2.6.12-xen0/localversion-xen
diff -Naur linux-2.6.12 linux-2.6.12-xen0 > archive/linux-2.6.12-xen0.patch

cd linux-2.6.12
patch -p1 < ../archive/linux-2.6.12-xen0.patch

3.2.2.1 Apply additional patches for Xen 3.0 (unstable) for dom0 Kernel

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/smp-alts.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/rcu-nohz.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/pmd-shared.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/net-csum.patch

Patch –p1 < ~/xen/xen-unstable/patches/linux-2.6.12/cpu-hotplug-init.patch

cd ../
mkdir install
cd linux-2.6.12-xen0
cp arch/xen/configs/xen0_defconfig .config
ARCH=xen; INSTALL_PATH=../install; INSTALL_MOD_PATH=../install
export ARCH INSTALL_PATH INSTALL_MOD_PATH
make oldconfig
make menuconfig

In the kernel configuration menu that shows up we have to enable quota, iptables and the dummy network driver as modules. This is where you enable these modules:

File systems --> [*] Quota support
<M> Old quota format support
<M> Quota format v2 support

Device Drivers ---> Networking support ---> <M> Dummy net driver support

Device Drivers ---> Networking support ---> Networking options ---> [*] Network packet filtering (replaces ipchains) ---> IP: Netfilter Configuration ---> <M> IP tables support (required for filtering/masq/NAT)

[*] means: build into the kernel statically.
<M>
means: build as a kernel module.

Now we install the kernel to ~/xen/install:

make modules
make modules_install
make install

Finally, we copy the kernel to the "real" locations:

cd ../install/
cp boot/* /boot
cp -r lib/modules/2.6.12-xen0/ /lib/modules
cp -r usr/include/xen /usr/include

3.2.3 Configure The Bootloader And Reboot

I configured my kernel with SMP support, hence the name I have given my kernel. Make sure you change the other settings to suit as per falko’s howto.

title Xen 3.0.0 Debian SID dom0 2.6.12-6 SMP

root (hd0,0)

kernel /xen.gz dom0_mem=1024M vga=791

module /vmlinuz-2.6.12.6-xen0 root=/dev/hde2 ro console=tty0

From: Anonymous at: 2005-12-11 19:03:43

I had to make the ptmx device as well to get it up and running...

Issue the following command before mounting it:

mknod --mode=666 /dev/ptmx c 5 2

From: Anonymous at: 2005-12-05 22:47:08

Great document! Everything worked like a charm... except for one thing...

I recieved an error while running the base-config command. It constantly stopped with "Terminated" being output to the terminal. I ran strace against the command and I found that /dev/pts is not actually mounted in that file system, so it could not open the device (weird how that works). So, make sure to run the following command before running base-config to remedy this issue:

mount -t devpts -o rw,gid=5,mode=620 none /dev/pts

Also, FYI: Xen running a virtual machine inside of a VMWare GSX server does not work so well.

Thanks for the great write-up!

From: Anonymous at: 2006-02-24 15:07:16

Under Ubuntu 5.10 (breezy) instead of doing 'apt-get install localeconf' which will fail anyway unless you add the universe repository, just issue this command:

dpkg-reconfigure locales

Then you can select:

en_US ISO-8859-1

-j

From: Anonymous at: 2006-02-27 22:21:40

Some annotation to the last sentence of this page:

if you get a warning like this: umount: /vserver/vm_base: device is busy it probably means that you accidentally started some daemon out of your chroot. At least it happened to me. I chrooted into the vserver disk, installed ssh and it immediately stopped my regulary ssh daemon and started that of the chroot instead. That was the cause of the error message.

Otherwise thanks for this document!

From: Anonymous at: 2005-12-18 18:10:01

I had to create the hda2 entry under /dev in order to have the swap (I use the Xen 3.0 version):

mknod --mode=666 /dev/hda2 b 3 2 (for example)

Anyway, thanks a lot for this wonderful work!!

From: Anonymous at: 2005-11-15 08:15:20

Thx for this nice in depth article. Fine work. I would like to point out 2 things. As of Nov 13. Intel releases its VT processors[1], so you can run even an unmodifyed windows within a Xen 3.0 domain. There is another Xen related debian project called Xenophilia that is worth a look.

[1] http://www.nordichardware.com/news,2253.html
[2] http://cosi.clarkson.edu/xen/

the polarizer
http://www.codixx.de/polarizer.html




From: Anonymous at: 2006-01-30 13:15:59

needed for iptables:

apt-get install modconf module-init-tools

From: falko at: 2006-01-18 17:46:06

Yes, as soon as anybody cares to answer this question:

http://lists.xensource.com/archives/html/xen-users/2006-01/msg00432.html

It seems I'm having the same problem than you... :-(

From: Anonymous at: 2006-01-18 12:53:23

Hi there,

first of all, thanks for the really great setup tutorial. It worked out great for me.

Could you also provide help on how to set up the virtual local network in Xen 3.0, as this page seems not applicable to that version.

Kind regards

Nico

From: Anonymous at: 2006-01-27 11:42:06

Instead of dummy driver you could just use the IP alias feature (eth0:1)! correct?