Paravirtualization With Xen 4.0 On Debian Squeeze (AMD64)

Version 1.0
Author: Falko Timme
Follow me on Twitter

This tutorial provides step-by-step instructions on how to install Xen 4.0 on a Debian Squeeze (6.0) system (AMD64) and create paravirtualized guests (don't mix this up with fully virtualized guests, i.e. hardware virtualization (HVM)).

Xen lets you create guest operating systems (*nix operating systems like Linux and FreeBSD), so called "virtual machines" or domUs, under a host operating system (dom0). Using Xen you can separate your applications into different virtual machines that are totally independent from each other (e.g. a virtual machine for a mail server, a virtual machine for a high-traffic web site, another virtual machine that serves your customers' web sites, a virtual machine for DNS, etc.), but still use the same hardware. This saves money, and what is even more important, it's more secure. If the virtual machine of your DNS server gets hacked, it has no effect on your other virtual machines. Plus, you can move virtual machines from one Xen server to the next one.

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

I'm using a Debian Squeeze system (x86_64) with the hostname server1.example.com and the IP address 192.168.0.100 as the host system (dom0). (The setup might differ slightly if you are on an i386 system.) I will use Debian Squeeze for the virtual machines (domU) as well (but also show the changes you need if you want to install an Ubuntu Maverick guest).

This guide will explain how to set up image-based virtual machines and also LVM-based virtual machines.

 

2 Installing Xen

To install Xen, we simply run

apt-get install xen-hypervisor xen-linux-system xen-utils xenstore-utils xenwatch xen-tools

Afterwards we open /etc/modules and make sure that we have the line loop max_loop=64 in it (this step is needed only if you want to create image-based virtual machines - you can skip it if you want to create LVM-based virtual machines):

vi /etc/modules
[...]
loop max_loop=64

Next we open /etc/xen/xend-config.sxp...

vi /etc/xen/xend-config.sxp

... and uncomment/add the line (network-script 'network-bridge antispoof=yes') and comment out all other (network-script ...) lines (antispoof=yes enables the Xen firewall which prevents that a VM can use an IP address that it is not allowed to use, like, for example, your gateway IP). Also make sure that the line (vif-script vif-bridge) is enabled:

[...]
(network-script 'network-bridge antispoof=yes')
[...]
(vif-script vif-bridge)
[...]

Then reboot the system:

reboot

Run

uname -r

and your new Xen kernel should show up:

root@server1:~# uname -r
2.6.32-5-xen-amd64
root@server1:~#

Now run

xm dmesg 

If you get output like this...

root@server1:~# xm dmesg
WARNING! Can't find hypervisor information in sysfs!
Error: Unable to connect to xend: No such file or directory. Is xend running?
root@server1:~#

... this means you are just running on a Xen-ready kernel, but not the Xen hypervisor itself (which means you will not be able to boot virtual machines). To change this, change the boot order of your kernels as follows and update the GRUB bootloader:

mv /etc/grub.d/10_linux /etc/grub.d/50_linux
update-grub2

This makes sure that the next time you boot, the Xen hypervisor gets loaded.

Reboot again:

reboot

Afterwards,

xm dmesg

should show you that the hypervisor is running:

root@server1:~# xm dmesg
(XEN) Xen version 4.0.1 (Debian 4.0.1-2) ([email protected]) (gcc version 4.4.5 (Debian 4.4.5-10) ) Wed Jan 12 14:04:06 UTC 2011
(XEN) Bootloader: GRUB 1.98+20100804-14
(XEN) Command line: placeholder
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000affa0000 (usable)
(XEN)  00000000affa0000 - 00000000affae000 (ACPI data)
(XEN)  00000000affae000 - 00000000afff0000 (ACPI NVS)
(XEN)  00000000afff0000 - 00000000b0000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fee00000 - 00000000fef00000 (reserved)
(XEN)  00000000ff700000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000140000000 (usable)
(XEN) ACPI: RSDP 000FA310, 0014 (r0 ACPIAM)
(XEN) ACPI: RSDT AFFA0000, 0044 (r1 052608 RSDT1415 20080526 MSFT       97)
(XEN) ACPI: FACP AFFA0200, 0084 (r1 052608 FACP1415 20080526 MSFT       97)
(XEN) ACPI: DSDT AFFA0450, 960C (r1  1AAAA 1AAAA000        0 INTL 20051117)
(XEN) ACPI: FACS AFFAE000, 0040
(XEN) ACPI: APIC AFFA0390, 0080 (r1 052608 APIC1415 20080526 MSFT       97)
(XEN) ACPI: MCFG AFFA0410, 003C (r1 052608 OEMMCFG  20080526 MSFT       97)
(XEN) ACPI: OEMB AFFAE040, 0073 (r1 052608 OEMB1415 20080526 MSFT       97)
(XEN) ACPI: HPET AFFA9A60, 0038 (r1 052608 OEMHPET0 20080526 MSFT       97)
(XEN) ACPI: INFO AFFAE0C0, 0124 (r1 052608 AMDINFO  20080526 MSFT       97)
(XEN) ACPI: NVHD AFFAE1F0, 0284 (r1 052608  NVHDCP  20080526 MSFT       97)
(XEN) ACPI: SSDT AFFA9AA0, 028A (r1 A M I  POWERNOW        1 AMD         1)
(XEN) System RAM: 3839MB (3931388kB)
(XEN) Domain heap initialised
(XEN) Processor #0 15:11 APIC version 16
(XEN) Processor #1 15:11 APIC version 16
(XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2600.088 MHz processor.
(XEN) Initing memory sharing.
(XEN) HVM: ASIDs enabled.
(XEN) HVM: SVM enabled
(XEN) AMD-Vi: IOMMU not found!
(XEN) I/O virtualisation disabled
(XEN) AMD: Disabling C1 Clock Ramping Node #0
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) checking TSC synchronization across 2 CPUs:
(XEN) CPU#0 had -2 usecs TSC skew, fixed it up.
(XEN) CPU#1 had 2 usecs TSC skew, fixed it up.
(XEN) Platform timer is 25.000MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 2 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x16ba000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000138000000->000000013c000000 (921455 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff816ba000
(XEN)  Init. ramdisk: ffffffff816ba000->ffffffff8323b600
(XEN)  Phys-Mach map: ffffffff8323c000->ffffffff83963b78
(XEN)  Start info:    ffffffff83964000->ffffffff839644b4
(XEN)  Page tables:   ffffffff83965000->ffffffff83986000
(XEN)  Boot stack:    ffffffff83986000->ffffffff83987000
(XEN)  TOTAL:         ffffffff80000000->ffffffff83c00000
(XEN)  ENTRY ADDRESS: ffffffff81509200
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 176kB init memory.
(XEN) MCE: MSR 417 is not MCA MSR
(XEN) MCE: MSR 417 is not MCA MSR
root@server1:~#

 

3 Creating Image-Based Virtual Machines

We will use xen-tools to create virtual machines. xen-tools make it very easy to create virtual machines - please read this tutorial to learn more: https://www.howtoforge.com/xen_tools_xen_shell_argo. We've already installed xen-tools in the previous step (chapter 2).

Now we edit /etc/xen-tools/xen-tools.conf. This file contains the default values that are used by the xen-create-image script unless you specify other values on the command line. I changed the following values and left the rest untouched:

vi /etc/xen-tools/xen-tools.conf
[...]
dir = /home/xen
[...]
# lvm = vg0
[...]
dist   = `xt-guess-suite-and-mirror --suite` # Default distribution to install.
[...]
gateway    = 192.168.0.1
netmask    = 255.255.255.0
broadcast  = 192.168.0.255
[...]
passwd = 1
[...]
kernel = /boot/vmlinuz-`uname -r`
initrd = /boot/initrd.img-`uname -r`
[...]
mirror = `xt-guess-suite-and-mirror --mirror`
[...]
serial_device = hvc0 #default
[...]
disk_device = xvda #default
[...]

The dir line specifies where the virtual machine images will be stored. Make sure that at the same time, lvm is commented out (these settings are mutually exclusive!).

dist specifies the distribution to be installed in the virtual machines (if you run the command

xt-guess-suite-and-mirror --suite

on the command line, you will see that it translates to squeeze, so in this case Debian Squeeze would be installed unless you specify anything else on the command line).

To find out which distributions you can install in a virtual machine, run:

gunzip /usr/share/doc/xen-tools/README.gz
cat /usr/share/doc/xen-tools/README

The passwd = 1 line makes that you can specify a root password when you create a new guest domain.

The kernel and initrd lines specify the kernel and ramdisk that get installed in a virtual machine. If you want to install Debian Squeeze, please use the settings as shown above (otherwise the Debian Squeeze guest might not boot). If you want to install Ubuntu Maverick, for example, you can use the same settings, or you comment out both lines, in which case the default Ubuntu kernel would be used (yes, Xen 4.0 allows you to use non-Xen kernels in a guest!).

Make sure you specify a gateway, netmask, and broadcast address. If you don't, and you don't specify a gateway and netmask on the command line when using xen-create-image, your guest domains won't have networking even if you specified an IP address!

The mirror line specifies the mirror to use (the command

xt-guess-suite-and-mirror --mirror

translates to a Debian mirror by default. Of course, you can specify another mirror, e.g. as follows:

mirror = http://ftp.de.debian.org/debian/

or

mirror = http://archive.ubuntu.com/ubuntu (for Ubuntu; in this case make sure that you specify an Ubuntu version in the dist line, e.g. dist = maverick)

)

It is very important that you add the line serial_device = hvc0 because otherwise your virtual machines might not boot properly!

disk_device = xvda makes that the virtualized disks are named xvda1, xvda2, etc. This is the correct setting for a Debian Squeeze guest; Ubuntu guests, especially when using a non-Xen kernel, might not boot with this setting, but you can override it with the --scsi switch that you pass to the xen-create-image command - in this case the virtualized disks will be named sda1, sda2, etc.

To summarize, the above settings are perfect for Debian Squeeze guests; if you want to install Ubuntu Maverick instead, you will have to override some of the settings in /etc/xen-tools/xen-tools.conf (--dist, --mirror, --scsi switches; maybe also comment out kernel and initrd) - I'll come to that in a moment.

Before we go on, we must create the directory where the virtual machine images should be stored:

mkdir /home/xen 

Now let's create our first guest domain, xen1.example.com, with the IP address 192.168.0.101:

xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev

Options that you specify on the command line override the settings in /etc/xen-tools/xen-tools.conf. Options that are not specified on the command line are taken from /etc/xen-tools/xen-tools.conf. Please make sure that you add --role=udev, or your virtual machine might not boot properly!

(To learn more about the available options, take a look at the xen-create-image man page:

man xen-create-image

)

The xen-create-image command will now create the xen1.example.com virtual machine for us. This can take a few minutes. The output should be similar to this one:

root@server1:~# xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev

General Information
--------------------
Hostname       :  xen1.example.com
Distribution   :  squeeze
Mirror         :  http://ftp.de.debian.org/debian/
Partitions     :  swap            256Mb (swap)
                  /               4Gb   (ext3)
Image type     :  sparse
Memory size    :  256Mb
Kernel path    :  /boot/vmlinuz-2.6.32-5-xen-amd64
Initrd path    :  /boot/initrd.img-2.6.32-5-xen-amd64

Networking Information
----------------------
IP Address 1   : 192.168.0.101 [MAC: 00:16:3E:E8:61:97]
Netmask        : 255.255.255.0
Broadcast      : 192.168.0.255
Gateway        : 192.168.0.1


Creating partition image: /home/xen/domains/xen1.example.com/swap.img
Done

Creating swap on /home/xen/domains/xen1.example.com/swap.img
Done

Creating partition image: /home/xen/domains/xen1.example.com/disk.img
Done

Creating ext3 filesystem on /home/xen/domains/xen1.example.com/disk.img
Done
Installation method: debootstrap
Done

Running hooks
Done

Role: udev
        File: /etc/xen-tools/role.d/udev
Role script completed.

Creating Xen configuration file
Done
Setting up root password
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
All done


Logfile produced at:
         /var/log/xen-tools/xen1.example.com.log

Installation Summary
---------------------
Hostname        :  xen1.example.com
Distribution    :  squeeze
IP-Address(es)  :  192.168.0.101
RSA Fingerprint :  81:d1:7a:44:3b:07:41:3a:0f:46:d5:d5:30:bc:26:43
Root Password   :  N/A

root@server1:~#

For an Ubuntu Maverick guest, you should use this command instead:

xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev --scsi --dist maverick --mirror=http://archive.ubuntu.com/ubuntu 

(To use the default Ubuntu kernel instead of Debian's Xen kernel in the guest, you can also comment out the kernel and initrd lines in /etc/xen-tools/xen-tools.conf.)

There should now be a xen1.example.com configuration file - /etc/xen/xen1.example.com.cfg. Take a look at it to become familiar with virtual machines configuration files:

cat /etc/xen/xen1.example.com.cfg
#
# Configuration file for the Xen instance xen1.example.com, created
# by xen-tools 4.2 on Fri Mar 25 15:56:51 2011.
#

#
#  Kernel + memory size
#
kernel      = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk     = '/boot/initrd.img-2.6.32-5-xen-amd64'

vcpus       = '1'
memory      = '256'

#
#  Disk device(s).
#
root        = '/dev/xvda2 ro'
disk        = [
                  'file:/home/xen/domains/xen1.example.com/disk.img,xvda2,w',
                  'file:/home/xen/domains/xen1.example.com/swap.img,xvda1,w',
              ]


#
#  Physical volumes
#


#
#  Hostname
#
name        = 'xen1.example.com'

#
#  Networking
#
vif         = [ 'ip=192.168.0.101,mac=00:16:3E:E8:61:97' ]

#
#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

(Please note: if you have a dual-core or quad-core CPU and want the virtual machine to use all CPU cores, please change the vcpus line to vcpus = '2' or vcpus = '4'.)

To start the virtual machine, run

xm create /etc/xen/xen1.example.com.cfg

root@server1:~# xm create /etc/xen/xen1.example.com.cfg
Using config file "/etc/xen/xen1.example.com.cfg".
Started domain xen1.example.com (id=1)
root@server1:~#

Run

xm console xen1.example.com

to log in on that virtual machine (type CTRL+] if you are at the console, or CTRL+5 if you're using PuTTY to go back to dom0), or use an SSH client to connect to it (192.168.0.101).

To get a list of running virtual machines, type

xm list 

The output should look like this:

root@server1:~# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2811     2     r-----    137.2
xen1.example.com                             1   256     1     -b----      1.7
root@server1:~#

To shut down xen1.example.com, do this:

xm shutdown xen1.example.com 

If you want xen1.example.com to start automatically at the next boot of the system, then do this:

mkdir /etc/xen/auto
ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto

Here are the most important Xen commands:

xm create -c /path/to/config - Start a virtual machine.
xm shutdown <name> - Stop a virtual machine.
xm destroy <name> - Stop a virtual machine immediately without shutting it down. It's as if you switch off the power button.
xm list - List all running systems.
xm console <name> - Log in on a virtual machine.
xm help - List of all commands.

A list of all virtual machines that were created with the xen-create-image command is available under

xen-list-images

root@server1:~# xen-list-images
Name: xen1.example.com
Memory: 256
IP: 192.168.0.101
root@server1:~#

To learn more about what you can do with xen-tools, take a look at this tutorial: https://www.howtoforge.com/xen_tools_xen_shell_argo

Share this page:

11 Comment(s)