Paravirtualization With Xen 4.0 On Debian Squeeze (AMD64)

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Tue, 2011-03-29 16:14. :: Debian | Xen | Virtualization

Paravirtualization With Xen 4.0 On Debian Squeeze (AMD64)

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Follow me on Twitter
Last edited 03/25/2011

This tutorial provides step-by-step instructions on how to install Xen 4.0 on a Debian Squeeze (6.0) system (AMD64) and create paravirtualized guests (don't mix this up with fully virtualized guests, i.e. hardware virtualization (HVM)).

Xen lets you create guest operating systems (*nix operating systems like Linux and FreeBSD), so called "virtual machines" or domUs, under a host operating system (dom0). Using Xen you can separate your applications into different virtual machines that are totally independent from each other (e.g. a virtual machine for a mail server, a virtual machine for a high-traffic web site, another virtual machine that serves your customers' web sites, a virtual machine for DNS, etc.), but still use the same hardware. This saves money, and what is even more important, it's more secure. If the virtual machine of your DNS server gets hacked, it has no effect on your other virtual machines. Plus, you can move virtual machines from one Xen server to the next one.

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

I'm using a Debian Squeeze system (x86_64) with the hostname server1.example.com and the IP address 192.168.0.100 as the host system (dom0). (The setup might differ slightly if you are on an i386 system.) I will use Debian Squeeze for the virtual machines (domU) as well (but also show the changes you need if you want to install an Ubuntu Maverick guest).

This guide will explain how to set up image-based virtual machines and also LVM-based virtual machines.

 

2 Installing Xen

To install Xen, we simply run

apt-get install xen-hypervisor xen-linux-system xen-utils xenstore-utils xenwatch xen-tools

Afterwards we open /etc/modules and make sure that we have the line loop max_loop=64 in it (this step is needed only if you want to create image-based virtual machines - you can skip it if you want to create LVM-based virtual machines):

vi /etc/modules

[...]
loop max_loop=64

Next we open /etc/xen/xend-config.sxp...

vi /etc/xen/xend-config.sxp

... and uncomment/add the line (network-script 'network-bridge antispoof=yes') and comment out all other (network-script ...) lines (antispoof=yes enables the Xen firewall which prevents that a VM can use an IP address that it is not allowed to use, like, for example, your gateway IP). Also make sure that the line (vif-script vif-bridge) is enabled:

[...]
(network-script 'network-bridge antispoof=yes')
[...]
(vif-script vif-bridge)
[...]

Then reboot the system:

reboot

Run

uname -r

and your new Xen kernel should show up:

root@server1:~# uname -r
2.6.32-5-xen-amd64
root@server1:~#

Now run

xm dmesg

If you get output like this...

root@server1:~# xm dmesg
WARNING! Can't find hypervisor information in sysfs!
Error: Unable to connect to xend: No such file or directory. Is xend running?
root@server1:~#

... this means you are just running on a Xen-ready kernel, but not the Xen hypervisor itself (which means you will not be able to boot virtual machines). To change this, change the boot order of your kernels as follows and update the GRUB bootloader:

mv /etc/grub.d/10_linux /etc/grub.d/50_linux
update-grub2

This makes sure that the next time you boot, the Xen hypervisor gets loaded.

Reboot again:

reboot

Afterwards,

xm dmesg

should show you that the hypervisor is running:

root@server1:~# xm dmesg
(XEN) Xen version 4.0.1 (Debian 4.0.1-2) (waldi@debian.org) (gcc version 4.4.5 (Debian 4.4.5-10) ) Wed Jan 12 14:04:06 UTC 2011
(XEN) Bootloader: GRUB 1.98+20100804-14
(XEN) Command line: placeholder
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000affa0000 (usable)
(XEN)  00000000affa0000 - 00000000affae000 (ACPI data)
(XEN)  00000000affae000 - 00000000afff0000 (ACPI NVS)
(XEN)  00000000afff0000 - 00000000b0000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fee00000 - 00000000fef00000 (reserved)
(XEN)  00000000ff700000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000140000000 (usable)
(XEN) ACPI: RSDP 000FA310, 0014 (r0 ACPIAM)
(XEN) ACPI: RSDT AFFA0000, 0044 (r1 052608 RSDT1415 20080526 MSFT       97)
(XEN) ACPI: FACP AFFA0200, 0084 (r1 052608 FACP1415 20080526 MSFT       97)
(XEN) ACPI: DSDT AFFA0450, 960C (r1  1AAAA 1AAAA000        0 INTL 20051117)
(XEN) ACPI: FACS AFFAE000, 0040
(XEN) ACPI: APIC AFFA0390, 0080 (r1 052608 APIC1415 20080526 MSFT       97)
(XEN) ACPI: MCFG AFFA0410, 003C (r1 052608 OEMMCFG  20080526 MSFT       97)
(XEN) ACPI: OEMB AFFAE040, 0073 (r1 052608 OEMB1415 20080526 MSFT       97)
(XEN) ACPI: HPET AFFA9A60, 0038 (r1 052608 OEMHPET0 20080526 MSFT       97)
(XEN) ACPI: INFO AFFAE0C0, 0124 (r1 052608 AMDINFO  20080526 MSFT       97)
(XEN) ACPI: NVHD AFFAE1F0, 0284 (r1 052608  NVHDCP  20080526 MSFT       97)
(XEN) ACPI: SSDT AFFA9AA0, 028A (r1 A M I  POWERNOW        1 AMD         1)
(XEN) System RAM: 3839MB (3931388kB)
(XEN) Domain heap initialised
(XEN) Processor #0 15:11 APIC version 16
(XEN) Processor #1 15:11 APIC version 16
(XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2600.088 MHz processor.
(XEN) Initing memory sharing.
(XEN) HVM: ASIDs enabled.
(XEN) HVM: SVM enabled
(XEN) AMD-Vi: IOMMU not found!
(XEN) I/O virtualisation disabled
(XEN) AMD: Disabling C1 Clock Ramping Node #0
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) checking TSC synchronization across 2 CPUs:
(XEN) CPU#0 had -2 usecs TSC skew, fixed it up.
(XEN) CPU#1 had 2 usecs TSC skew, fixed it up.
(XEN) Platform timer is 25.000MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 2 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x16ba000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000138000000->000000013c000000 (921455 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff816ba000
(XEN)  Init. ramdisk: ffffffff816ba000->ffffffff8323b600
(XEN)  Phys-Mach map: ffffffff8323c000->ffffffff83963b78
(XEN)  Start info:    ffffffff83964000->ffffffff839644b4
(XEN)  Page tables:   ffffffff83965000->ffffffff83986000
(XEN)  Boot stack:    ffffffff83986000->ffffffff83987000
(XEN)  TOTAL:         ffffffff80000000->ffffffff83c00000
(XEN)  ENTRY ADDRESS: ffffffff81509200
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 176kB init memory.
(XEN) MCE: MSR 417 is not MCA MSR
(XEN) MCE: MSR 417 is not MCA MSR
root@server1:~#

 

3 Creating Image-Based Virtual Machines

We will use xen-tools to create virtual machines. xen-tools make it very easy to create virtual machines - please read this tutorial to learn more: http://www.howtoforge.com/xen_tools_xen_shell_argo. We've already installed xen-tools in the previous step (chapter 2).

Now we edit /etc/xen-tools/xen-tools.conf. This file contains the default values that are used by the xen-create-image script unless you specify other values on the command line. I changed the following values and left the rest untouched:

vi /etc/xen-tools/xen-tools.conf

[...]
dir = /home/xen
[...]
# lvm = vg0
[...]
dist   = `xt-guess-suite-and-mirror --suite` # Default distribution to install.
[...]
gateway    = 192.168.0.1
netmask    = 255.255.255.0
broadcast  = 192.168.0.255
[...]
passwd = 1
[...]
kernel = /boot/vmlinuz-`uname -r`
initrd = /boot/initrd.img-`uname -r`
[...]
mirror = `xt-guess-suite-and-mirror --mirror`
[...]
serial_device = hvc0 #default
[...]
disk_device = xvda #default
[...]

The dir line specifies where the virtual machine images will be stored. Make sure that at the same time, lvm is commented out (these settings are mutually exclusive!).

dist specifies the distribution to be installed in the virtual machines (if you run the command

xt-guess-suite-and-mirror --suite

on the command line, you will see that it translates to squeeze, so in this case Debian Squeeze would be installed unless you specify anything else on the command line).

To find out which distributions you can install in a virtual machine, run:

gunzip /usr/share/doc/xen-tools/README.gz
cat /usr/share/doc/xen-tools/README

The passwd = 1 line makes that you can specify a root password when you create a new guest domain.

The kernel and initrd lines specify the kernel and ramdisk that get installed in a virtual machine. If you want to install Debian Squeeze, please use the settings as shown above (otherwise the Debian Squeeze guest might not boot). If you want to install Ubuntu Maverick, for example, you can use the same settings, or you comment out both lines, in which case the default Ubuntu kernel would be used (yes, Xen 4.0 allows you to use non-Xen kernels in a guest!).

Make sure you specify a gateway, netmask, and broadcast address. If you don't, and you don't specify a gateway and netmask on the command line when using xen-create-image, your guest domains won't have networking even if you specified an IP address!

The mirror line specifies the mirror to use (the command

xt-guess-suite-and-mirror --mirror

translates to a Debian mirror by default. Of course, you can specify another mirror, e.g. as follows:

mirror = http://ftp.de.debian.org/debian/

or

mirror = http://archive.ubuntu.com/ubuntu (for Ubuntu; in this case make sure that you specify an Ubuntu version in the dist line, e.g. dist = maverick)

)

It is very important that you add the line serial_device = hvc0 because otherwise your virtual machines might not boot properly!

disk_device = xvda makes that the virtualized disks are named xvda1, xvda2, etc. This is the correct setting for a Debian Squeeze guest; Ubuntu guests, especially when using a non-Xen kernel, might not boot with this setting, but you can override it with the --scsi switch that you pass to the xen-create-image command - in this case the virtualized disks will be named sda1, sda2, etc.

To summarize, the above settings are perfect for Debian Squeeze guests; if you want to install Ubuntu Maverick instead, you will have to override some of the settings in /etc/xen-tools/xen-tools.conf (--dist, --mirror, --scsi switches; maybe also comment out kernel and initrd) - I'll come to that in a moment.

Before we go on, we must create the directory where the virtual machine images should be stored:

mkdir /home/xen

Now let's create our first guest domain, xen1.example.com, with the IP address 192.168.0.101:

xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev

Options that you specify on the command line override the settings in /etc/xen-tools/xen-tools.conf. Options that are not specified on the command line are taken from /etc/xen-tools/xen-tools.conf. Please make sure that you add --role=udev, or your virtual machine might not boot properly!

(To learn more about the available options, take a look at the xen-create-image man page:

man xen-create-image

)

The xen-create-image command will now create the xen1.example.com virtual machine for us. This can take a few minutes. The output should be similar to this one:

root@server1:~# xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev

General Information
--------------------
Hostname       :  xen1.example.com
Distribution   :  squeeze
Mirror         :  http://ftp.de.debian.org/debian/
Partitions     :  swap            256Mb (swap)
                  /               4Gb   (ext3)
Image type     :  sparse
Memory size    :  256Mb
Kernel path    :  /boot/vmlinuz-2.6.32-5-xen-amd64
Initrd path    :  /boot/initrd.img-2.6.32-5-xen-amd64

Networking Information
----------------------
IP Address 1   : 192.168.0.101 [MAC: 00:16:3E:E8:61:97]
Netmask        : 255.255.255.0
Broadcast      : 192.168.0.255
Gateway        : 192.168.0.1


Creating partition image: /home/xen/domains/xen1.example.com/swap.img
Done

Creating swap on /home/xen/domains/xen1.example.com/swap.img
Done

Creating partition image: /home/xen/domains/xen1.example.com/disk.img
Done

Creating ext3 filesystem on /home/xen/domains/xen1.example.com/disk.img
Done
Installation method: debootstrap
Done

Running hooks
Done

Role: udev
        File: /etc/xen-tools/role.d/udev
Role script completed.

Creating Xen configuration file
Done
Setting up root password
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
All done


Logfile produced at:
         /var/log/xen-tools/xen1.example.com.log

Installation Summary
---------------------
Hostname        :  xen1.example.com
Distribution    :  squeeze
IP-Address(es)  :  192.168.0.101
RSA Fingerprint :  81:d1:7a:44:3b:07:41:3a:0f:46:d5:d5:30:bc:26:43
Root Password   :  N/A

root@server1:~#

For an Ubuntu Maverick guest, you should use this command instead:

xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev --scsi --dist maverick --mirror=http://archive.ubuntu.com/ubuntu

(To use the default Ubuntu kernel instead of Debian's Xen kernel in the guest, you can also comment out the kernel and initrd lines in /etc/xen-tools/xen-tools.conf.)

There should now be a xen1.example.com configuration file - /etc/xen/xen1.example.com.cfg. Take a look at it to become familiar with virtual machines configuration files:

cat /etc/xen/xen1.example.com.cfg

#
# Configuration file for the Xen instance xen1.example.com, created
# by xen-tools 4.2 on Fri Mar 25 15:56:51 2011.
#

#
#  Kernel + memory size
#
kernel      = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk     = '/boot/initrd.img-2.6.32-5-xen-amd64'

vcpus       = '1'
memory      = '256'

#
#  Disk device(s).
#
root        = '/dev/xvda2 ro'
disk        = [
                  'file:/home/xen/domains/xen1.example.com/disk.img,xvda2,w',
                  'file:/home/xen/domains/xen1.example.com/swap.img,xvda1,w',
              ]


#
#  Physical volumes
#


#
#  Hostname
#
name        = 'xen1.example.com'

#
#  Networking
#
vif         = [ 'ip=192.168.0.101,mac=00:16:3E:E8:61:97' ]

#
#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

(Please note: if you have a dual-core or quad-core CPU and want the virtual machine to use all CPU cores, please change the vcpus line to vcpus = '2' or vcpus = '4'.)

To start the virtual machine, run

xm create /etc/xen/xen1.example.com.cfg

root@server1:~# xm create /etc/xen/xen1.example.com.cfg
Using config file "/etc/xen/xen1.example.com.cfg".
Started domain xen1.example.com (id=1)
root@server1:~#

Run

xm console xen1.example.com

to log in on that virtual machine (type CTRL+] if you are at the console, or CTRL+5 if you're using PuTTY to go back to dom0), or use an SSH client to connect to it (192.168.0.101).

To get a list of running virtual machines, type

xm list

The output should look like this:

root@server1:~# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2811     2     r-----    137.2
xen1.example.com                             1   256     1     -b----      1.7
root@server1:~#

To shut down xen1.example.com, do this:

xm shutdown xen1.example.com

If you want xen1.example.com to start automatically at the next boot of the system, then do this:

mkdir /etc/xen/auto
ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto

Here are the most important Xen commands:

xm create -c /path/to/config - Start a virtual machine.
xm shutdown <name> - Stop a virtual machine.
xm destroy <name> - Stop a virtual machine immediately without shutting it down. It's as if you switch off the power button.
xm list - List all running systems.
xm console <name> - Log in on a virtual machine.
xm help - List of all commands.

A list of all virtual machines that were created with the xen-create-image command is available under

xen-list-images

root@server1:~# xen-list-images
Name: xen1.example.com
Memory: 256
IP: 192.168.0.101
root@server1:~#

To learn more about what you can do with xen-tools, take a look at this tutorial: http://www.howtoforge.com/xen_tools_xen_shell_argo


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Anonymous (not registered) on Thu, 2013-01-17 23:02.

Ganeti is a tool developped by google providing Xen with HA features like failover, live vm migration, secondary instance, ... Ganeti is quite tricky to install because of the amount of configuration needed to transform a standard debian squeeze to a ganeto node. The sitehttp://www.xenclustering.com provides some guidelines and a script for automated ganeti node initialization.

Submitted by MHASAN (not registered) on Wed, 2012-09-05 14:57.
Many many thanks for this tutorial. I done successfully. But when I've tried to intall Ubuntu 12 Server I got an error
I used following command arguments to install Ubuntu 12 Server 

xen-create-image --hostname=xen-ubuntu --size=32Gb --swap=4096Mb --ip=122.102.33.150 --memory=2048Mb --arch=amd64 --role=udev --scsi --dist=precise --mirror=http://mirrors.ispros.com.bd/ubuntu-release/

ERROR: 'dist' argument takes a distribution name (see /usr/lib/xen-tools for valid values).

How to add more distribution to my xen system ??
Submitted by Alee (not registered) on Sat, 2012-03-03 14:34.
Hi dude!!

xen-create-image --hostname=xen1.example.com --size=4Gb --swap=256Mb --ip=192.168.0.101 --memory=256Mb --arch=amd64 --role=udev --scsi --dist maverick --mirror=http://archive.ubuntu.com/ubuntuthis will install maverick but how can i install other distos like natty or oneiric? and is there any way to define path of iso stored on my hard drive to install OS?

Submitted by Alee (not registered) on Fri, 2012-03-02 14:21.
Its owesm..

I followed all the steps n successfully created vm.

At the end while asking about :

Creating Xen configuration file
Done
Setting up root password
Enter new UNIX password:

Retype new UNIX password:

passwd: Authentication token manipulation error

passwd: password unchanged

All done

 

i made a mistake over there my passwrd mismatch.

so when i start vm its asking about username and password.

so how i can log in to system? which  username and password i should provide?

 thx

 

Submitted by Umar Draz (not registered) on Thu, 2012-02-16 13:32.

HI

 is this possible to install lucid or hardy

 through PV. As you use maveric.

 Best Regards

Submitted by Netzmeister (registered user) on Sat, 2012-01-14 20:14.

Hi Falko
I followed your instruction on: http://www.howtoforge.com/paravirtualization-with-xen-4.0-on-debian-squeeze-amd64. I set up the guest as image. At the point to start the VM wit <font="Courier New">xm console</font> the VM boots up but hangs on: 
<code>Gave up waiting for root device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/xvda2 does not exist.  Dropping to a shell!
</code>
Here ist the content of the <font="Courier New">xen1.hostname.local.cfg</font>:
<code>kernel      = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk     = '/boot/initrd.img-2.6.32-5-xen-amd64'
vcpus       = '1'
memory      = '1024'
root        = '/dev/xvda2 ro'
disk        = [ 'file:/usr/local/xen/domains/lweb.eas-4.name/disk.img,xvda2,w', 'file:/usr/local/xen/domains/lweb.eas-4.name/swap.img,xvda1,w', ]
name        = 'xen1.hostname.local'
vif         = [ 'ip=192.168.0.1,mac=01:12:4E:DD:EE:86' ]
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'</code>
Any suggestions how to solve my problem. I browsed the web, but I did not find any solutions.
Thank you for your help.
Kind regards, Laszlo

Submitted by Norbul (not registered) on Mon, 2014-06-16 20:49.

IN /etc/xen-tools/xen-tools.conf


Check if: 

fs     = ext4 
Submitted by Anders (not registered) on Tue, 2011-08-02 02:43.
I do think that network-bridge and vif-bridge (actually all network-* and vif-*) are not supported by Debian Xen. You are supposed to set up the network in /etc/network/interfaces as usuall.
Submitted by Milleman (registered user) on Fri, 2011-04-01 18:33.

Great HowTo, Falko!

Worked fine for me!! Do you have an HowTo for the above configuration in order to install Windows as HVM?

Submitted by dtdeg (registered user) on Tue, 2011-10-11 14:03.

When using the xm create command i've had this error: Error: 'NoneType' object has no attribute 'rfind'

 using apt-get install xen-qemu-dm-4.0 solved this issue

Submitted by mario (not registered) on Tue, 2012-02-07 18:52.

root@ziomario:/home/mario# xm create /etc/xen/xen1.example.com.cfg

Using config file "/etc/xen/xen1.example.com.cfg".

Error: (1, 'Internal error', 'panic: xc_dom_boot.c:79: launch_vm: SETVCPUCONTEXT failed (rc=-1)')

 root@ziomario:/home/mario# nano /etc/xen/xen1.example.com.cfg

#bootloader = '/usr/lib/xen-default/bin/pygrub'
##  Disk device(s).#root        = '/dev/sda2 ro'disk        = [                  'file:/home/xen/domains/xen1.example.com/disk.img,sda2,w',                  'file:/home/xen/domains/xen1.example.com/swap.img,sda1,w',              ]##  Physical volumes#

#  Hostname
#
name        = 'xen1.example.com'

#  Networking
#
vif         = [ 'ip=192.168.0.101,mac=00' ]

#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'