How To Set Up Xen 4.3 On Debian Wheezy (7.0.2) And Then Upgrade To Jessie

Want to support HowtoForge? Become a subscriber!
 
Submitted by petero (Contact Author) (Forums) on Thu, 2013-12-19 14:35. :: Xen

How To Set Up Xen 4.3 On Debian Wheezy (7.0.2) And Then Upgrade To Jessie

Version 1.0
Author: Peter Okupski <okupski [at] widzew [dot] net>
Last edited 10.12.2013

This will be a quick and easy setup of XEN(dom0-hypervisor) and one virtual system (domU-guest). I wanted to test out XEN as my second experience after VMware and since many web pages are outdated and have many old fixed bugs and errors. I have decided to give it a spin with XEN.

Important note:

Who will benefit from this tutorial: Only beginners who would like to test simple virtual machines without any real Cloud appliance like XenOrchestra, Cloudstack, Opencloud and anything that uses XAPI for Xen Cloud Platform (XCP).I hope this will save time for anyone who need a real cloud appliance with at least web gui and few functions for storage/memory/CPU allocation, which at the moment can't be applied to simple Xen Xypervisor.(or its too much work:)

Who it isn't for: For any users/company that needs any HighAvability and real Cloud appliance

I encourage all of you Xen newbies to watch this video to get the feel and understanding of Xen.

Tutorial will be divided into following sections:
  1. Debian instalation with fast and simple LVM setup
  2. Upgrade to Jessie, so we will up to date with last XEN fixes
  3. XEN 4.3 installation
  4. XEN few tuning tips
  5. XEN simple bridge setup
  6. XEN setup to use templates to create hosts
  7. Creating first domU Guest system with oneliner;)


1. Simple Debian installation with LVM setup

I suggest to use unofficial(but Debians) ISOs which already have non-free firmware needed for Broadcom network cards and others, that might broke this nice and easy process.

Debian Unoffcial iso's

After initial setup of debian installer please choose custom partition setup as here below: I created the following partitions:

Mount point Size Comment
/boot 200 MB (Primary) (Location for the new partition: Beginning) (ext3) (Bootable flag: on <-- important, otherwise your system will not boot!)
swap 1GB (Logical) (Location for the new partition: Beginning) ( dom0 mem will be cut to 512M)
/ 6GB (Logical) (Location for the new partition: Beginning) (ext3)

We can set rest of the disk as LVM.We need to accept changes and write them to disk.

Your instalation should finish, and you need to write grub into your disk. After that installation will reboot.

After installing your partitions should look like this:

root@xen-dom0:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda6 6.0G 2.0G 2.4G 30% /
udev 10M 0 10M 0% /dev
tmpfs 38M 268K 38M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 266M 0 266M 0% /run/shm
/dev/sda1 180M 28M 140M 17% /boot

Now we will install lvm, and create first physical volume.

apt-get install lvm2

pvcreate /dev/sda7

  Physical volume "/dev/sda7" successfully created
  root@xen-dom0:/etc/xen-tools# pvscan
  PV /dev/sda7                      lvm2 [273.51 GiB]
  Total: 1 [273.51 GiB] / in use: 0 [0   ] / in no VG: 1 [273.51 GiB]
  root@xen-dom0:/etc/xen-tools# pvdisplay 
  "/dev/sda7" is a new physical volume of "273.51 GiB"
  --- NEW Physical volume ---
  PV Name               /dev/sda7
  VG Name               
  PV Size               273.51 GiB
  Allocatable           NO
  PE Size               0   
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               1vKp2c-KbmM-w8Ql-noOe-U00J-hlr0-FsKwOM

It's time to create volume group (vg0)(notice the change of letters p/v! before create)

vgcreate vg0 /dev/sda7

Volume group "vg0" successfully created

Lets look it up, if all went well.

vgdisplay

--- Volume group ---
  VG Name               vg0
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               273.51 GiB
  PE Size               4.00 MiB
  Total PE              70018
  Alloc PE / Size       0 / 0   
  Free  PE / Size       70018 / 273.51 GiB
  VG UUID               2a3kn9-DvVA-3Qac-g4Wz-f2Xo-s5hk-Yh5fBc

After setting LVM , we can move to cleanup our dom0 with those unnecessary packages:

apt-get remove bsd-mailx exim4 exim4-base exim4-daemon-light nfs-common

And then we can install few we need/might need:

apt-get install mc screen ssh debootstrap python iproute bridge-utils libcurl3-dev

After install pure Debian 7.0.2 takes about 760MB.

rootfs                                                  4.6G  764M  3.7G  18% /

 

2. Upgrade to Debian Jessie

Now we can do an upgrade to testing (Jessie) since we need XEN to be as fresh as possible

Use this generator - it will save you time:) and few errors. Or just change "wheezy" to " jessie" , and "stable" to "testing" in /etc/apt/sources.list.

/etc/apt/sources.list

deb http://ftp.pl.debian.org/debian testing main contrib non-free
deb-src http://ftp.pl.debian.org/debian testing main contrib non-free
deb http://ftp.debian.org/debian/ jessie-updates main contrib non-free
deb-src http://ftp.debian.org/debian/ jessie-updates main contrib non-free
deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free

First:

apt-get update; apt-get -y upgrade

During update you will be asked to restart services - choose Yes. Next we will download all .deb we need(just for safety reasons):

apt-get -y --download-only dist-upgrade

Then we can proceed to real update.It's best to reboot after this, so the kernel will change from 3.2 to 3.11.

apt-get -y dist-upgrade

Tidy up:

apt-get clean; apt-get autoclean;apt-get autoremove

 

3. XEN 4.3 Installation

First we should check if our CPU supports hardware virtualization

egrep '(vmx|svm)' --color=always /proc/cpuinfo

The output should look like this

root@xxxx:~# egrep '(vmx|svm)' --color=always /proc/cpuinfo
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm tpr_shadow vnmi flexpriority
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm tpr_shadow vnmi flexpriority

Then we can procees for installing XEN : the hypervisor, xen aware kernel and xen tools. This can be done by a metapackage:

apt-get install xen-linux-system xen-tools

Debian Wheezy uses Grub 2 and as default it lists normal kernels first, and only then lists the Xen hypervisor and its kernels.You can change this to cause Grub 2 to prefer to boot Xen, by changing the priority of Grub's Xen configuration script (20_linux_xen) to be higher than the standard Linux config (10_linux). This is most easily done using dpkg-divert:

dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen

After any update to the Grub configuration you must apply the configuration by running:

update-grub

 

4. Xen tunning

  1. Set Dom0 memory to 512Mb and disable Ballooning so DomUs will never took Dom0's memory
  2. Prevent virtual machines on logical volumes from appearing in the boot menu
  3. Disable Xendomains save & restore
  4. Set only 1st CPU for dom0

 

1. Set Dom0 memory to 512Mb and disable balooning

/etc/xen/xend-config.sxp

(dom0-min-mem 512)
(enable-dom0-ballooning no)

 

2. Prevent virtual machines on logical volumes from appearing in the boot menu

Modify grub:

/etc/default/grub

#allocate memory to dom0
GRUB_CMDLINE_XEN="dom0_mem=512M"
# Disable OS prober
GRUB_DISABLE_OS_PROBER=true

update-grub;reboot

 

3. Disable Xendomains save & restore

/etc/default/xendomains

#XENDOMAINS_SAVE=/var/lib/xen/save
XENDOMAINS_SAVE=                
 
#XENDOMAINS_RESTORE=true
XENDOMAINS_RESTORE=false

 

4. Limit dom0 to 1st CPU only

By default all CPUs are shared among dom0 and all domU (guests). It may broke dom0 responsibility if guests consume too much CPU time. To avoid this, it is possible to grant one (or more) processor core to dom0 and also pin it to dom0.

Add following options to /etc/default/grub to allocate one cpu core to dom0:

/etc/default/grub

# Xen boot parameters for all Xen boots
GRUB_CMDLINE_XEN="dom0_mem=512M dom0_max_vcpus=1 dom0_vcpus_pin"
# Disable OS prober
GRUB_DISABLE_OS_PROBER=true

Remeber to run update-grub after those!

update-grub

Also make such changes in /etc/xen/xend-config.sxp:

/etc/xen/xend-config.sxp

(dom0-cpus 1)

!! Before reboot check if you did update-grub and then reboot your server !!

 

5. XEN simple bridge setup

Most setups will use BRIDGEed configuration, and that is enough if you are planing for simple environment, but I highly encourage to use OpenVSwitch if you require VLANs or any ACL etc.

Here is the easy version for just a simple bridge:

apt-get install bridge-utils

We need to edit: /etc/network/interfaces

/etc/network/interfaces

 ##The loopback network interface 
auto lo iface lo inet loopback
iface eth0 inet manual
auto xenbr0
iface xenbr0 inet static address 10.0.9.100 netmask 255.255.255.0 network 10.0.9.0 broadcast 10.0.9.255 gateway 10.0.9.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 10.0.9.2 dns-search yourdomain.net bridge_ports eth0
#other possibly useful options in a virtualized environment
bridge_stp off # disable Spanning Tree Protocol bridge_waitport 0 # no delay before a port becomes available bridge_fd 0 # no forwarding delay
After that you should restart networking with

/etc/init.d/networking restart

And check if your bridge has come up

brctl show

bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.001a6479cd8c       no              eth0

 

6. XEN-Tools setup to use templates for your virtual machines (guests-domU)

Now lets create our first image of new domU with a template. We need to edit the file:

/etc/xen-tools/xen-tools.conf

lvm = vg0 # your volume group which we created before
install-method = debootstrap # This will launch debian installer
size   = 10Gb      # Disk image size.
memory = 512Mb    # Memory size
swap   = 2Gb    # Swap size
noswap = 1      # Don't use swap at all for the new system.
fs     = ext4     # use the EXT3 filesystem for the disk image.
dist   = `xt-guess-suite-and-mirror --suite` # Default distribution to install.
image  = sparse   # Specify sparse vs. full disk images.
gateway    = 10.0.9.1 # Your gateway, that dom0 is also using with bridge
netmask    = 255.255.255.0 # no comments needed;)
broadcast  = 10.0.9.255 # Your broadcast 
bridge = xenbr0 # Here we should use the bridge we created, xen will make it into eth0 on domU.
cachedir = /var/cache/xen-tools/archives/ # Place for *.debs for further use
passwd = 1 #This will ask for a root password to create on new machine
kernel = /boot/vmlinuz-`uname -r` #It will use same kernel as dom0 is using( this is crucial, to avoid problems)
initrd = /boot/initrd.img-`uname -r` #Same here for initrd
mirror = `xt-guess-suite-and-mirror --mirror` 
mirror = ftp://ftp.pl.debian.org/debian/ #Enter you mirror of debian
ext3_options     = noatime,nodiratime,errors=remount-ro #Simple options to apply for new machine
ext2_options     = noatime,nodiratime,errors=remount-ro
xfs_options      = defaults
reiserfs_options = defaults
btrfs_options    = defaults
boot = 1 # Do we want to boot our new machine after creating it. Hell yeah!

 

7. Creating first domU Guest

Now we can create our first domU host:

xen-create-image --hostname domU-test --dist=jessie --vcpus=4 --ip=10.0.9.101

This will take about 3-4 minutes , depending on speed of your connection , CPU and disk speed. Once is done we can look it up, if its working.

xm list

Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   512     1     r-----     93.8
domU-test                                    1   512     4     -b----     23.9

I recommend to read xen-create-image --help as it will give you an idea what you can change providing it with options to create any distribution.Example: run commands after and all other stuff you might need. There is also an option to generate an image and use it for ready to launch system-template, without having to run installer for 3 minutes every time.

Now you can log onto you new host with ssh.

Below you can find links I used in this tutorial:

  1. Xen in Debian Wiki
  2. OpenVSwitch
  3. Xen 4 in Debian Wheezy
  4. Tuning XEN - XEN wiki
  5. Apt Sources Generator
  6. Xen Best Practises
  7. How to use LVM

Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.