How To Set Up Xen 4.3 On Debian Wheezy (7.0.2) And Then Upgrade To Jessie

Version 1.0
Author: Peter Okupski <okupski [at] widzew [dot] net>

This will be a quick and easy setup of XEN(dom0-hypervisor) and one virtual system (domU-guest). I wanted to test out XEN as my second experience after VMware and since many web pages are outdated and have many old fixed bugs and errors. I have decided to give it a spin with XEN.

Important note:

Who will benefit from this tutorial: Only beginners who would like to test simple virtual machines without any real Cloud appliance like XenOrchestra, Cloudstack, Opencloud and anything that uses XAPI for Xen Cloud Platform (XCP).I hope this will save time for anyone who need a real cloud appliance with at least web gui and few functions for storage/memory/CPU allocation, which at the moment can't be applied to simple Xen Xypervisor.(or its too much work:)

Who it isn't for: For any users/company that needs any HighAvability and real Cloud appliance

I encourage all of you Xen newbies to watch this video to get the feel and understanding of Xen.

Tutorial will be divided into following sections:

  1. Debian instalation with fast and simple LVM setup
  2. Upgrade to Jessie, so we will up to date with last XEN fixes
  3. XEN 4.3 installation
  4. XEN few tuning tips
  5. XEN simple bridge setup
  6. XEN setup to use templates to create hosts
  7. Creating first domU Guest system with oneliner;)

    1. Simple Debian installation with LVM setup

    I suggest to use unofficial(but Debians) ISOs which already have non-free firmware needed for Broadcom network cards and others, that might broke this nice and easy process.

    Debian Unoffcial iso's

    After initial setup of debian installer please choose custom partition setup as here below: I created the following partitions:

    Mount point Size Comment
    /boot 200 MB (Primary) (Location for the new partition: Beginning) (ext3) (Bootable flag: on <-- important, otherwise your system will not boot!)
    swap 1GB (Logical) (Location for the new partition: Beginning) ( dom0 mem will be cut to 512M)
    / 6GB (Logical) (Location for the new partition: Beginning) (ext3)

    We can set rest of the disk as LVM.We need to accept changes and write them to disk.

    Your instalation should finish, and you need to write grub into your disk. After that installation will reboot.

    After installing your partitions should look like this:

    root@xen-dom0:~# df -h 
    Filesystem Size Used Avail Use% Mounted on
    /dev/sda6 6.0G 2.0G 2.4G 30% /
    udev 10M 0 10M 0% /dev
    tmpfs 38M 268K 38M 1% /run
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 266M 0 266M 0% /run/shm
    /dev/sda1 180M 28M 140M 17% /boot

    Now we will install lvm, and create first physical volume.

    apt-get install lvm2
    pvcreate /dev/sda7
      Physical volume "/dev/sda7" successfully created
      root@xen-dom0:/etc/xen-tools# pvscan
      PV /dev/sda7                      lvm2 [273.51 GiB]
      Total: 1 [273.51 GiB] / in use: 0 [0   ] / in no VG: 1 [273.51 GiB]
      root@xen-dom0:/etc/xen-tools# pvdisplay 
      "/dev/sda7" is a new physical volume of "273.51 GiB"
      --- NEW Physical volume ---
      PV Name               /dev/sda7
      VG Name               
      PV Size               273.51 GiB
      Allocatable           NO
      PE Size               0   
      Total PE              0
      Free PE               0
      Allocated PE          0
      PV UUID               1vKp2c-KbmM-w8Ql-noOe-U00J-hlr0-FsKwOM

    It's time to create volume group (vg0)(notice the change of letters p/v! before create)

    vgcreate vg0 /dev/sda7
    Volume group "vg0" successfully created

    Lets look it up, if all went well.

    --- Volume group ---
      VG Name               vg0
      System ID             
      Format                lvm2
      Metadata Areas        1
      Metadata Sequence No  1
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                0
      Open LV               0
      Max PV                0
      Cur PV                1
      Act PV                1
      VG Size               273.51 GiB
      PE Size               4.00 MiB
      Total PE              70018
      Alloc PE / Size       0 / 0   
      Free  PE / Size       70018 / 273.51 GiB
      VG UUID               2a3kn9-DvVA-3Qac-g4Wz-f2Xo-s5hk-Yh5fBc

    After setting LVM , we can move to cleanup our dom0 with those unnecessary packages:

    apt-get remove bsd-mailx exim4 exim4-base exim4-daemon-light nfs-common

    And then we can install few we need/might need:

    apt-get install mc screen ssh debootstrap python iproute bridge-utils libcurl3-dev

    After install pure Debian 7.0.2 takes about 760MB.

    rootfs                                                  4.6G  764M  3.7G  18% /


    2. Upgrade to Debian Jessie

    Now we can do an upgrade to testing (Jessie) since we need XEN to be as fresh as possible

    Use this generator - it will save you time:) and few errors. Or just change "wheezy" to " jessie" , and "stable" to "testing" in /etc/apt/sources.list.


    deb testing main contrib non-free
    deb-src testing main contrib non-free
    deb jessie-updates main contrib non-free
    deb-src jessie-updates main contrib non-free
    deb jessie/updates main contrib non-free
    deb-src jessie/updates main contrib non-free


    apt-get update; apt-get -y upgrade

    During update you will be asked to restart services - choose Yes. Next we will download all .deb we need(just for safety reasons):

    apt-get -y --download-only dist-upgrade

    Then we can proceed to real update.It's best to reboot after this, so the kernel will change from 3.2 to 3.11.

    apt-get -y dist-upgrade

    Tidy up:

    apt-get clean; apt-get autoclean;apt-get autoremove


    3. XEN 4.3 Installation

    First we should check if our CPU supports hardware virtualization

    egrep '(vmx|svm)' --color=always /proc/cpuinfo

    The output should look like this

    root@xxxx:~# egrep '(vmx|svm)' --color=always /proc/cpuinfo
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm tpr_shadow vnmi flexpriority
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm tpr_shadow vnmi flexpriority

    Then we can procees for installing XEN : the hypervisor, xen aware kernel and xen tools. This can be done by a metapackage:

    apt-get install xen-linux-system xen-tools

    Debian Wheezy uses Grub 2 and as default it lists normal kernels first, and only then lists the Xen hypervisor and its kernels.You can change this to cause Grub 2 to prefer to boot Xen, by changing the priority of Grub's Xen configuration script (20_linux_xen) to be higher than the standard Linux config (10_linux). This is most easily done using dpkg-divert:

    dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen

    After any update to the Grub configuration you must apply the configuration by running:



    4. Xen tunning

    1. Set Dom0 memory to 512Mb and disable Ballooning so DomUs will never took Dom0's memory
    2. Prevent virtual machines on logical volumes from appearing in the boot menu
    3. Disable Xendomains save & restore
    4. Set only 1st CPU for dom0


    1. Set Dom0 memory to 512Mb and disable balooning


    (dom0-min-mem 512)
    (enable-dom0-ballooning no)


    2. Prevent virtual machines on logical volumes from appearing in the boot menu

    Modify grub:


    #allocate memory to dom0
    # Disable OS prober


    3. Disable Xendomains save & restore




    4. Limit dom0 to 1st CPU only

    By default all CPUs are shared among dom0 and all domU (guests). It may broke dom0 responsibility if guests consume too much CPU time. To avoid this, it is possible to grant one (or more) processor core to dom0 and also pin it to dom0.

    Add following options to /etc/default/grub to allocate one cpu core to dom0:


    # Xen boot parameters for all Xen boots
    GRUB_CMDLINE_XEN="dom0_mem=512M dom0_max_vcpus=1 dom0_vcpus_pin"
    # Disable OS prober

    Remeber to run update-grub after those!


    Also make such changes in /etc/xen/xend-config.sxp:


    (dom0-cpus 1)

    !! Before reboot check if you did update-grub and then reboot your server !!


    5. XEN simple bridge setup

    Most setups will use BRIDGEed configuration, and that is enough if you are planing for simple environment, but I highly encourage to use OpenVSwitch if you require VLANs or any ACL etc.

    Here is the easy version for just a simple bridge:

    apt-get install bridge-utils

    We need to edit: /etc/network/interfaces


     ##The loopback network interface 
    auto lo iface lo inet loopback
    iface eth0 inet manual
    auto xenbr0
    iface xenbr0 inet static address netmask network broadcast gateway # dns-* options are implemented by the resolvconf package, if installed dns-nameservers dns-search bridge_ports eth0
    #other possibly useful options in a virtualized environment
    bridge_stp off # disable Spanning Tree Protocol bridge_waitport 0 # no delay before a port becomes available bridge_fd 0 # no forwarding delay

    After that you should restart networking with

    /etc/init.d/networking restart

    And check if your bridge has come up

    brctl show
    bridge name     bridge id               STP enabled     interfaces
    xenbr0          8000.001a6479cd8c       no              eth0


    6. XEN-Tools setup to use templates for your virtual machines (guests-domU)

    Now lets create our first image of new domU with a template. We need to edit the file:


    lvm = vg0 # your volume group which we created before
    install-method = debootstrap # This will launch debian installer
    size   = 10Gb      # Disk image size.
    memory = 512Mb    # Memory size
    swap   = 2Gb    # Swap size
    noswap = 1      # Don't use swap at all for the new system.
    fs     = ext4     # use the EXT3 filesystem for the disk image.
    dist   = `xt-guess-suite-and-mirror --suite` # Default distribution to install.
    image  = sparse   # Specify sparse vs. full disk images.
    gateway    = # Your gateway, that dom0 is also using with bridge
    netmask    = # no comments needed;)
    broadcast  = # Your broadcast 
    bridge = xenbr0 # Here we should use the bridge we created, xen will make it into eth0 on domU.
    cachedir = /var/cache/xen-tools/archives/ # Place for *.debs for further use
    passwd = 1 #This will ask for a root password to create on new machine
    kernel = /boot/vmlinuz-`uname -r` #It will use same kernel as dom0 is using( this is crucial, to avoid problems)
    initrd = /boot/initrd.img-`uname -r` #Same here for initrd
    mirror = `xt-guess-suite-and-mirror --mirror` 
    mirror = #Enter you mirror of debian
    ext3_options     = noatime,nodiratime,errors=remount-ro #Simple options to apply for new machine
    ext2_options     = noatime,nodiratime,errors=remount-ro
    xfs_options      = defaults
    reiserfs_options = defaults
    btrfs_options    = defaults
    boot = 1 # Do we want to boot our new machine after creating it. Hell yeah!


    7. Creating first domU Guest

    Now we can create our first domU host:

    xen-create-image --hostname domU-test --dist=jessie --vcpus=4 --ip=

    This will take about 3-4 minutes , depending on speed of your connection , CPU and disk speed. Once is done we can look it up, if its working.

    xm list
    Name                                        ID   Mem VCPUs      State   Time(s)
    Domain-0                                     0   512     1     r-----     93.8
    domU-test                                    1   512     4     -b----     23.9

    I recommend to read xen-create-image --help as it will give you an idea what you can change providing it with options to create any distribution.Example: run commands after and all other stuff you might need. There is also an option to generate an image and use it for ready to launch system-template, without having to run installer for 3 minutes every time.

    Now you can log onto you new host with ssh.

    Below you can find links I used in this tutorial:

    1. Xen in Debian Wiki
    2. OpenVSwitch
    3. Xen 4 in Debian Wheezy
    4. Tuning XEN - XEN wiki
    5. Apt Sources Generator
    6. Xen Best Practises
    7. How to use LVM
    Share this page:

    4 Comment(s)