Xen On Debian Wheezy With LVM

Author: Jason Norwood-Young jason [at] freespeechpub dot co dot za

So you've got some serious server needs. You want a bunch of servers, with a couple of operating systems, running their own crazy sites doing righteous stuff, serving the world and making it a better place. There's only one problem - you're poor. Too poor to fire up AWS servers and Rackspace servers without counting the hours they're up, and weeping every time you recieve the bill.

Don't fret - you can have what you're looking for, for way less than those cloud servers cost ya, especially if you need them up 24/7/365. With virtualization, you can use one beeeg server to host lots of little servers. It's almost like having your own cloud!

This tutorial covers installing Xen and your first virtual machine on top of Debian Wheezy. If you're using Squeeze, check out Falko Timme's tutorial, which I used as inspiration for mine (even though we do things a little differently).


0. Picking a server to run this on

So I use Hetzner Germany. Why? Because they're bloody cheap. With a virtualised environment, what you're really looking for is lots of memory. The more memory, the more virtual systems you can cram on there, and the bigger they can be. I have two 32GB servers, which lets me run about 8 servers between 1GB and 4GB each with no hassle. This isn't a recommendation of any sort and I have no relationship with Hetzner apart from forking over my hard-earned cash to them every month.

What you're looking for is a provider that will give you a root server nice and cheap, with lots and lots of memory, and the ability to remotely reboot and rescue in case you really screw up.


1. Installing Xen

I assume you've installed Wheezy, you're logged in as root, and you've done all the apt-get updates and apt-get upgrades you need to do. So let's get cracking!

apt-get install xen-linux-system xen-tools

This installs Xen, some high-mem stuff, and the very useful xen-tools which we'll use to create our virtual images later.

Next we want to tell Xen how to handle networking. I've got a small confession to make: I'm not a networking guy. In fact I detest networking. It's the plumbing of the internet, and I refuse to bend over and show my plumber's crack. And Xen networking is complicated, with weird routes and bridges and god-knows what else. So I just do the easiest, quickest, dirtiest solution - I stick it in NAT mode. (No, I don't really know what NAT is, but it works, okay? Stop asking questions.)

pico /etc/xen/xend-config.sxp

Uncomment the following two lines:

    (network-script network-nat)
    (vif-script     vif-nat)

Even though we've installed Xen, it's not actually running. Xen runs on the kernel level, which means you actually have to boot into Xen.

First, we need to tell Debian to boot Xen instead of its usual kernel:

dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen update-grub

Then we need to reboot:


I know you can't wait to ssh into your new, awesome Xen system, but give it a second to reboot.

Okay, try it now.

How about now?

You in? We good? Good. Let's carry on.


2. Controlling Xen

So now you want to make sure that Xen is actually running. On Squeeze, you could just run uname -r, and you'd see something like: 2.6.32-5-xen-amd64. For some mysterious reason unknown to mankind, this doesn't work on Wheezy, so instead do this:

xen dmesg

You should see something like:

(XEN) Xen version 4.1.4 (Debian 4.1.4-3+deb7u1) ([email protected]) (gcc version 4.7.2 (Debian 4.7.2-5) ) Sun May  5 14:44:49 UTC 2013
(XEN) Bootloader: GRUB 1.99-27+deb7u2
(XEN) Command line: placeholder
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures

If you don't see that, you've done something wrong. Go back and try again. For everyone else, SUCCESS! Congratulations! You now have an awesome Dom0 instance!

Dom-what? Who you calling dom?

Relax chap. Dom0, or Domain 0, is your host server, the thing you're on now, the bottom of the pile. The guest servers are referred to as DomU, possibly because it also sounds slightly insulting.

xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 31089     8     r——      8.8

That command lists all our virtual servers, including Dom0. At the moment that's the only server we have running.

We can do a lot with xm, including xm create which brings a server up, xm destroy which kills a server, xm reboot which reboots a server, and xm top which monitors all the running servers.


3. Configuring LVM

You can create Xen servers either as block devices or as LVM instances. I like LVM, because it's more manly, and it also makes it waaay easier to resize the partitions later.

First we need an empty, unmounted drive to put LVM onto. I don't have an empty, unmounted drive, so I'm just going to steal one away from my server.

df -h
Filesystem                                              Size  Used Avail Use% Mounted on
rootfs                                                 1008G  915M  956G   1% /
udev                                                     10M     0   10M   0% /dev
tmpfs                                                   3.0G  292K  3.0G   1% /run
/dev/disk/by-uuid/5642a4c4-1bcb-49e2-b89c-7d754e6a521a 1008G  915M  956G   1% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   9.2G     0  9.2G   0% /run/shm
/dev/md1                                                496M   34M  437M   8% /boot
/dev/md3                                                1.7T  196M  1.7T   1% /home

1.7TB for /home? Are you serious? That seems totally unneccessary.

pico /etc/fstab

Comment out the offending drive

#/dev/md/3 /home ext4 defaults 0 0

Unmount the drive

umount /dev/md3

Right, now we can create our LVM partition thingy.

pvcreate /dev/md3
  Writing physical volume data to disk "/dev/md3"
  Physical volume "/dev/md3" successfully created
vgcreate xen-vol /dev/md3
  --- Volume group ---
  VG Name               xen-vol
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1.71 TiB
  PE Size               4.00 MiB
  Total PE              448996
  Alloc PE / Size       0 / 0
  Free  PE / Size       448996 / 1.71 TiB
  VG UUID               wa0QCN-lBc1-545B-zMwf-vIuS-VXXI-GZAfbW



4. Configuring Xen Tools

Xen Tools makes our Xen life much easier, especially when it comes to creating new servers. But first we need to configure it a bit.

pico /etc/xen-tools/xen-tools.conf
lvm = xen-vol
size   = 20Gb      # Disk image size.
memory = 4Gb    # Memory size
swap   = 512Mb    # Swap size
gateway    =
netmask    =
broadcast  =
mirror = http://ftp.de.debian.org/debian/
mirror_precise = http://archive.ubuntu.com/ubuntu # (and any others you want)
serial_device = hvc0
disk_device = xvda #default
copyhosts = 1


5. Creating a guest server

Time for the main event! This is the process you'll repeat every time you want to create a new Xen guest server.

For our first test server, I'm going to make something a bit smaller than the defaults we've set up in xen-tools.conf. You can override any of the defaults on the command line, depending on what you want. Since we're not using a dhcp server, you'll need to set "hostname" and "ip" every time. Everything else is optional.

xen-create-image --hostname=test --ip= --memory=512Mb --size=5G --swap=256Kb
General Information
Hostname       :  test
Distribution   :  wheezy
Mirror         :  http://ftp.de.debian.org/debian/
Partitions     :  swap            256Kb (swap)
                  /               5G    (ext3)
Image type     :  full
Memory size    :  512Mb
Kernel path    :  /boot/vmlinuz-3.2.0-4-amd64
Initrd path    :  /boot/initrd.img-3.2.0-4-amd64
Networking Information
IP Address 1   : [MAC: 00:16:3E:27:33:2D]
Netmask        :
Broadcast      :
Gateway        :

Creating swap on /dev/xen-vol/test-swap
Creating ext3 filesystem on /dev/xen-vol/test-disk
Installation method: debootstrap
Running hooks
No role scripts were specified.  Skipping
Creating Xen configuration file
No role scripts were specified.  Skipping
Setting up root password
Generating a password for the new guest.
All done

Logfile produced at:
Installation Summary
Hostname        :  test
Distribution    :  wheezy
IP-Address(es)  :
RSA Fingerprint :  de:92:b7:63:55:20:74:46:d3:de:cc:ff:55:22:69:24
Root Password   :  MlkaIn8

This creates the config file and the partition for us, but doesn't boot the server for us. The config file is located at /etc/xen/hostname.cfg. You can edit this file directly if you need to change stuff like memory later.

Let's go ahead and start the server

xm create /etc/xen/test.cfg
Using config file "/etc/xen/test.cfg".
Started domain test (id=1)

What's pretty cool is that "test" will be added to our /etc/hosts file, which means we can just do this:

ssh test

Use the password that you received at the end of the xen-create-image output, and you're in!

Make sure you can see the interwebs:

ping google.com
PING google.com ( 56(84) bytes of data.
64 bytes from fa-in-f113.1e100.net ( icmp_req=1 ttl=48 time=6.05 ms
64 bytes from fa-in-f113.1e100.net ( icmp_req=2 ttl=48 time=6.03 ms
--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 6.030/6.044/6.058/0.014 ms


6. Have a beer

You deserve it!


7. Pointing IP addresses at your virtual servers

You'll need a bunch of IP addresses from your ISP (called a subnet). I pay a little more a month for an extra eight IP addresses. But getting them to go straight through to your virtual server takes a bit of work.

pico /etc/network/interfaces
auto br0
iface br0 inet static
  pre-up brctl addbr $IFACE
  post-down brctl delbr $IFACE
/etc/init.d/networking restart
ifconfig br0
br0       Link encap:Ethernet  HWaddr 82:18:d1:b8:37:c9
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::9118:d1ff:feb8:37c9/64 Scope:Link
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:172 (172.0 B)
ip addr add dev eth0
iptables -A PREROUTING -t nat -p all -i eth0 -d -j DNAT --to

Use --bridge=br0 when using xen-create. For existing /etc/xen/*.cfg files, edit this line:

    vif         = [ 'ip=,mac=00:15:3F:B4:AC:34' ]

to look like this:

    vif         = [ 'ip=,mac=00:15:3F:B4:AC:34','bridge=br0' ]

Destroy and create the virtual server if necessary.

Share this page:

2 Comment(s)