How To Set Up Xen 3.0 From Binaries In Ubuntu 6.06 LTS (Dapper Drake)

Want to support HowtoForge? Become a subscriber!
 
Submitted by istari (Contact Author) (Forums) on Tue, 2006-06-13 23:53. :: Ubuntu | Xen | Virtualization

How To Set Up Xen 3.0 From Binaries In Ubuntu 6.06 LTS (Dapper Drake)

Version 1.0
Author: Ásgeir Bjarni Ingvarsson <istari AT hlekkir DOT com>
Last Edited: June 13. 2006

This particular way of installing and configuring Xen is just the path of least resistance for me. There are many other possible ways of configuring the system.

1 Install a clean base server system.

Setup Ubuntu like you normally would. I have created 3 partitions on hda, but you can do it any way you like. Just keep your differences in mind when you edit the config files.

  • hda1 is swap
  • hda2 is 2 GB and mounted as /
  • hda3 is the rest of the disk and mounted as /xen-images

2 Update the system and install Xen

2.1 Install Xen and configure it

Open a root shell so you don't have to type sudo for every command.

sudo -s

Update the system and install nessecary packages.

apt-get update
apt-get upgrade
apt-get install iproute python python-twisted bridge-utils debootstrap

Download and extract the Xen 3.0 tarball from XenSource.

tar xvf xen-3.0.1-install-x86_32.tgz
cd xen-3.0.1-install
./install.sh

Check for error messages - it should say OK to all.

Next we create modules.dep and map files for the new kernel. (see /lib/modules for the correct kernel version)

/sbin/depmod -a 2.6.16-xen

Edit /etc/mkinitramfs/modules and append the following line:

loop max_loop=64

If you run out of loop devices later just increase max_loop and rebuild the initrd.

Create an initrd image. Use the same version number as before.

cd /boot
mkinitramfs -o initrd.img-2.6.16-xen 2.6.16-xen

Edit /boot/grub/menu.lst placing the following lines before the Automagic section

title Xen 3.0 / XenLinux 2.6 
kernel /boot/xen-3.gz
module /boot/vmlinuz-2.6-xen root=/dev/hda2 ro
module /boot/initrd.img-2.6.16-xen

Make Xen start up and autostart selected guests when the system starts up. xend must start before, and must be stopped after xendomains.

update-rc.d xend start 30 2 3 4 5 . stop 31 0 1 6 .
update-rc.d xendomains start 31 2 3 4 5 . stop 30 0 1 6 .

Disable Thread-Local Storage (remember to check this after every update)

mv /lib/tls /lib/tls.disabled

2.2 The following workarounds may not be required later, but I had to do them.

Rename xen-backend.rules

mv /etc/udev/rules.d/xen-backend.rules /etc/udev/rules.d/92-xen-backend.rules

To make sure that /var/run/xenstored and /var/run/xend exist. Edit /etc/init.d/xend and insert the following lines after the check for /proc/xen/capabilities

if [ ! -d /var/run/xend ] ; then
mkdir -p /var/run/xend
fi

if [ ! -d /var/run/xenstored ] ; then
mkdir -p /var/run/xenstored
fi

Edit /etc/init.d/xendomains and change the LOCKFILE line to read

LOCKFILE=/var/lock/xendomains

2.3 All done for this part so reboot.

When the system is back up try the following command to verify that everything is ok.

xm list

You should see something similar to this:

Name                              ID Mem(MiB) VCPUs State  Time(s)
Domain-0 0 463 1 r----- 42.3

3 Configuring the guest domains

3.1 Create disk images and bootstrap them

Create a mountpoint for the images.

mkdir -p /xen-images/mnt

Create a 1 GB image file and 500 MB swap file, for larger images increase count.

dd if=/dev/zero of=/xen-images/guest_base.img bs=1024k count=1000
dd if=/dev/zero of=/xen-images/guest_base-swap.img bs=1024k count=500

Change permissions for the image files. No one should have access to your Domain-0 computer since that would compromise security for all of the guest domains, but this is a good idea anyway and doesn't affect Xen.

chmod 640 /xen-images/guest_base*

Format guest_base.img as ext3 and then format guest_base-swap.img as swap. When it says "/xen-images/guest_base.img is not a block special device." answer yes to proceed anyway.

mkfs.ext3 /xen-images/guest_base.img
mkswap /xen-images/guest_base-swap.img

Mount the guest image and bootstrap it. You should replace http://archive.ubuntu.com/ubuntu/ with a mirror that is closer to you.

mount -o loop /xen-images/guest_base.img /xen-images/mnt
debootstrap --arch i386 dapper /xen-images/mnt http://archive.ubuntu.com/ubuntu/

Copy your /etc/apt/sources.list to the new image

cp /etc/apt/sources.list /xen-images/mnt/etc/apt/

Copy the kernel modules.

cp -dpR /lib/modules/2.6.16-xen /xen-images/mnt/lib/modules/

Disable Thread-Local Storage.

mv /xen-images/mnt/lib/tls /xen-images/mnt/lib/tls.disabled

Configure networking for the guest by editing /xen-images/mnt/etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
# Uncomment this and configure after the system has booted for the first time
#auto eth0
#iface eth0 inet static
# address 192.168.0.101
# netmask 255.255.255.0
# broadcast 192.168.0.255
# gateway 192.168.0.1
# dns-nameservers 192.168.0.1

Create /xen-images/mnt/etc/hosts and make it look like this.

127.0.0.1       localhost localhost.localdomain
127.0.0.1 guest

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Create /xen-images/mnt/etc/hostname

echo guest > /xen-images/mnt/etc/hostname

Edit /xen-images/mnt/etc/fstab so it looks like this.

proc            /proc       proc    defaults    0 0
/dev/hda1 / ext3 defaults,errors=remount-ro 0 1
/dev/hda2 none swap sw 0 0

Unmount the image.

umount /xen-images/mnt

If you get this warning "umount: /xen-images/mnt: device is busy" don't worry, it's not important.

3.2 Boot up the base image to finalize the configuration

Create a config file for the guest /etc/xen/baseimage

#  -*- mode: python; -*-
kernel = "/boot/vmlinuz-2.6.16-xen"
ramdisk = "/boot/initrd.img-2.6.16-xen"
memory = 128
name = "baseimage"
vif = ['bridge=xenbr0']
disk = ['file:/xen-images/guest_base.img,hda1,w','file:/xen-images/guest_base-swap.img,hda2,w']
ip = "192.168.0.101"
netmask = "255.255.255.0"
gateway = "192.168.0.1"
hostname = "baseimage"
root = "/dev/hda1 ro"
extra = "4"

Start the guest.

xm create baseimage -c

Login as root and set the root password.

Enable shadow passwords.

shadowconfig on

Create a backup of /etc/network/interfaces

cp /etc/network/interfaces /etc/network/interfaces.bak

Edit /etc/network/interfaces and put in correct values so you can access the Internet

Start networking.

ifup -a

Install packages (add ubuntu-desktop and more if you want).

apt-get update
apt-get install ubuntu-base configure-debian openssh-server

Disable Thread-Local Storage if it has been updated.

mv /lib/tls /lib/tls.disabled

Configure the system any way you like, I recommend that you try every menu item. You should do utils->console-data before utils->console-common.

configure-debian

Add the new user you created to /etc/sudoers

visudo

Stop networking.

ifdown -a

Replace /etc/network/interfaces with the backup copy.

mv /etc/network/interfaces.bak /etc/network/interfaces

You are now done setting up the base image so shut it down.

4 Creating your first guest domain

Make a copy of the base guest images.

cp /xen-images/guest_base.img /xen-images/guestdom1.img
cp /xen-images/guest_base-swap.img /xen-images/guestdom1-swap.img

Make a copy of /etc/xen/baseimage for the new guest.

cp /etc/xen/baseimage /etc/xen/guestdom1

Edit /etc/xen/guestdom1 and change name, disk and hostname to the following values.

name = "guestdom1"
disk = ['file:/xen-images/guestdom1.img,hda1,w','file:/xen-images/guestdom1-swap.img,hda2,w']
hostname = "guestdom1"

You will want to change the following files for each new guest you create.

  • /etc/hostname
  • /etc/hosts
  • /etc/network/interfaces

To make the guest start up automatically and make Domain 0 wait for it before shutting down.

ln -s /etc/xen/guestdom1 /etc/xen/auto/

Or if you want to start and stop the guest manually.

xm create guestdom1

For further information check the Xen documentation

References:
http://www.howtoforge.com/perfect_xen_setup_debian_ubuntu
https://wiki.ubuntu.com/XenOnUbuntuBinaryInstall
http://www.ubuntuforums.org/showthread.php?t=178027

Other points of interest:
https://wiki.ubuntu.com/XenVirtualMachine/XenOnUbuntuDapper
http://www.shorewall.net/Xen.html
http://www.shorewall.net/XenMyWay.html


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by guojing (registered user) on Wed, 2007-04-25 18:37.

    If you want to follow this howto to the letters, get Xen 3.0.2 even though this howto said Xen 3.0.1.


 I have tried both versions. 3.0.1 not only has slightly different version of Linux kernel, but it didn't work at all no matter how I tweaked it accordingly (to this howto and to the Ubuntu 100 hacks book). For one example, in the /lib/modules directory, it contains both a Xen0 and XenU directory. In later versions, this has simplified to just a Xen directory.

Submitted by edu7 (registered user) on Thu, 2007-03-15 14:24.

Great Howto. Thanks for that.

I wanted to share that when I tried to install xen 3.0.4 I went into some problems.
There was already a known bug report on xen bugtracker.
Sorry that I don't have the link for you here, I forgot which error that I had and can't find the link now.

Just for those who want it smooth install xen 3.0.3.

Submitted by bz1 (registered user) on Wed, 2007-01-31 06:12.

Excellent writeup! Worked like a charm! I used these instructions for Edgy.

 The only issue I ran into was when I was creating Dom1 as I received this error msg:

Error: Device 0 (vif) could not be reconnected. Hotplug scripts not working.

 I also found that /var/log/xen-hotplug.log had:

trap: 53: ERR: bad trap

The fix is to change the first line in all the scripts in /etc/xen/scripts from /bin/sh to /bin/bash and then I could successfully create multiple DomU's!!!! I found the fix in:

http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2006-July/000569.html

Thanks to all!

 -bz 

Submitted by lindsay_keir (registered user) on Mon, 2006-09-04 20:34.
  • I did not partition my hda1 but left it all under "/"
  • I installed the xen kernel using you instructions (thank you istari * 2)
  • Installed xen-tools and created the default location
 sudo apt-get install xen-tools
sudo mkdir /home/xen
sudo mkdir /home/xen/domains
sudo chmod a+rw /home/xen
sudo chmod a+rw /home/xen/domains
sudo xen-create-images --kernel=/boot/vmlinuz-2.2.16-xen --hostname=test
sudo xm create test.cfg -c
test login: admin1 # use YOUR sudo user
Password: ****** 

 Oh yeah!

  • The /home/xen/domains default is why I used a single partition.
  •  xen's default is 2.2.12 - you MUST name the kernel
  • no internet required - the image is created from the "host" system.
  • I just did this  - all tailoring and testing has NOT been done
Submitted by Anonymous (not registered) on Thu, 2006-08-24 15:38.

Never seen such a beautiful how-to. If we have included bootstrapping directly from ubuntu CD that would have been really great.

 

 

Submitted by Anonymous (not registered) on Sat, 2006-08-05 13:30.

Hi !

IMHO using loopbacks for Xen is a very poor setup. LVM partitions backends are A LOT better (faster), and that's what we do at GPLHost.

Also, if you guys want to give it a try, we have added some xen functionnalities to our open source control panel. It's based on a python SOAP server that runs on the dom0, while a client in PHP (the control panel) connects to it to know the state of the VM, start, shutdown and destroy it. It also includes a network and CPU graphing tool.

 

Thomas (from GPLHost.com)

 

Submitted by Anonymous (not registered) on Wed, 2006-07-26 04:29.

Your instructions (which worked for me - thanks) include a suggestion to change the permissions on /etc/sudoers, edit it and change the permissions back.  The 'right' way to edit the file is to issue this command:

 

visudo

 

If you want to use an editor other than 'vi' you can set the EDITOR environment variable to point to your favoured editor (presumably in .bashrc or similar).  Using visudo ensures that you cannot install a sudoers file which has errors in it. 

Submitted by Anonymous (not registered) on Mon, 2006-07-17 15:42.

Because the Ubuntu Kernel has EVMS patch in it, I got some errors with booting. I also couldn't mount my boot partition that was on a FLASH disk at /dev/hda1.

 After removing the EVMS package all problems where solved.

Joost 

Submitted by voipfc (registered user) on Wed, 2007-04-11 18:21.

I can't mount partitions on /dev/sdb when Xen is started.[admin@server ~] mount /dev/sdb1 /backup1
mount: /dev/sdb1 already mounted or /backup1 busy
When I reboot with the non Xen kernel everything is fine.After googling around it seems that some lvm or dm issues may be the cause and that sounds like the EVMS issue you mention here.
How is EVMS disabled? 

Submitted by Anonymous (not registered) on Tue, 2006-06-20 11:22.
I wanted to try out Xen and followed these instructions to the letter. It all works perfectly. Thank you for putting together such a clean and clear guide. You have saved me days of work. If anyone else wants to try Xen - start here !
Submitted by Anonymous (not registered) on Sat, 2006-06-24 20:24.
all instructions worked well. now i started using xen. thanks a lot !
bvidinli (www.ehcp.net, easy hosting control panel..)