I have begun experimenting with ispconfig3 lxc and the new btrfs filesystem with considerable success and implications for making life much easier for us in the future. In my setup, the ISPConfig 3 master is installed on the physical host on the booting ext4 partition of my 1TB hard drive. Recently it has become possible to format your entire hard drive using the btrfs file system when installing ubuntu server (to boot from a btrfs partition) however the btrfs fsck facility is still under heavy development and I much prefer the stability of working from an ext4 environment for now. For the moment, this is the best of both worlds. During server install I formatted my hard drive as follows: <- / (ext4 file system) -><- swap -><- /btone (btrfs file system) -><- /bttwo (btrfs file system) -> This is my fstab: # <file system> <mount point> <type> <options> <dump> <pass> proc /proc proc nodev,noexec,nosuid 0 0 # / was on /dev/sda1 during installation UUID=2bc66489-e9a0-424a-8753-92ec87a9f3f5 / ext4 errors=remount-ro 0 0 # swap was on /dev/sda2 during installation UUID=a93f94b1-6d78-4183-97e3-4839b77991b0 none swap sw 0 0 # /btone was on /dev/sda3 during installation UUID=ec65eb1a-bede-4338-aa16-352a6783d27a /btone btrfs defaults 0 0 # /bttwo was on /dev/sda4 during installation UUID=54cc801f-3e49-4071-83fd-f1164a1ed344 /bttwo btrfs defaults 0 0 It does not appear to be possible yet to mount a btrfs partition with anything other than the "defaults" (I tried errors=remount-ro and a few other things and the partition simply refuses to mount). At time of writing, I have set up 5 lxc containers (servers in a multiserver setup) called ns1, ns2, web, mail and db on the partition /btone in 5 seperate btrfs subvolumes I have then taken btrfs snapshots of the running servers and moved the servers to my other btrfs partition (/bttwo). This is what I did (ns1 node only). On the host: configure the network bridge: aptitude install bridge-utils vi /etc/network/interfaces Here is mine: # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface # auto eth0 # iface eth0 inet dhcp # Bridge Setup auto br0 iface br0 inet static address 192.168.1.XXX netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 bridge_ports eth0 bridge_fd 0 bridge_maxwait 0 bridge_stp off post-up /usr/sbin/brctl setfd br0 0 Change the red to suit your setup. aptitude install lxc debootstrap libcap-dev debian-archive-keyring libcap2 libcap2-bin (this installed cgroup-lite for me which did not work as well as I had hoped so I removed cgroup-lite and rebooted the server before mounting cgroup the old way) apt-get remove --purge cgroup-lite* reboot note that I mount cgroup on the same partition as the lxc-containers I am running (it may not matter which partition cgroup is mounted on) mkdir /btone/cgroup echo "none /btone/cgroup cgroup defaults 0 0" >> /etc/fstab mount -a cp /usr/lib/lxc/templates/lxc-debian /usr/lib/lxc/templates/lxc-debian.ORIG vi /usr/lib/lxc/templates/lxc-debian Edit the lxc-debian template to your liking - I usually change the locale settings from: chroot $rootfs locale-gen en_US.UTF-8 UTF-8 chroot $rootfs update-locale LANG=en_US.UTF-8 to: chroot $rootfs locale-gen en_AU.UTF-8 UTF-8 chroot $rootfs update-locale LANG=en_AU.UTF-8 and change the package list from: packages=\ ifupdown,\ locales,\ libui-dialog-perl,\ dialog,\ dhcp3-client,\ netbase,\ net-tools,\ iproute,\ openssh-server to: packages=\ ifupdown,\ locales,\ libui-dialog-perl,\ dialog,\ dhcp3-client,\ netbase,\ net-tools,\ iputils-ping,\ ssh,\ cron,\ sudo,\ vim-nox,\ rsyslog,\ iproute,\ apt-utils,\ debconf,\ openssh-server Create a subvolume on your btrfs partition btrfs subvolume create /btone/ns1 Download a squeeze server into your new subvolume (this will be slowish the first time downloading a new distro but is very fast in subsequent same system containers since lxc builds an archived cache of the system and simply copies it accross in new servers) /usr/lib/lxc/templates/lxc-debian -p /btone/ns1 Edit the configuration file of your new container and add the following lines: # networking lxc.utsname = ns1 lxc.network.type = veth lxc.network.flags = up lxc.network.link = br0 lxc.network.name = eth0 lxc.network.ipv4 = 192.168.1.XXX/24 192.168.1.255 (change red to suit your setup) My configuration files are slightly more complex and look something like this: # networking lxc.utsname = ns1 lxc.network.type = veth lxc.network.veth.pair = vethns1 (this can be whatever you want to call the host side of the veth pair) lxc.network.flags = up lxc.network.link = br0 lxc.network.name = eth0 lxc.network.ipv4 = 192.168.1.XXX/24 192.168.1.255 lxc.network.ipv6 = XXXX::XXXX:XXXX:XXXX:XXXX/64 lxc.network.hwaddr = XX:XX:XX:XX:XX:XX (I prefer to let lxc decide what the mac address is and also let it work out my ipv6 address) mkdir /var/lib/lxc/ns1 cp /btone/ns1/config /var/lib/lxc/ns1/config Copy the configuration file accross to the host partition where lxc expects to find the container config file: Start the container in daemon mode lxc-start -n ns1 -d Log in to your new ns1 node using putty or similar and configure server as you normally would. Now take a snapshot of the server - note that there is no difference between a subvolume and a snapshot as far as btrfs is concerned. mkdir /btone/snapshots btrfs subvolume snapshot /btone/ns1 /btone/snapshots/ns1 lxc-info -n ns1 shows that ns1 is still running so lets just stop it while we move it over to a different partition. lxc-stop -n ns1 Create a new subvoloume on the /bttwo partition btrfs subvolume create /bttwo/ns1 Move the rootfs and config file across to the new subvolume mv /btone/snapshots/ns1/rootfs /bttwo/ns1/rootfs mv /btone/snapshots/ns1/config /bttwo/ns1/config delete the empty snapshot btrfs subvolume delete /btone/snapshots/ns1 Now it is very important to reflect the location changes in ns1 lxc container config file before re starting the container as follows: rm /var/lib/lxc/ns1/config vi /bttwo/ns1/config and change the following lines from: lxc.rootfs = /btone/ns1/rootfs lxc.mount.entry=proc /btone/ns1/rootfs/proc proc nodev,noexec,nosuid 0 0 lxc.mount.entry=sysfs /btone/ns1/rootfs/sys sysfs defaults 0 0 to: lxc.rootfs = /bttwo/ns1/rootfs lxc.mount.entry=proc /bttwo/ns1/rootfs/proc proc nodev,noexec,nosuid 0 0 lxc.mount.entry=sysfs /bttwo/ns1/rootfs/sys sysfs defaults 0 0 cp /bttwo/ns1/config /var/lib/lxc/ns1/config Now you can simply start the ns1 server like this: lxc-start -n ns1 -d or you can move cgroup to the new partition (like I do) and set up the containers to autostart before rebooting the host: Edit /ect/default/lxc to look like this: # Comment out to run the lxc init script RUN=yes # Directory containing the container configurations CONF_DIR=/etc/lxc #MIRROR="http://archive.ubuntu.com/ubuntu" Symlink the configuration file for the container into the autostart directory like this: ln -s /var/lib/lxc/ns1/config /etc/lxc/ns1.conf Better to symlink rather than copy the configuration file at this point since fewer changes will be needed if you want to switch containers at a later date. Now when you reboot the host, your container will automatically start - further we have left a complete backup of ns1 on a seperate partition and we have the potential to take snapshots of either subvolume using a script. Again I stress that this is a highly experimental setup and documentation is thin on the ground at this stage. Any advice with regard to a useful snapshot/backup script for use with ispconfig 3 is appreciated. I have been looking at this: https://github.com/mmehnert/btrfs-snapshot-rotation Discussion very welcome. Regards to all.