Rollback To A Working State With btrfs + apt-btrfs-snapshot On Ubuntu 12.10
This tutorial explains how you can revert failed apt operations (like apt-get upgrade) and roll back to the previous system state with apt-btrfs-snapshot on an Ubuntu 12.10 system that uses the btrfs file system. apt-btrfs-snapshot creates a snapshot of the system before the apt operation. Being able to easily restore the previous system state after a failed apt operation takes away much of the pain system administrators have to deal with normally and is one of the greatest features of the btrfs file system.
I do not issue any guarantee that this will work for you!
1 Preliminary Note
In this tutorial I have installed the whole system on a btrfs file system, i.e., there's no separate /boot partition on an ext file system. If you use a separate /boot partition and apt installs anything in that partition (like a new kernel), you cannot undo changes to the /boot partition with apt-btrfs-snapshot- only changes on the btrfs partition can be reverted.
My hard drive is named /dev/sda in this tutorial, my system partition is /dev/sda1.
A note for Ubuntu users:
Because we must run all the steps from this tutorial with root privileges, we can either prepend all commands in this tutorial with the string sudo, or we become root right now by typing
sudo su
2 Install apt-btrfs-snapshot
apt-btrfs-snapshot can be installed as follows:
apt-get install apt-btrfs-snapshot
To check if apt-btrfs-snapshot is able to create snapshots on apt operations, run
apt-btrfs-snapshot supported
It should display:
root@server1:~# apt-btrfs-snapshot supported
Supported
root@server1:~#
If it doesn't, your btrfs subvolume layout probably differs from Ubuntu's default layout which is as follows:
- @ subvolume: mounted to /.
- @home subvolume: mounted to /home.
This is the default Ubuntu subvolume layout:
btrfs subvolume list /
root@server1:~# btrfs subvolume list /
ID 256 top level 5 path @
ID 258 top level 5 path @home
root@server1:~#
If apt-btrfs-snapshot supports your system, you can proceed to chapter 3.
3 Do An apt Operation
Now let's do some apt operation like apt-get upgrade to test if we can rollback to the previous state.
Update your package database...
apt-get update
... and upgrade your system:
apt-get upgrade
root@server1:~# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages have been kept back:
linux-headers-generic linux-image-generic
The following packages will be upgraded:
apport base-files isc-dhcp-client isc-dhcp-common libwhoopsie0 linux-generic lsb-base lsb-release python3-apport python3-distupgrade python3-problem-report python3.2 python3.2-minimal
ubuntu-release-upgrader-core vim vim-common vim-runtime vim-tiny whoopsie
19 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Need to get 14.4 MB of archives.
After this operation, 3,072 B of additional disk space will be used.
Do you want to continue [Y/n]? <-- ENTER
Get:1 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main base-files amd64 6.5ubuntu12 [69.6 kB]
Get:2 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main whoopsie amd64 0.2.7 [25.1 kB]
Get:3 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main libwhoopsie0 amd64 0.2.7 [7,054 B]
Get:4 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main lsb-base all 4.0-0ubuntu26.1 [10.3 kB]
Get:5 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main isc-dhcp-client amd64 4.2.4-1ubuntu10.1 [775 kB]
Get:6 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main isc-dhcp-common amd64 4.2.4-1ubuntu10.1 [836 kB]
Get:7 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main lsb-release all 4.0-0ubuntu26.1 [10.7 kB]
Get:8 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main python3.2 amd64 3.2.3-6ubuntu3.1 [2,585 kB]
Get:9 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main python3.2-minimal amd64 3.2.3-6ubuntu3.1 [1,798 kB]
Get:10 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main vim amd64 2:7.3.547-4ubuntu1.1 [1,051 kB]
Get:11 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main vim-tiny amd64 2:7.3.547-4ubuntu1.1 [413 kB]
Get:12 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main vim-runtime all 2:7.3.547-4ubuntu1.1 [6,317 kB]
Get:13 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main vim-common amd64 2:7.3.547-4ubuntu1.1 [85.7 kB]
Get:14 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main ubuntu-release-upgrader-core all 1:0.190.4 [27.7 kB]
Get:15 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main python3-distupgrade all 1:0.190.4 [141 kB]
Get:16 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main python3-problem-report all 2.6.1-0ubuntu6 [9,578 B]
Get:17 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main python3-apport all 2.6.1-0ubuntu6 [85.7 kB]
Get:18 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main apport all 2.6.1-0ubuntu6 [164 kB]
Get:19 http://de.archive.ubuntu.com/ubuntu/ quantal-updates/main linux-generic amd64 3.5.0.18.21 [1,714 B]
Fetched 14.4 MB in 2s (5,465 kB/s)
Supported
Create a snapshot of '/tmp/apt-btrfs-snapshot-mp-jnW7I_/@' in '/tmp/apt-btrfs-snapshot-mp-jnW7I_/@apt-snapshot-2012-11-22_11:50:38'
(Reading database ... 52666 files and directories currently installed.)
Preparing to replace base-files 6.5ubuntu11 (using .../base-files_6.5ubuntu12_amd64.deb) ...
Unpacking replacement base-files ...
Processing triggers for man-db ...
Processing triggers for install-info ...
Processing triggers for plymouth-theme-ubuntu-text ...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.5.0-17-generic
Setting up base-files (6.5ubuntu12) ...
(Reading database ... 52666 files and directories currently installed.)
Preparing to replace whoopsie 0.2.5 (using .../whoopsie_0.2.7_amd64.deb) ...
whoopsie stop/waiting
Unpacking replacement whoopsie ...
Preparing to replace libwhoopsie0 0.2.5 (using .../libwhoopsie0_0.2.7_amd64.deb) ...
Unpacking replacement libwhoopsie0 ...
Preparing to replace lsb-base 4.0-0ubuntu26 (using .../lsb-base_4.0-0ubuntu26.1_all.deb) ...
Unpacking replacement lsb-base ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Setting up lsb-base (4.0-0ubuntu26.1) ...
(Reading database ... 52666 files and directories currently installed.)
Preparing to replace isc-dhcp-client 4.2.4-1ubuntu10 (using .../isc-dhcp-client_4.2.4-1ubuntu10.1_amd64.deb) ...
Unpacking replacement isc-dhcp-client ...
Preparing to replace isc-dhcp-common 4.2.4-1ubuntu10 (using .../isc-dhcp-common_4.2.4-1ubuntu10.1_amd64.deb) ...
Unpacking replacement isc-dhcp-common ...
Preparing to replace lsb-release 4.0-0ubuntu26 (using .../lsb-release_4.0-0ubuntu26.1_all.deb) ...
Unpacking replacement lsb-release ...
Preparing to replace python3.2 3.2.3-6ubuntu3 (using .../python3.2_3.2.3-6ubuntu3.1_amd64.deb) ...
Unpacking replacement python3.2 ...
Preparing to replace python3.2-minimal 3.2.3-6ubuntu3 (using .../python3.2-minimal_3.2.3-6ubuntu3.1_amd64.deb) ...
Unpacking replacement python3.2-minimal ...
Preparing to replace vim 2:7.3.547-4ubuntu1 (using .../vim_2%3a7.3.547-4ubuntu1.1_amd64.deb) ...
Unpacking replacement vim ...
Preparing to replace vim-tiny 2:7.3.547-4ubuntu1 (using .../vim-tiny_2%3a7.3.547-4ubuntu1.1_amd64.deb) ...
Unpacking replacement vim-tiny ...
Preparing to replace vim-runtime 2:7.3.547-4ubuntu1 (using .../vim-runtime_2%3a7.3.547-4ubuntu1.1_all.deb) ...
Unpacking replacement vim-runtime ...
Preparing to replace vim-common 2:7.3.547-4ubuntu1 (using .../vim-common_2%3a7.3.547-4ubuntu1.1_amd64.deb) ...
Unpacking replacement vim-common ...
Preparing to replace ubuntu-release-upgrader-core 1:0.190.1 (using .../ubuntu-release-upgrader-core_1%3a0.190.4_all.deb) ...
Unpacking replacement ubuntu-release-upgrader-core ...
Preparing to replace python3-distupgrade 1:0.190.1 (using .../python3-distupgrade_1%3a0.190.4_all.deb) ...
Unpacking replacement python3-distupgrade ...
Preparing to replace python3-problem-report 2.6.1-0ubuntu3 (using .../python3-problem-report_2.6.1-0ubuntu6_all.deb) ...
Unpacking replacement python3-problem-report ...
Preparing to replace python3-apport 2.6.1-0ubuntu3 (using .../python3-apport_2.6.1-0ubuntu6_all.deb) ...
Unpacking replacement python3-apport ...
Preparing to replace apport 2.6.1-0ubuntu3 (using .../apport_2.6.1-0ubuntu6_all.deb) ...
apport stop/waiting
Unpacking replacement apport ...
Preparing to replace linux-generic 3.5.0.17.19 (using .../linux-generic_3.5.0.18.21_amd64.deb) ...
Unpacking replacement linux-generic ...
Processing triggers for man-db ...
Processing triggers for mime-support ...
Processing triggers for ureadahead ...
Setting up libwhoopsie0 (0.2.7) ...
Setting up whoopsie (0.2.7) ...
whoopsie start/running, process 7859
Setting up isc-dhcp-common (4.2.4-1ubuntu10.1) ...
Setting up isc-dhcp-client (4.2.4-1ubuntu10.1) ...
Setting up lsb-release (4.0-0ubuntu26.1) ...
Setting up python3.2-minimal (3.2.3-6ubuntu3.1) ...
Setting up python3.2 (3.2.3-6ubuntu3.1) ...
Setting up vim-common (2:7.3.547-4ubuntu1.1) ...
Setting up vim-runtime (2:7.3.547-4ubuntu1.1) ...
Processing /usr/share/vim/addons/doc
Setting up vim (2:7.3.547-4ubuntu1.1) ...
Setting up vim-tiny (2:7.3.547-4ubuntu1.1) ...
Setting up python3-distupgrade (1:0.190.4) ...
Setting up ubuntu-release-upgrader-core (1:0.190.4) ...
Setting up python3-problem-report (2.6.1-0ubuntu6) ...
Setting up python3-apport (2.6.1-0ubuntu6) ...
Setting up apport (2.6.1-0ubuntu6) ...
apport start/running
Setting up linux-generic (3.5.0.18.21) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
root@server1:~#
As you see, apt-btrfs-snapshot has automatically created as snapshot of our system (called @apt-snapshot-2012-11-22_11:50:38 in this example) before the upgrade. You can check that with...
btrfs subvolume list /
root@server1:~# btrfs subvolume list /
ID 256 top level 5 path @
ID 258 top level 5 path @home
ID 260 top level 5 path @apt-snapshot-2012-11-22_11:50:38
root@server1:~#
... and:
apt-btrfs-snapshot list
root@server1:~# apt-btrfs-snapshot list
Available snapshots:
@apt-snapshot-2012-11-22_11:50:38
root@server1:~#
4 Rollback
Now let's assume the last apt operation turned our working system into one that isn't working as expected anymore. That's why we want to restore the previous system state, i.e., we want to do a rollback.
Therefore we mount the btrfs filesystem to a separate location, e.g. /mnt:
mount /dev/sda1 /mnt
We can now see our subvolumes in the output of:
ls -l /mnt/
root@server1:~# ls -l /mnt/
total 0
drwxr-xr-x 1 root root 230 Nov 22 10:46 @
drwxr-xr-x 1 root root 230 Nov 22 10:46 @apt-snapshot-2012-11-22_11:50:38
drwxr-xr-x 1 root root 26 Nov 22 10:57 @home
root@server1:~#
@apt-snapshot-2012-11-22_11:50:38 is a snapshot of our working root filesystem (@) before the apt operation. In order to make the system boot from that working snapshot instead of from the current subvolume, we rename @ to something else and then @apt-snapshot-2012-11-22_11:50:38 to @:
mv /mnt/@ /mnt/@_badroot
mv /mnt/@apt-snapshot-2012-11-22_11:50:38 /mnt/@
Now reboot:
reboot
5 Check If The Rollback Was Successful
After the reboot we should check if the rollback was successful. To do this, we repeat the apt operation which made our system unusable, e.g.:
apt-get update
apt-get upgrade
If the rollback was successful, apt-get upgrade should show the same packages available for update as before (as this is just a check if the rollback was successful, don't install the updates again):
root@server1:~# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages have been kept back:
linux-headers-generic linux-image-generic
The following packages will be upgraded:
apport base-files isc-dhcp-client isc-dhcp-common libwhoopsie0 linux-generic lsb-base lsb-release python3-apport python3-distupgrade python3-problem-report python3.2 python3.2-minimal
ubuntu-release-upgrader-core vim vim-common vim-runtime vim-tiny whoopsie
19 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Need to get 0 B/14.4 MB of archives.
After this operation, 3,072 B of additional disk space will be used.
Do you want to continue [Y/n]? <-- n
6 Delete The @ Subvolume (Optional)
If you are sure the rollback was successful and you don't need the old @ subvolume (now named @_badroot) anymore, you can delete it to free up some space.
mount /dev/sda1 /mnt
ls -l /mnt/
root@server1:~# ls -l /mnt/
total 0
drwxr-xr-x 1 root root 230 Nov 22 10:46 @
drwxr-xr-x 1 root root 230 Nov 22 10:46 @_badroot
drwxr-xr-x 1 root root 26 Nov 22 10:57 @home
root@server1:~#
btrfs subvolume delete /mnt/@_badroot
umount /mnt
