How To Set Up Software RAID1 On A Running LVM System (Incl. GRUB2 Configuration) (Ubuntu 11.10)
Version 1.0
Author: Falko Timme
This guide explains how to set up software RAID1 on an already running LVM system (Ubuntu 11.10). The GRUB2 bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).
I do not issue any guarantee that this will work for you!
1 Preliminary Note
In this tutorial I'm using an Ubuntu 11.10 system with two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sdb is currently unused, and /dev/sda has the following partitions (this is the default Ubuntu 11.10 LVM partitioning scheme - you should find something similar on your system unless you chose to manually partition during the installation of the system):
- /dev/sda1: /boot partition, ext2;
- /dev/sda2: extended, contains /dev/sda5;
- /dev/sda5: is used for LVM (volume group server1) and contains / (volume root) and swap (volume swap_1).
In the end I want to have the following situation:
- /dev/md0 (made up of /dev/sda1 and /dev/sdb1): /boot partition, ext2;
- /dev/md1 (made up of /dev/sda5 and /dev/sdb5): LVM (volume group server1), contains / (volume root) and swap (volume swap_1).
This is the current situation:
df -h
root@server1:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/server1-root
4.2G 995M 3.0G 25% /
udev 238M 4.0K 238M 1% /dev
tmpfs 99M 216K 99M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 247M 0 247M 0% /run/shm
/dev/sda1 228M 24M 193M 11% /boot
root@server1:~#
fdisk -l
root@server1:~# fdisk -l
Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0000ad1b
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 499711 248832 83 Linux
/dev/sda2 501758 10483711 4990977 5 Extended
/dev/sda5 501760 10483711 4990976 8e Linux LVM
Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/mapper/server1-root: 4563 MB, 4563402752 bytes
255 heads, 63 sectors/track, 554 cylinders, total 8912896 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/server1-root doesn't contain a valid partition table
Disk /dev/mapper/server1-swap_1: 532 MB, 532676608 bytes
255 heads, 63 sectors/track, 64 cylinders, total 1040384 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/server1-swap_1 doesn't contain a valid partition table
root@server1:~#
pvdisplay
root@server1:~# pvdisplay
--- Physical volume ---
PV Name /dev/sda5
VG Name server1
PV Size 4.76 GiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 1218
Free PE 3
Allocated PE 1215
PV UUID dKWi5I-GMPP-wzoP-qXfi-e93A-YVmp-A7O1IK
root@server1:~#
vgdisplay
root@server1:~# vgdisplay
--- Volume group ---
VG Name server1
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 4.76 GiB
PE Size 4.00 MiB
Total PE 1218
Alloc PE / Size 1215 / 4.75 GiB
Free PE / Size 3 / 12.00 MiB
VG UUID kwDyrp-sFA7-3s3i-FVWc-AGck-NX6H-yo4Pyt
root@server1:~#
lvdisplay
root@server1:~# lvdisplay
--- Logical volume ---
LV Name /dev/server1/root
VG Name server1
LV UUID dNn3NY-YhPm-qE8r-Dr8L-k8CG-ECLp-YjRnGf
LV Write Access read/write
LV Status available
# open 1
LV Size 4.25 GiB
Current LE 1088
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 252:0
--- Logical volume ---
LV Name /dev/server1/swap_1
VG Name server1
LV UUID HKIiwv-7X8Y-rzeg-aedK-5RZo-g3Km-QjxkdL
LV Write Access read/write
LV Status available
# open 2
LV Size 508.00 MiB
Current LE 127
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 252:1
root@server1:~#
2 Installing mdadm
The most important tool for setting up RAID is mdadm. Let's install it like this:
apt-get install initramfs-tools mdadm
General type of mail configuration: <-- Internet Site
System mail name: <-- server1.example.com
Afterwards, we load a few kernel modules (to avoid a reboot):
modprobe linear
modprobe multipath
modprobe raid0
modprobe raid1
modprobe raid5
modprobe raid6
modprobe raid10
Now run
cat /proc/mdstat
The output should look as follows:
root@server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>
root@server1:~#
3 Preparing /dev/sdb
To create a RAID1 array on our already running system, we must prepare the /dev/sdb hard drive for RAID1, then copy the contents of our /dev/sda hard drive to it, and finally add /dev/sda to the RAID1 array.
First, we copy the partition table from /dev/sda to /dev/sdb so that both disks have exactly the same layout:
sfdisk -d /dev/sda | sfdisk --force /dev/sdb
The output should be as follows:
root@server1:~# sfdisk -d /dev/sda | sfdisk --force /dev/sdb
Checking that no-one is using this disk right now ...
OK
Disk /dev/sdb: 652 cylinders, 255 heads, 63 sectors/track
sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdb: unrecognized partition table type
Old situation:
No partitions found
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
New situation:
Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
/dev/sdb1 * 2048 499711 497664 83 Linux
/dev/sdb2 501758 10483711 9981954 5 Extended
/dev/sdb3 0 - 0 0 Empty
/dev/sdb4 0 - 0 0 Empty
/dev/sdb5 501760 10483711 9981952 8e Linux LVM
Warning: partition 1 does not end at a cylinder boundary
Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@server1:~#
The command
fdisk -l
should now show that both HDDs have the same layout:
root@server1:~# fdisk -l
Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0000ad1b
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 499711 248832 83 Linux
/dev/sda2 501758 10483711 4990977 5 Extended
/dev/sda5 501760 10483711 4990976 8e Linux LVM
Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb1 * 2048 499711 248832 83 Linux
/dev/sdb2 501758 10483711 4990977 5 Extended
/dev/sdb5 501760 10483711 4990976 8e Linux LVM
Disk /dev/mapper/server1-root: 4563 MB, 4563402752 bytes
255 heads, 63 sectors/track, 554 cylinders, total 8912896 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/server1-root doesn't contain a valid partition table
Disk /dev/mapper/server1-swap_1: 532 MB, 532676608 bytes
255 heads, 63 sectors/track, 64 cylinders, total 1040384 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/server1-swap_1 doesn't contain a valid partition table
root@server1:~#
Next we must change the partition type of our three partitions on /dev/sdb to Linux raid autodetect:
fdisk /dev/sdb
root@server1:~# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): <-- m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): <-- t
Partition number (1-5): <-- 1
Hex code (type L to list codes): <-- L
0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris
1 FAT12 39 Plan 9 82 Linux swap / So c1 DRDOS/sec (FAT-
2 XENIX root 3c PartitionMagic 83 Linux c4 DRDOS/sec (FAT-
3 XENIX usr 40 Venix 80286 84 OS/2 hidden C: c6 DRDOS/sec (FAT-
4 FAT16 <32M 41 PPC PReP Boot 85 Linux extended c7 Syrinx
5 Extended 42 SFS 86 NTFS volume set da Non-FS data
6 FAT16 4d QNX4.x 87 NTFS volume set db CP/M / CTOS / .
7 HPFS/NTFS 4e QNX4.x 2nd part 88 Linux plaintext de Dell Utility
8 AIX 4f QNX4.x 3rd part 8e Linux LVM df BootIt
9 AIX bootable 50 OnTrack DM 93 Amoeba e1 DOS access
a OS/2 Boot Manag 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O
b W95 FAT32 52 CP/M 9f BSD/OS e4 SpeedStor
c W95 FAT32 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs
e W95 FAT16 (LBA) 54 OnTrackDM6 a5 FreeBSD ee GPT
f W95 Ext'd (LBA) 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/
10 OPUS 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b
11 Hidden FAT12 5c Priam Edisk a8 Darwin UFS f1 SpeedStor
12 Compaq diagnost 61 SpeedStor a9 NetBSD f4 SpeedStor
14 Hidden FAT16 <3 63 GNU HURD or Sys ab Darwin boot f2 DOS secondary
16 Hidden FAT16 64 Novell Netware af HFS / HFS+ fb VMware VMFS
17 Hidden HPFS/NTF 65 Novell Netware b7 BSDI fs fc VMware VMKCORE
18 AST SmartSleep 70 DiskSecure Mult b8 BSDI swap fd Linux raid auto
1b Hidden W95 FAT3 75 PC/IX bb Boot Wizard hid fe LANstep
1c Hidden W95 FAT3 80 Old Minix be Solaris boot ff BBT
1e Hidden W95 FAT1
Hex code (type L to list codes): <-- fd
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): <-- t
Partition number (1-5): <-- 5
Hex code (type L to list codes): <-- fd
Changed system type of partition 5 to fd (Linux raid autodetect)
Command (m for help): <-- w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
root@server1:~#
The command
fdisk -l
should now show that /dev/sdb1 and /dev/sdb5 are of the type Linux raid autodetect:
root@server1:~# fdisk -l
Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0000ad1b
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 499711 248832 83 Linux
/dev/sda2 501758 10483711 4990977 5 Extended
/dev/sda5 501760 10483711 4990976 8e Linux LVM
Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb1 * 2048 499711 248832 fd Linux raid autodetect
/dev/sdb2 501758 10483711 4990977 5 Extended
/dev/sdb5 501760 10483711 4990976 fd Linux raid autodetect
Disk /dev/mapper/server1-root: 4563 MB, 4563402752 bytes
255 heads, 63 sectors/track, 554 cylinders, total 8912896 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/server1-root doesn't contain a valid partition table
Disk /dev/mapper/server1-swap_1: 532 MB, 532676608 bytes
255 heads, 63 sectors/track, 64 cylinders, total 1040384 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/server1-swap_1 doesn't contain a valid partition table
root@server1:~#
To make sure that there are no remains from previous RAID installations on /dev/sdb, we run the following commands:
mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb5
If there are no remains from previous RAID installations, each of the above commands will throw an error like this one (which is nothing to worry about):
root@server1:~# mdadm --zero-superblock /dev/sdb1
mdadm: Unrecognised md component device - /dev/sdb1
root@server1:~#
Otherwise the commands will not display anything at all.