How To Set Up Software RAID1 On A Running LVM System (Incl. GRUB Configuration) (CentOS 5.3)
Version 1.0
Author: Falko Timme
Follow me on Twitter
This guide explains how to set up software RAID1 on an already running LVM system (CentOS 5.3). The GRUB bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).
I do not issue any guarantee that this will work for you!
1 Preliminary Note
In this tutorial I'm using a CentOS 5.3 system with two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sdb is currently unused, and /dev/sda has the following partitions (this is the default CentOS partitioning scheme - you should find something similar on your system unless you chose to manually partition during the installation of the system):
- /dev/sda1: /boot partition, ext3;
- /dev/sda2: is used for LVM (volume group VolGroup00) and contains / (volume LogVol00) and swap (volume LogVol01).
In the end I want to have the following situation:
- /dev/md0 (made up of /dev/sda1 and /dev/sdb1): /boot partition, ext3;
- /dev/md1 (made up of /dev/sda2 and /dev/sdb2): LVM (volume group VolGroup00), contains / (volume LogVol00) and swap (volume LogVol01).
This is the current situation:
df -h
[root@server1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
8.6G 1.4G 6.8G 17% /
/dev/sda1 99M 13M 82M 14% /boot
tmpfs 250M 0 250M 0% /dev/shm
[root@server1 ~]#
fdisk -l
[root@server1 ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1305 10377990 8e Linux LVM
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
[root@server1 ~]#
pvdisplay
[root@server1 ~]# pvdisplay
/dev/hdc: open failed: No medium found
--- Physical volume ---
PV Name /dev/sda2
VG Name VolGroup00
PV Size 9.90 GB / not usable 22.76 MB
Allocatable yes (but full)
PE Size (KByte) 32768
Total PE 316
Free PE 0
Allocated PE 316
PV UUID aikFEP-FB15-nB9C-Nfq0-eGMG-hQid-GOsDuj
[root@server1 ~]#
vgdisplay
[root@server1 ~]# vgdisplay
/dev/hdc: open failed: No medium found
--- Volume group ---
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 9.88 GB
PE Size 32.00 MB
Total PE 316
Alloc PE / Size 316 / 9.88 GB
Free PE / Size 0 / 0
VG UUID ZPvC10-cN09-fI0S-Vc8l-vOuZ-wM6F-tlz0Mj
[root@server1 ~]#
lvdisplay
[root@server1 ~]# lvdisplay
/dev/hdc: open failed: No medium found
--- Logical volume ---
LV Name /dev/VolGroup00/LogVol00
VG Name VolGroup00
LV UUID vYlky0-Ymx4-PNeK-FTpk-qxvm-PmoZ-3vcNTd
LV Write Access read/write
LV Status available
# open 1
LV Size 8.88 GB
Current LE 284
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--- Logical volume ---
LV Name /dev/VolGroup00/LogVol01
VG Name VolGroup00
LV UUID Ml9MMN-DcOA-Lb6V-kWPU-h6IK-P0ww-Gp9vd2
LV Write Access read/write
LV Status available
# open 1
LV Size 1.00 GB
Current LE 32
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1
[root@server1 ~]#
2 Installing mdadm
The most important tool for setting up RAID is mdadm. Let's install it like this:
yum install mkinitrd mdadm
Afterwards, we load a few kernel modules (to avoid a reboot):
modprobe linear
modprobe multipath
modprobe raid0
modprobe raid1
modprobe raid5
modprobe raid6
modprobe raid10
Now run
cat /proc/mdstat
The output should look as follows:
[root@server1 ~]# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>
[root@server1 ~]#
3 Preparing /dev/sdb
To create a RAID1 array on our already running system, we must prepare the /dev/sdb hard drive for RAID1, then copy the contents of our /dev/sda hard drive to it, and finally add /dev/sda to the RAID1 array.
First, we copy the partition table from /dev/sda to /dev/sdb so that both disks have exactly the same layout:
sfdisk -d /dev/sda | sfdisk /dev/sdb
The output should be as follows:
[root@server1 ~]# sfdisk -d /dev/sda | sfdisk /dev/sdb
Checking that no-one is using this disk right now ...
OK
Disk /dev/sdb: 1305 cylinders, 255 heads, 63 sectors/track
sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
/dev/sdb1 * 63 208844 208782 83 Linux
/dev/sdb2 208845 20964824 20755980 8e Linux LVM
/dev/sdb3 0 - 0 0 Empty
/dev/sdb4 0 - 0 0 Empty
Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
[root@server1 ~]#
The command
fdisk -l
should now show that both HDDs have the same layout:
[root@server1 ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1305 10377990 8e Linux LVM
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 13 104391 83 Linux
/dev/sdb2 14 1305 10377990 8e Linux LVM
[root@server1 ~]#
Next we must change the partition type of our two partitions on /dev/sdb to Linux raid autodetect:
fdisk /dev/sdb
[root@server1 ~]# fdisk /dev/sdb
The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): <-- m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): <-- t
Partition number (1-4): <-- 1
Hex code (type L to list codes): <-- L
0 Empty 1e Hidden W95 FAT1 80 Old Minix bf Solaris
1 FAT12 24 NEC DOS 81 Minix / old Lin c1 DRDOS/sec (FAT-
2 XENIX root 39 Plan 9 82 Linux swap / So c4 DRDOS/sec (FAT-
3 XENIX usr 3c PartitionMagic 83 Linux c6 DRDOS/sec (FAT-
4 FAT16 <32M 40 Venix 80286 84 OS/2 hidden C: c7 Syrinx
5 Extended 41 PPC PReP Boot 85 Linux extended da Non-FS data
6 FAT16 42 SFS 86 NTFS volume set db CP/M / CTOS / .
7 HPFS/NTFS 4d QNX4.x 87 NTFS volume set de Dell Utility
8 AIX 4e QNX4.x 2nd part 88 Linux plaintext df BootIt
9 AIX bootable 4f QNX4.x 3rd part 8e Linux LVM e1 DOS access
a OS/2 Boot Manag 50 OnTrack DM 93 Amoeba e3 DOS R/O
b W95 FAT32 51 OnTrack DM6 Aux 94 Amoeba BBT e4 SpeedStor
c W95 FAT32 (LBA) 52 CP/M 9f BSD/OS eb BeOS fs
e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi ee EFI GPT
f W95 Ext'd (LBA) 54 OnTrackDM6 a5 FreeBSD ef EFI (FAT-12/16/
10 OPUS 55 EZ-Drive a6 OpenBSD f0 Linux/PA-RISC b
11 Hidden FAT12 56 Golden Bow a7 NeXTSTEP f1 SpeedStor
12 Compaq diagnost 5c Priam Edisk a8 Darwin UFS f4 SpeedStor
14 Hidden FAT16 <3 61 SpeedStor a9 NetBSD f2 DOS secondary
16 Hidden FAT16 63 GNU HURD or Sys ab Darwin boot fb VMware VMFS
17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fc VMware VMKCORE
18 AST SmartSleep 65 Novell Netware b8 BSDI swap fd Linux raid auto
1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid fe LANstep
1c Hidden W95 FAT3 75 PC/IX be Solaris boot ff BBT
Hex code (type L to list codes): <-- fd
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): <-- t
Partition number (1-4): <-- 2
Hex code (type L to list codes): <-- fd
Changed system type of partition 2 to fd (Linux raid autodetect)
Command (m for help): <-- w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@server1 ~]#
The command
fdisk -l
should now show that /dev/sdb1 and /dev/sdb2 are of the type Linux raid autodetect:
[root@server1 ~]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1305 10377990 8e Linux LVM
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 13 104391 fd Linux raid autodetect
/dev/sdb2 14 1305 10377990 fd Linux raid autodetect
[root@server1 ~]#
To make sure that there are no remains from previous RAID installations on /dev/sdb, we run the following commands:
mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb2
If there are no remains from previous RAID installations, each of the above commands will throw an error like this one (which is nothing to worry about):
[root@server1 ~]# mdadm --zero-superblock /dev/sdb1
mdadm: Unrecognised md component device - /dev/sdb1
[root@server1 ~]#
Otherwise the commands will not display anything at all.
