How To Set Up Software RAID1 On A Running LVM System (Incl. GRUB2 Configuration) (Ubuntu 11.10) - Page 2
4 Creating Our RAID Arrays
Now let's create our RAID arrays /dev/md0 and /dev/md1. /dev/sdb1 will be added to /dev/md0 and/dev/sdb5 to /dev/md1. /dev/sda1 and /dev/sda5 can't be added right now (because the system is currently running on them), therefore we use the placeholder missing in the following two commands:
mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb5
You might see the following message for each command - just press y to continue:
root@server1:~# mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb5
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? <-- y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.
root@server1:~#
The command
cat /proc/mdstat
should now show that you have two degraded RAID arrays ([_U] or [U_] means that an array is degraded while [UU] means that the array is ok):
root@server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdb5[1]
4989940 blocks super 1.2 [2/1] [_U]
md0 : active raid1 sdb1[1]
248820 blocks super 1.2 [2/1] [_U]
unused devices: <none>
root@server1:~#
Next we create a filesystem (ext2) on our non-LVM RAID array /dev/md0:
mkfs.ext2 /dev/md0
Now we come to our LVM RAID array /dev/md1. To prepare it for LVM, we run:
pvcreate /dev/md1
Then we add /dev/md1 to our volume group server1:
vgextend server1 /dev/md1
The output of
pvdisplay
should now be similar to this:
root@server1:~# pvdisplay
--- Physical volume ---
PV Name /dev/sda5
VG Name server1
PV Size 4.76 GiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 1218
Free PE 3
Allocated PE 1215
PV UUID dKWi5I-GMPP-wzoP-qXfi-e93A-YVmp-A7O1IK
--- Physical volume ---
PV Name /dev/md1
VG Name server1
PV Size 4.76 GiB / not usable 1012.00 KiB
Allocatable yes
PE Size 4.00 MiB
Total PE 1218
Free PE 1218
Allocated PE 0
PV UUID w1Mg12-OHEj-paLg-9xyJ-jQuU-cQHT-p2qVKf
root@server1:~#
The output of
vgdisplay
should be as follows:
root@server1:~# vgdisplay
--- Volume group ---
VG Name server1
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 9.52 GiB
PE Size 4.00 MiB
Total PE 2436
Alloc PE / Size 1215 / 4.75 GiB
Free PE / Size 1221 / 4.77 GiB
VG UUID kwDyrp-sFA7-3s3i-FVWc-AGck-NX6H-yo4Pyt
root@server1:~#
Next we must adjust /etc/mdadm/mdadm.conf (which doesn't contain any information about our new RAID arrays yet) to the new situation:
cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf_orig
mdadm --examine --scan >> /etc/mdadm/mdadm.conf
Display the contents of the file:
cat /etc/mdadm/mdadm.conf
In the file you should now see details about our two (degraded) RAID arrays:
# mdadm.conf # # Please refer to mdadm.conf(5) for information about this file. # # by default, scan all partitions (/proc/partitions) for MD superblocks. # alternatively, specify devices to scan, using wildcards if desired. DEVICE partitions # auto-create devices with Debian standard permissions CREATE owner=root group=disk mode=0660 auto=yes # automatically tag new arrays as belonging to the local system HOMEHOST <system> # instruct the monitoring daemon where to send mail alerts MAILADDR root # definitions of existing MD arrays # This file was auto-generated on Tue, 20 Mar 2012 15:40:06 +0100 # by mkconf $Id$ ARRAY /dev/md/0 metadata=1.2 UUID=2d5659ba:1978bfac:40d0b815:229d3382 name=server1.example.com:0 ARRAY /dev/md/1 metadata=1.2 UUID=3c524dfa:445bb555:b4d039e9:b39553e1 name=server1.example.com:1 |
Next we modify /etc/fstab. Comment out the current /boot partition and add the line /dev/md0 /boot ext2 defaults 0 2 instead so that the file looks as follows:
vi /etc/fstab
# /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # <file system> <mount point> <type> <options> <dump> <pass> proc /proc proc nodev,noexec,nosuid 0 0 /dev/mapper/server1-root / ext4 errors=remount-ro 0 1 # /boot was on /dev/sda1 during installation #UUID=33ea6f8d-e79c-4776-8c6b-d08b043cfec1 /boot ext2 defaults 0 2 /dev/md0 /boot ext2 defaults 0 2 /dev/mapper/server1-swap_1 none swap sw 0 0 /dev/fd0 /media/floppy0 auto rw,user,noauto,exec,utf8 0 0 |
Next replace /dev/sda1 with /dev/md0 in /etc/mtab:
vi /etc/mtab
/dev/mapper/server1-root / ext4 rw,errors=remount-ro 0 0 proc /proc proc rw,noexec,nosuid,nodev 0 0 sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0 fusectl /sys/fs/fuse/connections fusectl rw 0 0 none /sys/kernel/debug debugfs rw 0 0 none /sys/kernel/security securityfs rw 0 0 udev /dev devtmpfs rw,mode=0755 0 0 devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0 tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0 none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0 none /run/shm tmpfs rw,nosuid,nodev 0 0 /dev/mdd0 /boot ext2 rw 0 0 |
Now up to the GRUB2 boot loader. Create the file /etc/grub.d/09_swraid1_setup as follows:
cp /etc/grub.d/40_custom /etc/grub.d/09_swraid1_setup
vi /etc/grub.d/09_swraid1_setup
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
menuentry 'Ubuntu, with Linux 3.0.0-12-server' --class ubuntu --class gnu-linux --class gnu --class os {
insmod raid
insmod mdraid
insmod part_msdos
insmod ext2
set root='(md/0)'
echo 'Loading Linux 3.0.0-12-server ...'
linux /vmlinuz-3.0.0-12-server root=/dev/mapper/server1-root ro quiet
echo 'Loading initial ramdisk ...'
initrd /initrd.img-3.0.0-12-server
}
|
Make sure you use the correct kernel version in the menuentry stanza (in the linux and initrd lines). You can find it out by running
uname -r
or by taking a look at the current menuentry stanzas in the ### BEGIN /etc/grub.d/10_linux ### section in /boot/grub/grub.cfg. Also make sure that you use the correct volume group in the linux line - if your volume group isn't named server1, you must use something else than root=/dev/mapper/server1-root. Again, take a look at the current menuentry stanzas in the ### BEGIN /etc/grub.d/10_linux ### section in /boot/grub/grub.cfg to find out the correct value.
The important part in our new menuentry stanza is the line set root='(md/0)' - it makes sure that we boot from our RAID1 array /dev/md0 (which will hold the /boot partition) instead of /dev/sda or /dev/sdb which is important if one of our hard drives fails - the system will still be able to boot.
Because we don't use UUIDs for our block devices, open /etc/default/grub...
vi /etc/default/grub
... and uncomment the line GRUB_DISABLE_LINUX_UUID=true:
# If you change this file, run 'update-grub' afterwards to update # /boot/grub/grub.cfg. # For full documentation of the options in this file, see: # info -f grub -n 'Simple configuration' GRUB_DEFAULT=0 #GRUB_HIDDEN_TIMEOUT=0 GRUB_HIDDEN_TIMEOUT_QUIET=true GRUB_TIMEOUT=2 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="" GRUB_CMDLINE_LINUX="" # Uncomment to enable BadRAM filtering, modify to suit your needs # This works with Linux (no patch required) and with any kernel that obtains # the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...) #GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef" # Uncomment to disable graphical terminal (grub-pc only) #GRUB_TERMINAL=console # The resolution used on graphical terminal # note that you can use only modes which your graphic card supports via VBE # you can see them in real GRUB with the command `vbeinfo' #GRUB_GFXMODE=640x480 # Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux GRUB_DISABLE_LINUX_UUID=true # Uncomment to disable generation of recovery mode menu entries #GRUB_DISABLE_RECOVERY="true" # Uncomment to get a beep at grub start #GRUB_INIT_TUNE="480 440 1" |
Next open /etc/initramfs-tools/conf.d/mdadm...
vi /etc/initramfs-tools/conf.d/mdadm
... and set BOOT_DEGRADED to true so that the system can boot from a degraded array without asking (otherwise the system would ask at the beginning of the boot process if you want to boot from a degraded array which is a bad thing if the server is in a remote location):
# mdadm boot_degraded configuration # # You can run 'dpkg-reconfigure mdadm' to modify the values in this file, if # you want. You can also change the values here and changes will be preserved. # Do note that only the values are preserved; the rest of the file is # rewritten. # # BOOT_DEGRADED: # Do you want to boot your system if a RAID providing your root filesystem # becomes degraded? # # Running a system with a degraded RAID could result in permanent data loss # if it suffers another hardware fault. # # However, you might answer "yes" if this system is a server, expected to # tolerate hardware faults and boot unattended. BOOT_DEGRADED=true |
Now run
update-grub
to write our new kernel stanza from /etc/grub.d/09_swraid1_setup to /boot/grub/grub.cfg.
Next we adjust our ramdisk to the new situation:
update-initramfs -u
5 Moving Our Data To The RAID Arrays
Now that we've modified all configuration files, we can copy the contents of /dev/sda to /dev/sdb (including the configuration changes we've made in the previous chapter).
To move the contents of our LVM partition /dev/sda5 to our LVM RAID array /dev/md1, we use the pvmove command:
pvmove -i 2 /dev/sda5 /dev/md1
This can take some time, so please be patient.
Afterwards, we remove /dev/sda5 from the volume group server1...
vgreduce server1 /dev/sda5
... and tell the system to not use /dev/sda5 anymore for LVM:
pvremove /dev/sda5
The output of
pvdisplay
should now be as follows:
root@server1:/boot/grub# pvdisplay
--- Physical volume ---
PV Name /dev/md1
VG Name server1
PV Size 4.76 GiB / not usable 1012.00 KiB
Allocatable yes
PE Size 4.00 MiB
Total PE 1218
Free PE 3
Allocated PE 1215
PV UUID w1Mg12-OHEj-paLg-9xyJ-jQuU-cQHT-p2qVKf
root@server1:/boot/grub#
Next we change the partition type of /dev/sda5 to Linux raid autodetect and add /dev/sda5 to the /dev/md1 array:
fdisk /dev/sda
root@server1:~# fdisk /dev/sda
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): <-- t
Partition number (1-5): <-- 5
Hex code (type L to list codes): <-- fd
Changed system type of partition 5 to fd (Linux raid autodetect)
Command (m for help): <-- w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
root@server1:~#
mdadm --add /dev/md1 /dev/sda5
Now take a look at
cat /proc/mdstat
... and you should see that the RAID array /dev/md1 is being synchronized:
root@server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda5[2] sdb5[1]
4989940 blocks super 1.2 [2/1] [_U]
[====>................] recovery = 20.1% (1005376/4989940) finish=3.7min speed=17482K/sec
md0 : active raid1 sdb1[1]
248820 blocks super 1.2 [2/1] [_U]
unused devices: <none>
root@server1:~#
(You can run
watch cat /proc/mdstat
to get an ongoing output of the process. To leave watch, press CTRL+C.)
Wait until the synchronization has finished (the output should then look like this:
root@server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda5[2] sdb5[1]
4989940 blocks super 1.2 [2/2] [UU]
md0 : active raid1 sdb1[1]
248820 blocks super 1.2 [2/1] [_U]
unused devices: <none>
root@server1:~#
).
Now let's mount /dev/md0:
mkdir /mnt/md0
mount /dev/md0 /mnt/md0
You should now find the array in the output of
mount
root@server1:~# mount
/dev/mapper/server1-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/mdd0 on /boot type ext2 (rw)
/dev/md0 on /mnt/md0 type ext2 (rw)
root@server1:~#
Now we copy the contents of /dev/sda1 to /dev/md0 (which is mounted on /mnt/md0):
cd /boot
cp -dpRx . /mnt/md0
