How To Set Up Software RAID1 On A Running LVM System (Incl. GRUB2 Configuration) (Debian Squeeze) - Page 4

8 Testing

Now let's simulate a hard drive failure. It doesn't matter if you select /dev/sda or /dev/sdb here. In this example I assume that /dev/sdb has failed.

To simulate the hard drive failure, you can either shut down the system and remove /dev/sdb from the system, or you (soft-)remove it like this:

mdadm --manage /dev/md0 --fail /dev/sdb1
mdadm --manage /dev/md1 --fail /dev/sdb5

mdadm --manage /dev/md0 --remove /dev/sdb1
mdadm --manage /dev/md1 --remove /dev/sdb5

Shut down the system:

shutdown -h now

Then put in a new /dev/sdb drive (if you simulate a failure of /dev/sda, you should now put /dev/sdb in /dev/sda's place and connect the new HDD as /dev/sdb!) and boot the system. It should still start without problems.

Now run

cat /proc/mdstat

and you should see that we have a degraded array:

root@server1:~# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda5[2]
      4989940 blocks super 1.2 [2/1] [U_]

md0 : active raid1 sda1[2]
      248820 blocks super 1.2 [2/1] [U_]

unused devices: <none>
root@server1:~#

The output of

fdisk -l

should look as follows:

root@server1:~# fdisk -l

Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003717c

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2              32         653     4990977    5  Extended
Partition 2 does not end on cylinder boundary.
/dev/sda5              32         653     4990976   fd  Linux raid autodetect

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/md0: 254 MB, 254791680 bytes
2 heads, 4 sectors/track, 62205 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md1: 5109 MB, 5109698560 bytes
2 heads, 4 sectors/track, 1247485 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

Disk /dev/dm-0: 4844 MB, 4844421120 bytes
255 heads, 63 sectors/track, 588 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 264 MB, 264241152 bytes
255 heads, 63 sectors/track, 32 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table
root@server1:~#

Now we copy the partition table of /dev/sda to /dev/sdb:

sfdisk -d /dev/sda | sfdisk --force /dev/sdb

root@server1:~# sfdisk -d /dev/sda | sfdisk --force /dev/sdb
Checking that no-one is using this disk right now ...
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
OK

Disk /dev/sdb: 652 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *      2048    499711     497664  fd  Linux raid autodetect
/dev/sdb2        501758  10483711    9981954   5  Extended
/dev/sdb3             0         -          0   0  Empty
/dev/sdb4             0         -          0   0  Empty
/dev/sdb5        501760  10483711    9981952  fd  Linux raid autodetect
Warning: partition 1 does not end at a cylinder boundary
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@server1:~#

Afterwards we remove any remains of a previous RAID array from /dev/sdb...

mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb5

... and add /dev/sdb to the RAID array:

mdadm -a /dev/md0 /dev/sdb1
mdadm -a /dev/md1 /dev/sdb5

Now take a look at

cat /proc/mdstat

root@server1:~# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb5[3] sda5[2]
      4989940 blocks super 1.2 [2/1] [U_]
      [========>............]  recovery = 44.7% (2233024/4989940) finish=0.2min speed=159501K/sec

md0 : active raid1 sdb1[3] sda1[2]
      248820 blocks super 1.2 [2/2] [UU]

unused devices: <none>
root@server1:~#

Wait until the synchronization has finished:

root@server1:~# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb5[3] sda5[2]
      4989940 blocks super 1.2 [2/2] [UU]

md0 : active raid1 sdb1[3] sda1[2]
      248820 blocks super 1.2 [2/2] [UU]

unused devices: <none>
root@server1:~#

Then install the bootloader on both HDDs:

grub-install /dev/sda
grub-install /dev/sdb

That's it. You've just replaced a failed hard drive in your RAID1 array.

 

9 Links

Share this page:

4 Comment(s)

Add comment

Comments

From: at: 2013-04-09 12:36:16

Hi All,

First, Falco,a big thank you for this great tutorial!
I wanted to move my TrunKey Linux 12 fileserver  - which is basically a Debian Squeeze - to RAID1. Since the box is productive in an office, I wanted the whole process to be tested in a VirtualBox machine first, and if OK, easily execute it on the physical machine as well ... as fast as possible to reduce downtime.

So I've put together all the commands in this tutorial  into a script. It was tested and working on a VirtualBox TurnKey fileserver machine, and since the procedure will be quite the same on the physical box, I will do the same on it after a data backup.

In practice this script had to be cut into 3 parts because of the 2 reboots needed during the procedure.

Please feel free to use them, but I highly recommend to get familiar with the usage of the script in a VirtualBox Debian Squeeze, and you should only do it on the Prod physical machin e after.

You can download the script(s) here:
https://www.dropbox.com/sh/n1trc69jukfo41r/HJMi3x-q-a

From: Olivier Berger at: 2011-10-28 23:08:40

About the regeneration of /boot/grub/device.map, may I suggest to use grub-mkdevicemap, and check the file's contents. It helps solving later issues with grub.

Hope this helps.

From: Mark at: 2011-09-09 21:36:37

I've just gone through this on my new Debian system (all except the hard drive failure simulation) and everything worked exactly as predicted. I love it when that happens.

From: at: 2013-05-30 10:16:51

I've put together the steps of the tutorial in a few scripts...Worked on TurnKey Linux 12.0 which is basically based on Debian 6.0.7 Squeeze ...

https://www.dropbox.com/sh/vxmpfxotwhcm8cf/TKcmyQV8Z9