How To Set Up Software RAID1 On A Running System (Incl. GRUB2 Configuration) (Debian Squeeze) - Page 4

9 Testing

Now let's simulate a hard drive failure. It doesn't matter if you select /dev/sda or /dev/sdb here. In this example I assume that /dev/sdb has failed.

To simulate the hard drive failure, you can either shut down the system and remove /dev/sdb from the system, or you (soft-)remove it like this:

mdadm --manage /dev/md0 --fail /dev/sdb1
mdadm --manage /dev/md1 --fail /dev/sdb2
mdadm --manage /dev/md2 --fail /dev/sdb3

mdadm --manage /dev/md0 --remove /dev/sdb1
mdadm --manage /dev/md1 --remove /dev/sdb2
mdadm --manage /dev/md2 --remove /dev/sdb3

Shut down the system:

shutdown -h now

Then put in a new /dev/sdb drive (if you simulate a failure of /dev/sda, you should now put /dev/sdb in /dev/sda's place and connect the new HDD as /dev/sdb!) and boot the system. It should still start without problems.

Now run

cat /proc/mdstat

and you should see that we have a degraded array:

root@server1:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sda3[2]
      4241396 blocks super 1.2 [2/1] [U_]

md1 : active (auto-read-only) raid1 sda2[2]
      499700 blocks super 1.2 [2/1] [U_]

md0 : active raid1 sda1[2]
      498676 blocks super 1.2 [2/1] [U_]

unused devices: <none>
root@server1:~#

The output of

fdisk -l

should look as follows:

root@server1:~# fdisk -l

Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e0f78

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          63      498688   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2              63         125      499712   fd  Linux raid autodetect
Partition 2 does not end on cylinder boundary.
/dev/sda3             125         653     4242432   fd  Linux raid autodetect
Partition 3 does not end on cylinder boundary.

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/md0: 510 MB, 510644224 bytes
2 heads, 4 sectors/track, 124669 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md1: 511 MB, 511692800 bytes
2 heads, 4 sectors/track, 124925 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

Disk /dev/md2: 4343 MB, 4343189504 bytes
2 heads, 4 sectors/track, 1060349 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md2 doesn't contain a valid partition table
root@server1:~#

Now we copy the partition table of /dev/sda to /dev/sdb:

sfdisk -d /dev/sda | sfdisk --force /dev/sdb

root@server1:~# sfdisk -d /dev/sda | sfdisk --force /dev/sdb
Checking that no-one is using this disk right now ...
OK

Disk /dev/sdb: 652 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *      2048    999423     997376  fd  Linux raid autodetect
/dev/sdb2        999424   1998847     999424  fd  Linux raid autodetect
/dev/sdb3       1998848  10483711    8484864  fd  Linux raid autodetect
/dev/sdb4             0         -          0   0  Empty
Warning: partition 1 does not end at a cylinder boundary
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@server1:~#

Afterwards we remove any remains of a previous RAID array from /dev/sdb...

mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb2
mdadm --zero-superblock /dev/sdb3

... and add /dev/sdb to the RAID array:

mdadm -a /dev/md0 /dev/sdb1
mdadm -a /dev/md1 /dev/sdb2
mdadm -a /dev/md2 /dev/sdb3

Now take a look at

cat /proc/mdstat

root@server1:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[3] sda3[2]
      4241396 blocks super 1.2 [2/1] [U_]
      [======>..............]  recovery = 32.2% (1367168/4241396) finish=1.0min speed=44102K/sec

md1 : active raid1 sdb2[3] sda2[2]
      499700 blocks super 1.2 [2/2] [UU]

md0 : active raid1 sdb1[3] sda1[2]
      498676 blocks super 1.2 [2/2] [UU]

unused devices: <none>
root@server1:~#

Wait until the synchronization has finished:

root@server1:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[3] sda3[2]
      4241396 blocks super 1.2 [2/2] [UU]

md1 : active raid1 sdb2[3] sda2[2]
      499700 blocks super 1.2 [2/2] [UU]

md0 : active raid1 sdb1[3] sda1[2]
      498676 blocks super 1.2 [2/2] [UU]

unused devices: <none>
root@server1:~#

Then install the bootloader on both HDDs:

grub-install /dev/sda
grub-install /dev/sdb

That's it. You've just replaced a failed hard drive in your RAID1 array.

 

10 Links

Share this page:

11 Comment(s)

Add comment

Comments

From: al biheiri at: 2011-06-08 12:16:27

Why would you modprobe multipath? I think that is only used for fiber cards. It shouldnt have anything to do with mdadm.

From: paul at: 2011-06-13 17:34:35

Hey.

Great Tutorial! 

Here are a couple of bumps you might run into, if you're trying to use this on Debian Unstable "Wheezy". 
 
 

First
If you want to use GPT enabled harddisks you'll have to create a small (1MB) BIOS Boot Partition or GRUB2 won't install on that drive. 
 
 

Second
If you ARE actually using GUID partition tables you won't be able to change your partitions using fdisk. You'll have to use gdisk (or parted) instead, which is its direct successor and fairly simple to use. Note however, that there's no direct way of duplicating the partition table from one disk to another. You'll have to use sgdisk for that, which installs with gdisk: 
sgdisk --replicate=/dev/sdb /dev/sda

sgdisk --randomize-guids --move-second-header /dev/sdb
The second line is important to distinguish the disks in your system later on. In addition it ensures that your Secondary (Backup) GPT Header really resides on the very last blocks of your disk.
 
 

Third:
This one took me a while :-\
If you're creating the ''/etc/grub.d/09_swraid1_setup'' as mentioned you MUST rename the module GRUB2 is instructed to load from
 
insmod mdraid to insmod mdraid1x

 

(at least for a RAID1 or 10 configuration), as the old-named module does no longer exist. Furthermore, if you don't have a separated /boot/partition you need to fix the path to the kernel and ramdisk (probably "/boot/vmlinuz-2.6…" and "/boot/initrd.img-2.6…"

From: Anonymous at: 2014-10-19 17:04:41

How can I do this on Debian 7? Now I've killed 6 VMs with this Turorial :D

From: Fredrik Falk at: 2013-07-10 19:02:01

Had issues with file not found in grub after reboot into the new array. Tried a couple of things until i found out the md devices werent listed by "ls" in grub rescue prompt. This guide added insmod mdraid in the /etc/grub.d/09_swraid1_setup file but in Debian7 this module doesnt exist. I added mdraid1x instead of mdraid and thereafter it booted fine.

From: at: 2013-11-14 20:23:26

In section:
5 Adjusting The System To RAID1
 
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64' --class debian --class gnu-linux --class gnu --class os {
        insmod raid
        insmod mdraid
        insmod part_msdos
        insmod ext2
        set root='(md/0)'
        echo    'Loading Linux 2.6.32-5-amd64 ...'
        linux   /vmlinuz-2.6.32-5-amd64 root=/dev/md2 ro  quiet
        echo    'Loading initial ramdisk ...'
        initrd  /initrd.img-2.6.32-5-amd64
}

Should be:

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64' --class debian --class gnu-linux --class gnu --class os {
        insmod raid
        insmod mdraid
        insmod part_msdos
        insmod ext2
        set root='(md/0)'
        echo    'Loading Linux 2.6.32-5-amd64 ...'
        linux   /boot/vmlinuz-2.6.32-5-amd64 root=/dev/md2 ro  quiet
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd.img-2.6.32-5-amd64
}

From: Alex at: 2014-06-23 11:23:06


If you are using GPT, then you can use sgdisk to clone the partition table from /dev/sda to the other two hard drives:

# sgdisk --backup=table /dev/sda
# sgdisk -G --load-backup=table /dev/sdb 
 
  -G, --randomize-guids
              Randomize  the  disk’s  GUID  and all partitions’ unique GUIDs (but not their partition type code GUIDs). This function may be used after
              cloning a disk in order to render all GUIDs once again unique.

 
 
Anyway make sure you know what you're doing.

From: kokolorum at: 2012-05-06 11:38:49

Please add this:

apt-get install grub2

dd if=/dev/sda of=/boot/backup.dd.sda

sfdisk -d /dev/sda | sfdisk /dev/sdb

sfdisk -d /dev/sda > /boot/backup.sfdisk.sda

From: simon at: 2012-06-17 23:39:12

Many thanks!

I was stuck without being enable to install RAID1 on a running squeeze install.

From: tv.debian at: 2011-06-08 09:41:50

Hi, thanks for this tutorial. I have been using this kind of manipulation several times, and reading the prompts a few observations:

Using sfdisk to dump the partition layout from the first disk to the second can lead to nasty data corruption, I have experienced it first hand, the raid superblocks and the file-system overlap and lead to all kind of problems. Since the original disk is wiped in the process there is no point doing this anyway, just create empty (no file-system) partitions on the new disk, then create the one disk raid, then format the raid volume. Then the raided disk partition table can safely be used on the second disk.

Alternate: shrink the original partitions by ±1 MB, go on with sfdisk, create your raid, then when everything is done  "resize2fs" the raid volume.

If the raid volume is already created, it's possible to fix it with
"e2fsck -cc", but it's very time consuming. You might want to try it on your setup though, you might be surprised by the result.

Regarding metadata, with 1.2 format which is now the default, it's not needed nor recommended to change the file-system type to "fd" (raid auto-detect). As long as an initramfs is used and mdadm.conf is filled in the system will boot just fine.

Lastly, with the new metadata format and a recent kernel, all raid1 are seen as partitionable, leading to possible shift in volume names under /dev. It's safer to use UUID, or the mdadm "--name=" option to avoid getting confused when "md1" will suddenly become "md1p1" or "md127" after an upgrade (or disaster recovery from a live system). Or force a non-partitionable raid with "--auto=md", should prevent weird raid volumes renaming.

 Have fun.

From: Louis at: 2012-07-24 10:28:34

Nice howto, but 1 sugestions.

dont use /dev/mdX in fstab.

use the command :  blkid

and set the UUID of the raid boot device.

like:

UUID=c3b87084-07c2-4818-b118-6c6764f81545       /                       ext4    defaults,errors=remount-ro       0       1


keep it more error free and universal.

 

From: Patrice Vigier at: 2012-10-31 15:26:05

I could not with this tuto have, in RAID1, the second disk to boot.

The solution came form this :

in file "/etc/default/grub", uncomment the line:

GRUB_TERMINAL=console

after do not forget to run 

update-grub

And it worked