How To Set Up Software RAID1 On A Running System (Incl. GRUB Configuration) (Debian Lenny) - Page 4

9 Testing

Now let's simulate a hard drive failure. It doesn't matter if you select /dev/sda or /dev/sdb here. In this example I assume that /dev/sdb has failed.

To simulate the hard drive failure, you can either shut down the system and remove /dev/sdb from the system, or you (soft-)remove it like this:

mdadm --manage /dev/md0 --fail /dev/sdb1
mdadm --manage /dev/md1 --fail /dev/sdb2
mdadm --manage /dev/md2 --fail /dev/sdb3

mdadm --manage /dev/md0 --remove /dev/sdb1
mdadm --manage /dev/md1 --remove /dev/sdb2
mdadm --manage /dev/md2 --remove /dev/sdb3

Shut down the system:

shutdown -h now

Then put in a new /dev/sdb drive (if you simulate a failure of /dev/sda, you should now put /dev/sdb in /dev/sda's place and connect the new HDD as /dev/sdb!) and boot the system. It should still start without problems.

Now run

cat /proc/mdstat

and you should see that we have a degraded array:

server1:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sda3[0]
      5550336 blocks [2/1] [U_]

md1 : active (auto-read-only) raid1 sda2[0]
      497920 blocks [2/1] [U_]

md0 : active raid1 sda1[0]
      240832 blocks [2/1] [U_]

unused devices: <none>
server1:~#

The output of

fdisk -l

should look as follows:

server1:~# fdisk -l

Disk /dev/sda: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000740fc

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          30      240943+  fd  Linux raid autodetect
/dev/sda2              31          92      498015   fd  Linux raid autodetect
/dev/sda3              93         783     5550457+  fd  Linux raid autodetect

Disk /dev/sdb: 6442 MB, 6442450944 bytes
255 heads, 63 sectors/track, 783 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/md0: 246 MB, 246611968 bytes
2 heads, 4 sectors/track, 60208 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md1: 509 MB, 509870080 bytes
2 heads, 4 sectors/track, 124480 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

Disk /dev/md2: 5683 MB, 5683544064 bytes
2 heads, 4 sectors/track, 1387584 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md2 doesn't contain a valid partition table
server1:~#

Now we copy the partition table of /dev/sda to /dev/sdb:

sfdisk -d /dev/sda | sfdisk /dev/sdb

(If you get an error, you can try the --force option:

sfdisk -d /dev/sda | sfdisk --force /dev/sdb

)

server1:~# sfdisk -d /dev/sda | sfdisk /dev/sdb
Checking that no-one is using this disk right now ...
OK

Disk /dev/sdb: 783 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *        63    481949     481887  fd  Linux raid autodetect
/dev/sdb2        481950   1477979     996030  fd  Linux raid autodetect
/dev/sdb3       1477980  12578894   11100915  fd  Linux raid autodetect
/dev/sdb4             0         -          0   0  Empty
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
server1:~#

Afterwards we remove any remains of a previous RAID array from /dev/sdb...

mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb2
mdadm --zero-superblock /dev/sdb3

... and add /dev/sdb to the RAID array:

mdadm -a /dev/md0 /dev/sdb1
mdadm -a /dev/md1 /dev/sdb2
mdadm -a /dev/md2 /dev/sdb3

Now take a look at

cat /proc/mdstat

server1:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[2] sda3[0]
      5550336 blocks [2/1] [U_]
      [====>................]  recovery = 23.2% (1288576/5550336) finish=1.1min speed=64428K/sec

md1 : active raid1 sdb2[2] sda2[0]
      497920 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdb1[1] sda1[0]
      240832 blocks [2/2] [UU]

unused devices: <none>
server1:~#

Wait until the synchronization has finished:

server1:~# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[1] sda3[0]
      5550336 blocks [2/2] [UU]

md1 : active raid1 sdb2[1] sda2[0]
      497920 blocks [2/2] [UU]

md0 : active raid1 sdb1[1] sda1[0]
      240832 blocks [2/2] [UU]

unused devices: <none>
server1:~#

Then run

grub

and install the bootloader on both HDDs:

root (hd0,0)
setup (hd0)
root (hd1,0)
setup (hd1)
quit

That's it. You've just replaced a failed hard drive in your RAID1 array.

 

10 Links

Share this page:

2 Comment(s)

Add comment

Comments

From: Anonymous at: 2009-08-28 01:43:48

in your configuration, you have 3 seperate raid devices that share the same head on the hard disk(s).  You would get better performance if you instead made a single raid device (md0) and partitioned that device with 3 partitions.  The reason for this is simple.  The IO scheduler would be aware that this is a single disk (or read/write queue) and schedule accordingly where having 3 different raid arrays all sharing a single disk head and the kernel will think that they operate independantly and not schedule in an optimal way.

This certainly wouldnt matter for the /boot partition because it is likely only read while the other arrays are not but the swap and root raids will be read and written too at the same time.

 

From: at: 2011-03-30 20:19:19

After upgrading the debian server failed to start. I now try to recover the boot through the rescue system, tried to mount / dev/md1 on / mnt/md1 but there are folders that are empty, such as root, etc, proc.
root@teste:/# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid1 sdb1[0] sda1[1]
97536 blocks [2/2] [UU]

md1 : active raid1 sda3[1]
296993088 blocks [2/1] [_U]

unused devices: <none>


######################################

root@teste:/# mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Thu Jan 27 16:55:06 2011
Raid Level : raid1
Array Size : 97536 (95.27 MiB 99.88 MB)
Used Dev Size : 97536 (95.27 MiB 99.88 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Wed Mar 30 19:10:44 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : adbe2b1b:4917a312:7792c71e:7dc17aa4
Events : 0.44

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 1 1 active sync /dev/sda1
######
root@xteste:/# mdadm --detail /dev/md1
/dev/md1:
Version : 00.90.03
Creation Time : Thu Jan 27 16:55:06 2011
Raid Level : raid1
Array Size : 296993088 (283.23 GiB 304.12 GB)
Used Dev Size : 296993088 (283.23 GiB 304.12 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 1
Persistence : Superblock is persistent

Update Time : Wed Mar 30 19:09:54 2011
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

UUID : 5c8e4766:9653e87b:7792c71e:7dc17aa4
Events : 0.15600

Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 3 1 active sync /dev/sda3
root@teste:/#
###############
 
root@teste:/# fdisk -l
Disk /dev/sda: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000b01cb
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          13       97656   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2              13         137     1000000   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3             137       37111   296993164   fd  Linux raid autodetect
Disk /dev/sdb: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000b32a6
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1          13       97656   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2              13         137     1000000   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sdb3             137       37111   296993164   fd  Linux raid autodetect
Disk /dev/md1: 304.1 GB, 304120922112 bytes
2 heads, 4 sectors/track, 74248272 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000
Disk /dev/md1 doesn't contain a valid partition table
Disk /dev/md0: 99 MB, 99876864 bytes
2 heads, 4 sectors/track, 24384 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000
Disk /dev/md0 doesn't contain a valid partition table
root@teste:/#
 
what should I do to solve the problem of raid1?