How To Set Up Software RAID1 On A Running System (Incl. GRUB2 Configuration) (Debian Squeeze)

Version 1.0
Author: Falko Timme
Follow me on Twitter
Last edited 05/24/2011

This guide explains how to set up software RAID1 on an already running Debian Squeeze system. The GRUB2 bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

In this tutorial I'm using a Debian Squeeze system with two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sdb is currently unused, and /dev/sda has the following partitions:

  • /dev/sda1: /boot partition, ext4;
  • /dev/sda2: swap;
  • /dev/sda3: / partition, ext4

In the end I want to have the following situation:

  • /dev/md0 (made up of /dev/sda1 and /dev/sdb1): /boot partition, ext4;
  • /dev/md1 (made up of /dev/sda2 and /dev/sdb2): swap;
  • /dev/md2 (made up of /dev/sda3 and /dev/sdb3): / partition, ext4

This is the current situation:

df -h

root@server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             4.0G  712M  3.1G  19% /
tmpfs                 249M     0  249M   0% /lib/init/rw
udev                  244M  100K  244M   1% /dev
tmpfs                 249M     0  249M   0% /dev/shm
/dev/sda1             472M   25M  423M   6% /boot
root@server1:~#

fdisk -l

root@server1:~# fdisk -l

Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b0ecb

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          63      498688   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              63         125      499712   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3             125         653     4242432   83  Linux
Partition 3 does not end on cylinder boundary.

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table
root@server1:~#

 

2 Installing mdadm

The most important tool for setting up RAID is mdadm. Let's install it like this:

apt-get install initramfs-tools mdadm

MD arrays needed for the root file system: <-- all

Afterwards, we load a few kernel modules (to avoid a reboot):

modprobe linear
modprobe multipath
modprobe raid0
modprobe raid1
modprobe raid5
modprobe raid6
modprobe raid10

Now run

cat /proc/mdstat

The output should look as follows:

root@server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>
root@server1:~#

 

3 Preparing /dev/sdb

To create a RAID1 array on our already running system, we must prepare the /dev/sdb hard drive for RAID1, then copy the contents of our /dev/sda hard drive to it, and finally add /dev/sda to the RAID1 array.

First, we copy the partition table from /dev/sda to /dev/sdb so that both disks have exactly the same layout:

sfdisk -d /dev/sda | sfdisk --force /dev/sdb

The output should be as follows:

root@server1:~# sfdisk -d /dev/sda | sfdisk --force /dev/sdb
Checking that no-one is using this disk right now ...
OK

Disk /dev/sdb: 652 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *      2048    999423     997376  83  Linux
/dev/sdb2        999424   1998847     999424  82  Linux swap / Solaris
/dev/sdb3       1998848  10483711    8484864  83  Linux
/dev/sdb4             0         -          0   0  Empty
Warning: partition 1 does not end at a cylinder boundary
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@server1:~#

The command

fdisk -l

should now show that both HDDs have the same layout:

root@server1:~# fdisk -l

Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b0ecb

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          63      498688   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              63         125      499712   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3             125         653     4242432   83  Linux
Partition 3 does not end on cylinder boundary.

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          63      498688   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sdb2              63         125      499712   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sdb3             125         653     4242432   83  Linux
Partition 3 does not end on cylinder boundary.
root@server1:~#

Next we must change the partition type of our three partitions on /dev/sdb to Linux raid autodetect:

fdisk /dev/sdb

root@server1:~# fdisk /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help):
 <-- m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help):
 <-- t
Partition number (1-4): <-- 1
Hex code (type L to list codes): <-- L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris
 1  FAT12           39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      41  PPC PReP Boot   85  Linux extended  c7  Syrinx
 5  Extended        42  SFS             86  NTFS volume set da  Non-FS data
 6  FAT16           4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS       4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility
 8  AIX             4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt
 9  AIX bootable    50  OnTrack DM      93  Amoeba          e1  DOS access
 a  OS/2 Boot Manag 51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O
 b  W95 FAT32       52  CP/M            9f  BSD/OS          e4  SpeedStor
 c  W95 FAT32 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs
 e  W95 FAT16 (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  GPT
 f  W95 Ext'd (LBA) 55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor
12  Compaq diagnost 61  SpeedStor       a9  NetBSD          f4  SpeedStor
14  Hidden FAT16 <3 63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary
16  Hidden FAT16    64  Novell Netware  af  HFS / HFS+      fb  VMware VMFS
17  Hidden HPFS/NTF 65  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  70  DiskSecure Mult b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 75  PC/IX           bb  Boot Wizard hid fe  LANstep
1c  Hidden W95 FAT3 80  Old Minix       be  Solaris boot    ff  BBT
1e  Hidden W95 FAT1
Hex code (type L to list codes):
 <-- fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help):
 <-- t
Partition number (1-4): <-- 2
Hex code (type L to list codes): <-- fd
Changed system type of partition 2 to fd (Linux raid autodetect)

Command (m for help):
 <-- t
Partition number (1-4): <-- 3
Hex code (type L to list codes): <-- fd
Changed system type of partition 3 to fd (Linux raid autodetect)

Command (m for help):
 <-- w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
root@server1:~#

To make sure that there are no remains from previous RAID installations on /dev/sdb, we run the following commands:

mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb2
mdadm --zero-superblock /dev/sdb3

If there are no remains from previous RAID installations, each of the above commands will throw an error like this one (which is nothing to worry about):

root@server1:~# mdadm --zero-superblock /dev/sdb1
mdadm: Unrecognised md component device - /dev/sdb1
root@server1:~#

Otherwise the commands will not display anything at all.

Share this page:

11 Comment(s)

Add comment

Comments

From: al biheiri at: 2011-06-08 12:16:27

Why would you modprobe multipath? I think that is only used for fiber cards. It shouldnt have anything to do with mdadm.

From: paul at: 2011-06-13 17:34:35

Hey.

Great Tutorial! 

Here are a couple of bumps you might run into, if you're trying to use this on Debian Unstable "Wheezy". 
 
 

First
If you want to use GPT enabled harddisks you'll have to create a small (1MB) BIOS Boot Partition or GRUB2 won't install on that drive. 
 
 

Second
If you ARE actually using GUID partition tables you won't be able to change your partitions using fdisk. You'll have to use gdisk (or parted) instead, which is its direct successor and fairly simple to use. Note however, that there's no direct way of duplicating the partition table from one disk to another. You'll have to use sgdisk for that, which installs with gdisk: 
sgdisk --replicate=/dev/sdb /dev/sda

sgdisk --randomize-guids --move-second-header /dev/sdb
The second line is important to distinguish the disks in your system later on. In addition it ensures that your Secondary (Backup) GPT Header really resides on the very last blocks of your disk.
 
 

Third:
This one took me a while :-\
If you're creating the ''/etc/grub.d/09_swraid1_setup'' as mentioned you MUST rename the module GRUB2 is instructed to load from
 
insmod mdraid to insmod mdraid1x

 

(at least for a RAID1 or 10 configuration), as the old-named module does no longer exist. Furthermore, if you don't have a separated /boot/partition you need to fix the path to the kernel and ramdisk (probably "/boot/vmlinuz-2.6…" and "/boot/initrd.img-2.6…"

From: Anonymous at: 2014-10-19 17:04:41

How can I do this on Debian 7? Now I've killed 6 VMs with this Turorial :D

From: Fredrik Falk at: 2013-07-10 19:02:01

Had issues with file not found in grub after reboot into the new array. Tried a couple of things until i found out the md devices werent listed by "ls" in grub rescue prompt. This guide added insmod mdraid in the /etc/grub.d/09_swraid1_setup file but in Debian7 this module doesnt exist. I added mdraid1x instead of mdraid and thereafter it booted fine.

From: at: 2013-11-14 20:23:26

In section:
5 Adjusting The System To RAID1
 
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64' --class debian --class gnu-linux --class gnu --class os {
        insmod raid
        insmod mdraid
        insmod part_msdos
        insmod ext2
        set root='(md/0)'
        echo    'Loading Linux 2.6.32-5-amd64 ...'
        linux   /vmlinuz-2.6.32-5-amd64 root=/dev/md2 ro  quiet
        echo    'Loading initial ramdisk ...'
        initrd  /initrd.img-2.6.32-5-amd64
}

Should be:

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64' --class debian --class gnu-linux --class gnu --class os {
        insmod raid
        insmod mdraid
        insmod part_msdos
        insmod ext2
        set root='(md/0)'
        echo    'Loading Linux 2.6.32-5-amd64 ...'
        linux   /boot/vmlinuz-2.6.32-5-amd64 root=/dev/md2 ro  quiet
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd.img-2.6.32-5-amd64
}

From: Alex at: 2014-06-23 11:23:06


If you are using GPT, then you can use sgdisk to clone the partition table from /dev/sda to the other two hard drives:

# sgdisk --backup=table /dev/sda
# sgdisk -G --load-backup=table /dev/sdb 
 
  -G, --randomize-guids
              Randomize  the  disk’s  GUID  and all partitions’ unique GUIDs (but not their partition type code GUIDs). This function may be used after
              cloning a disk in order to render all GUIDs once again unique.

 
 
Anyway make sure you know what you're doing.

From: kokolorum at: 2012-05-06 11:38:49

Please add this:

apt-get install grub2

dd if=/dev/sda of=/boot/backup.dd.sda

sfdisk -d /dev/sda | sfdisk /dev/sdb

sfdisk -d /dev/sda > /boot/backup.sfdisk.sda

From: simon at: 2012-06-17 23:39:12

Many thanks!

I was stuck without being enable to install RAID1 on a running squeeze install.

From: tv.debian at: 2011-06-08 09:41:50

Hi, thanks for this tutorial. I have been using this kind of manipulation several times, and reading the prompts a few observations:

Using sfdisk to dump the partition layout from the first disk to the second can lead to nasty data corruption, I have experienced it first hand, the raid superblocks and the file-system overlap and lead to all kind of problems. Since the original disk is wiped in the process there is no point doing this anyway, just create empty (no file-system) partitions on the new disk, then create the one disk raid, then format the raid volume. Then the raided disk partition table can safely be used on the second disk.

Alternate: shrink the original partitions by ±1 MB, go on with sfdisk, create your raid, then when everything is done  "resize2fs" the raid volume.

If the raid volume is already created, it's possible to fix it with
"e2fsck -cc", but it's very time consuming. You might want to try it on your setup though, you might be surprised by the result.

Regarding metadata, with 1.2 format which is now the default, it's not needed nor recommended to change the file-system type to "fd" (raid auto-detect). As long as an initramfs is used and mdadm.conf is filled in the system will boot just fine.

Lastly, with the new metadata format and a recent kernel, all raid1 are seen as partitionable, leading to possible shift in volume names under /dev. It's safer to use UUID, or the mdadm "--name=" option to avoid getting confused when "md1" will suddenly become "md1p1" or "md127" after an upgrade (or disaster recovery from a live system). Or force a non-partitionable raid with "--auto=md", should prevent weird raid volumes renaming.

 Have fun.

From: Louis at: 2012-07-24 10:28:34

Nice howto, but 1 sugestions.

dont use /dev/mdX in fstab.

use the command :  blkid

and set the UUID of the raid boot device.

like:

UUID=c3b87084-07c2-4818-b118-6c6764f81545       /                       ext4    defaults,errors=remount-ro       0       1


keep it more error free and universal.

 

From: Patrice Vigier at: 2012-10-31 15:26:05

I could not with this tuto have, in RAID1, the second disk to boot.

The solution came form this :

in file "/etc/default/grub", uncomment the line:

GRUB_TERMINAL=console

after do not forget to run 

update-grub

And it worked