Software raid. Two drives ok. One drive does not boot up.

Discussion in 'Technical' started by pvanthony, Sep 7, 2013.

  1. pvanthony

    pvanthony Member

    Hi,

    Did the following.
    1. Shutdown the server and replaced /dev/sdb with a new blank hard drive.

    2. Power up the server and I am getting the following error and does not boot up.

    mount: you must specify the filesystem type
    kernel panic - not syncing attempted to kill init

    3. Shutdown the server and put back the old hard drive and all is ok.

    What did I miss out? Need advice on how to solve this.


    Here are some details about the server.


    Grub.
    --------- start --------------------
    default=0
    timeout=5
    splashimage=(hd0,0)/grub/splash.xpm.gz
    hiddenmenu
    serial --unit=0
    terminal serial
    title CentOS (2.6.32-279.19.1.el6.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.19.1.el6.x86_64 ro root=/dev/md1 rd_NO_LUKS rd_NO_LVM rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet console=ttyS0
    initrd /initramfs-2.6.32-279.19.1.el6.x86_64.img
    title CentOS (2.6.32-279.11.1.el6.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-279.11.1.el6.x86_64 ro root=/dev/md1 rd_NO_LUKS rd_NO_LVM rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet console=ttyS0
    initrd /initramfs-2.6.32-279.11.1.el6.x86_64.img
    ----------- end --------------------



    Check if the grub is in both drives.
    ------------ start ----------------
    [root ~]# dd bs=512 count=1 if=/dev/sda 2>/dev/null | strings
    ZRrI
    D|f1
    GRUB
    Geom
    Hard Disk
    Read
    Error

    [root ~]# dd bs=512 count=1 if=/dev/sdb 2>/dev/null | strings
    ZRrI
    D|f1
    GRUB
    Geom
    Hard Disk
    Read
    Error
    ------------- end -----------------



    mdstat.
    ---------- start ------------------
    [root ~]# cat /proc/mdstat
    Personalities : [raid1]
    md1 : active raid1 sda3[0] sdb3[2]
    972646232 blocks super 1.2 [2/2] [UU]

    md0 : active raid1 sda1[0] sdb1[1]
    104320 blocks [2/2] [UU]

    unused devices: <none>

    ----------- end -------------------

    mdadm.
    ------------ start ---------------------
    [root ~]# mdadm --examine --scan
    ARRAY /dev/md0 UUID=353b8766:d1f48983:6b357bf1:4e91e489
    ARRAY /dev/md/1 metadata=1.2 UUID=a4c3c6cb:0f1a7b1e:e1b463da:06d1cd0d name=hostname.domain.com.sg:1
    -------------- end ---------------------


    fstab.
    ------------- start -------------------
    /dev/md1 / ext4 defaults,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 1 1
    /dev/md0 /boot ext3 defaults 1 2
    /dev/sda2 swap swap defaults 0 0
    /dev/sdb2 swap swap defaults 0 0
    tmpfs /dev/shm tmpfs defaults 0 0
    devpts /dev/pts devpts gid=5,mode=620 0 0
    sysfs /sys sysfs defaults 0 0
    proc /proc proc defaults 0 0

    -------------- end --------------------


    fdisk.
    ------------- start ---------------------
    [root ~]# fdisk -l

    Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
    255 heads, 63 sectors/track, 121601 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000

    Device Boot Start End Blocks Id System
    /dev/sda1 1 13 104391 fd Linux raid autodetect
    /dev/sda2 14 512 4008217+ 82 Linux swap / Solaris
    /dev/sda3 513 121601 972647392+ fd Linux raid autodetect

    Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
    255 heads, 63 sectors/track, 121601 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000

    Device Boot Start End Blocks Id System
    /dev/sdb1 1 13 104391 fd Linux raid autodetect
    /dev/sdb2 14 512 4008217+ 82 Linux swap / Solaris
    /dev/sdb3 513 121601 972647392+ fd Linux raid autodetect

    Disk /dev/md0: 106 MB, 106823680 bytes
    2 heads, 4 sectors/track, 26080 cylinders
    Units = cylinders of 8 * 512 = 4096 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000


    Disk /dev/md1: 996.0 GB, 995989741568 bytes
    2 heads, 4 sectors/track, 243161558 cylinders
    Units = cylinders of 8 * 512 = 4096 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000

    --------------- end ----------------------


    df -h
    ------------ start ----------------------
    [root ~]# df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/md1 914G 148G 719G 18% /
    tmpfs 7.8G 0 7.8G 0% /dev/shm
    /dev/md0 99M 64M 31M 68% /boot

    -------------- end ----------------------



    uname -r
    ----------- start ------------------
    [root ~]# uname -r
    2.6.32-279.19.1.el6.x86_64

    Using CentOS 6 64bit.
    ------------- end ------------------
     
  2. pvanthony

    pvanthony Member

    Just found out the UUID do not match also. Could this be the problem?

    ---------- start ------------------
    [root ~]$ blkid
    /dev/sda1: LABEL="/boot" UUID="7c7009d3-9e60-4b3c-8c1b-90d33d54d97f" TYPE="ext3"
    /dev/sda2: LABEL="SWAP-sda2" UUID="a0cc1000-9783-4926-9202-843463a06fbe" TYPE="swap"
    /dev/sda3: LABEL="/" UUID="b3d2736e-5048-409e-bffa-f52a9af089fc" TYPE="ext3"
    ----------- end -------------------

    ---------- start ---------------------
    [root ~]$ ls -l /dev/disk/by-uuid/
    total 0
    lrwxrwxrwx 1 root root 9 Sep 6 03:18 85a5e188-cbfd-4376-8531-f8c598e876c8 -> ../../md1
    lrwxrwxrwx 1 root root 10 Sep 6 03:16 86f84624-b3e7-4a01-bac2-95e70aa3d513 -> ../../sda2
    lrwxrwxrwx 1 root root 9 Sep 6 03:18 e00f19e1-15b5-4f3e-bba1-3320b3c3d19c -> ../../md0
    ---------- end -----------------------

    --------- start ---------------
    [root ~]# mdadm --examine --scan
    ARRAY /dev/md0 UUID=353b8766:d1f48983:6b357bf1:4e91e489
    ARRAY /dev/md/1 metadata=1.2 UUID=a4c3c6cb:0f1a7b1e:e1b463da:06d1cd0d name=hostname.domain.com.sg:1

    ---------- end ----------------
     
  3. pvanthony

    pvanthony Member

    Please help with advice and experience.

    P.V.Anthony
     
  4. jasonorland88

    jasonorland88 New Member

    While investigating, just found out the UUID do not match also. Could this be the problem that the raid could not boot up without one drive?

    ---------- start ------------------
    [root ~]$ blkid
    /dev/sda1: LABEL="/boot" UUID="7c7009d3-9e60-4b3c-8c1b-90d33d54d97f" TYPE="ext3"
    /dev/sda2: LABEL="SWAP-sda2" UUID="a0cc1000-9783-4926-9202-843463a06fbe" TYPE="swap"
    /dev/sda3: LABEL="/" UUID="b3d2736e-5048-409e-bffa-f52a9af089fc" TYPE="ext3"
    ----------- end -------------------

    ---------- start ---------------------
    [root ~]$ ls -l /dev/disk/by-uuid/
    total 0
    lrwxrwxrwx 1 root root 9 Sep 6 03:18 85a5e188-cbfd-4376-8531-f8c598e876c8 -> ../../md1
    lrwxrwxrwx 1 root root 10 Sep 6 03:16 86f84624-b3e7-4a01-bac2-95e70aa3d513 -> ../../sda2
    lrwxrwxrwx 1 root root 9 Sep 6 03:18 e00f19e1-15b5-4f3e-bba1-3320b3c3d19c -> ../../md0
    ---------- end -----------------------

    --------- start ---------------
    [root ~]# mdadm --examine --scan
    ARRAY /dev/md0 UUID=353b8766:d1f48983:6b357bf1:4e91e489
    ARRAY /dev/md/1 metadata=1.2 UUID=a4c3c6cb:0f1a7b1e:e1b463da:06d1cd0d name=hostname.domain.com.sg:1

    ---------- end ----------------
     
  5. Djamu

    Djamu New Member

    For sake of completeness, and preventing others from using OP's wrong commands.

    I assume OP used bs=512 for copying the MBR
    Code:
    dd bs=512 count=1 if=/dev/sda 2>/dev/null | strings
    no wonder it doesn't work the mbr is only 446 bytes long followed by 64 bytes partition table and 2 bytes signature.

    the correct way to prepare a new bootable raid1 disk
    1. first to make sure your partitiontables are identical.
    2. enable boot flag with fdisk cfdisk or whatever
    3. use :
    Code:
    dpkg-reconfige grub-pc
    and select all raw hd's ( not the mdadm ones )
    4. update initramfs
     

Share This Page