A Beginner's Guide To LVM - Page 6

6 Return To The System's Original State

In this chapter we will undo all changes from the previous chapters to return to the system's original state. This is just for training purposes so that you learn how to undo an LVM setup.

First we must unmount our logical volumes:

umount /var/share
umount /var/backup
umount /var/media

df -h

server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              19G  665M   17G   4% /
tmpfs                  78M     0   78M   0% /lib/init/rw
udev                   10M   92K   10M   1% /dev
tmpfs                  78M     0   78M   0% /dev/shm
/dev/sda1             137M   17M  114M  13% /boot

Then we delete each of them:

lvremove /dev/fileserver/share

server1:~# lvremove /dev/fileserver/share
Do you really want to remove active logical volume "share"? [y/n]:
 <-- y
  Logical volume "share" successfully removed

lvremove /dev/fileserver/backup

server1:~# lvremove /dev/fileserver/backup
Do you really want to remove active logical volume "backup"? [y/n]:
 <-- y
  Logical volume "backup" successfully removed

lvremove /dev/fileserver/media

server1:~# lvremove /dev/fileserver/media
Do you really want to remove active logical volume "media"? [y/n]:
 <-- y
  Logical volume "media" successfully removed

Next we remove the volume group fileserver:

vgremove fileserver

server1:~# vgremove fileserver
  Volume group "fileserver" successfully removed

Finally we do this:

pvremove /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1

server1:~# pvremove  /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
  Labels on physical volume "/dev/sdc1" successfully wiped
  Labels on physical volume "/dev/sdd1" successfully wiped
  Labels on physical volume "/dev/sde1" successfully wiped
  Labels on physical volume "/dev/sdf1" successfully wiped

vgdisplay

server1:~# vgdisplay
  No volume groups found

pvdisplay

should display nothing at all:

server1:~# pvdisplay

Now we must undo our changes in /etc/fstab to avoid that the system tries to mount non-existing devices. Fortunately we have made a backup of the original file that we can copy back now:

mv /etc/fstab_orig /etc/fstab

Reboot the system:

shutdown -r now

Afterwards the output of

df -h

should look like this:

server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              19G  666M   17G   4% /
tmpfs                  78M     0   78M   0% /lib/init/rw
udev                   10M   92K   10M   1% /dev
tmpfs                  78M     0   78M   0% /dev/shm
/dev/sda1             137M   17M  114M  13% /boot

Now the system is like it was in the beginning (except that the partitions /dev/sdb1 - /dev/sdf1 still exist - you could delete them with fdisk but we don't do this now - as well as the directories /var/share, /var/backup, and /var/media which we also don't delete).

 

7 LVM On RAID1

In this chapter we will set up LVM again and move it to a RAID1 array to guarantee for high-availability. In the end this should look like this:

This means we will make the RAID array /dev/md0 from the partitions /dev/sdb1 + /dev/sdc1, and the RAID array /dev/md1 from the partitions /dev/sdd1 + /dev/sde1. /dev/md0 and /dev/md1 will then be the physical volumes for LVM.

Before we come to that, we set up LVM as before:

pvcreate /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
vgcreate fileserver /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
lvcreate --name share --size 40G fileserver
lvcreate --name backup --size 5G fileserver
lvcreate --name media --size 1G fileserver

mkfs.ext3 /dev/fileserver/share
mkfs.xfs /dev/fileserver/backup
mkfs.reiserfs /dev/fileserver/media

Then we mount our logical volumes:

mount /dev/fileserver/share /var/share
mount /dev/fileserver/backup /var/backup
mount /dev/fileserver/media /var/media

The output of

df -h

should now look like this:

server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              19G  666M   17G   4% /
tmpfs                  78M     0   78M   0% /lib/init/rw
udev                   10M   92K   10M   1% /dev
tmpfs                  78M     0   78M   0% /dev/shm
/dev/sda1             137M   17M  114M  13% /boot
/dev/mapper/fileserver-share
                       40G  177M   38G   1% /var/share
/dev/mapper/fileserver-backup
                      5.0G  144K  5.0G   1% /var/backup
/dev/mapper/fileserver-media
                      1.0G   33M  992M   4% /var/media

Now we must move the contents of /dev/sdc1 and /dev/sde1 (/dev/sdc1 is the second partition of our future /dev/md0, /dev/sde1 the second partition of our future /dev/md1) to the remaining partitions, because we will afterwards remove them from LVM and format them with the type fd (Linux RAID autodetect) and move them to /dev/md0 resp. /dev/md1.

modprobe dm-mirror
pvmove /dev/sdc1

vgreduce fileserver /dev/sdc1
pvremove /dev/sdc1

pvdisplay

server1:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               fileserver
  PV Size               23.29 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              5961
  Free PE               0
  Allocated PE          5961
  PV UUID               USDJyG-VDM2-r406-OjQo-h3eb-c9Mp-4nvnvu

  --- Physical volume ---
  PV Name               /dev/sdd1
  VG Name               fileserver
  PV Size               23.29 GB / not usable 0
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              5961
  Free PE               4681
  Allocated PE          1280
  PV UUID               qdEB5d-389d-O5UA-Kbwv-mn1y-74FY-4zublN

  --- Physical volume ---
  PV Name               /dev/sde1
  VG Name               fileserver
  PV Size               23.29 GB / not usable 0
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              5961
  Free PE               1426
  Allocated PE          4535
  PV UUID               4vL1e0-sr2M-awGd-qDJm-ZrC9-wuxW-2lEqp2

pvmove /dev/sde1

vgreduce fileserver /dev/sde1
pvremove /dev/sde1

pvdisplay

server1:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               fileserver
  PV Size               23.29 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              5961
  Free PE               0
  Allocated PE          5961
  PV UUID               USDJyG-VDM2-r406-OjQo-h3eb-c9Mp-4nvnvu

  --- Physical volume ---
  PV Name               /dev/sdd1
  VG Name               fileserver
  PV Size               23.29 GB / not usable 0
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              5961
  Free PE               146
  Allocated PE          5815
  PV UUID               qdEB5d-389d-O5UA-Kbwv-mn1y-74FY-4zublN

Now we format /dev/sdc1 with the type fd (Linux RAID autodetect):

fdisk /dev/sdc

server1:~# fdisk /dev/sdc

The number of cylinders for this disk is set to 10443.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help):
 <-- m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help):
 <-- t
Selected partition 1
Hex code (type L to list codes):
 <-- L

 0  Empty           1e  Hidden W95 FAT1 80  Old Minix       be  Solaris boot
 1  FAT12           24  NEC DOS         81  Minix / old Lin bf  Solaris
 2  XENIX root      39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 5  Extended        41  PPC PReP Boot   85  Linux extended  c7  Syrinx
 6  FAT16           42  SFS             86  NTFS volume set da  Non-FS data
 7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility
 9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt
 a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e1  DOS access
 b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O
 c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          e4  SpeedStor
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  EFI GPT
10  OPUS            55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f4  SpeedStor
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fd  Linux raid auto
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fe  LANstep
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid ff  BBT
1c  Hidden W95 FAT3 75  PC/IX
Hex code (type L to list codes):
 <-- fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help):
 <-- w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Now do the same with /dev/sde1:

fdisk /dev/sde

The output of

fdisk -l

should now look like this:

server1:~# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          18      144553+  83  Linux
/dev/sda2              19        2450    19535040   83  Linux
/dev/sda4            2451        2610     1285200   82  Linux swap / Solaris

Disk /dev/sdb: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        3040    24418768+  8e  Linux LVM

Disk /dev/sdc: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        3040    24418768+  fd  Linux raid autodetect

Disk /dev/sdd: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1        3040    24418768+  8e  Linux LVM

Disk /dev/sde: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1        3040    24418768+  fd  Linux raid autodetect

Disk /dev/sdf: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1        3040    24418768+  8e  Linux LVM

Next we add /dev/sdc1 to /dev/md0 and /dev/sde1 to /dev/md1. Because the second nodes (/dev/sdb1 and /dev/sdd1) are not ready yet, we must specify missing in the following commands:

mdadm --create /dev/md0 --auto=yes -l 1 -n 2 /dev/sdc1 missing

server1:~# mdadm --create /dev/md0 --auto=yes -l 1 -n 2 /dev/sdc1 missing
mdadm: array /dev/md0 started.

mdadm --create /dev/md1 --auto=yes -l 1 -n 2 /dev/sde1 missing

server1:~# mdadm --create /dev/md1 --auto=yes -l 1 -n 2 /dev/sde1 missing
mdadm: array /dev/md1 started.

Share this page:

47 Comment(s)

Add comment

Comments

From: at: 2007-01-16 12:59:36

First of all i'll shall congratulate you for the great guide.

I'll rather call it a "Introduction Guide" than a "Beginner Guide" , never than less it's very usefull.

Instead of having LVM on top ou those 2 RAID-1 devices and considering the disks capacity, you can use 4 disk RAID-5 system thus have more 25% usable space.

This will make the process more complex but you will be rewarded with more 80GB ;)

This must be done after you replace the first 2 Harddrives.

  • Initilize only one disk, let's say /dev/sdc
    • pvcreate /dev/sdc
  • Add the 80GB disk to the volume
    • vgextend fileserver /dev/sdc
  • pvmove all all volumes from the md[01] devices to the 80GB disk
    • pvmove /dev/md0 /dev/md1
      • note: this is very slow better use -v for periodic update
  • Remove all other devices from the volume
    • vgreduce fileshare /dev/md0 /dev/md1
  • Reboot and replace the disks
  • Initialize the new disks for raid
    • fdisk /dev/sdb
    • fdisk /dev/sdd
    • fdisk /dev/sde 
  • create the raid-5 with one missing device
    • mdadm --create /dev/md0 -a -l 5 -n 4 /dev/sdb1 /dev/sdd1 /dev/sde1 missing
  • Add the new md0 device to the Volume
    • pvcreate /dev/md0 && vgextend fileserver /dev/md0
  • Move the data from the 80GB disk
    • pvmove /dev/sdc
  • (wait)
  • Remove the 80GB disk from the volume group
    • vgreduce fileshare /dev/sdc
  • Initialize the disk for RAID
    • fdisk /dev/sdc and change the type to fd (Linux raid autodetect)
  • Add the disk to the RAID md0
    • mdadm --manage /dev/md0 -add /dev/sdc1
  • Wait for full sync
    • cat /proc/mdstat
  • And you are now with a 240GB RAID-5 volume
    • df -h

A 4 disk RAID-5 is not as performant as the RAID-1 but that's the trade off .

 

José Borges Ferreira 

From: lingeswaran at: 2013-08-14 19:00:07


Step by Step Tutorial available in UnixArena.

 http://www.unixarena.com/2013/08/how-to-install-lvm-on-linux-and-disk.html

http://www.unixarena.com/2013/08/linux-lvm-volume-group-operations.html

http://www.unixarena.com/2013/08/linux-lvm-volume-creation-operation.html 

 

From: at: 2007-01-16 13:06:55

Be aware that when you initialize a device into a Volume or into a md RAID some unique IDs are assign and written into the first sector of that device. When you do some testing on some virtual enviorment such as VMWare you may ran into this problem. So as a part of the initilization process you better do a

#dd if=/dev/zero of=/dev/diskname bs=1k count=1
#blockdev --rereadpt /dev/sdc

before everything else.

 

José Borges Ferreira 

From: at: 2007-01-18 09:55:22

Source /dev/sda, destination /dev/sdb

 sfdisk -d /dev/sda|sfdisk /dev/sdb

From: at: 2007-01-19 17:00:40

I'm very sorry if I overlooked a note or a posting on this, but how do I set the CLI keyboard layout to qwerty (us 101/104) on Debian Etch.

I immediately ran into problems, it seems your vmware image was made using a german keyboard layout (?)

 

Thanx! 

From: admin at: 2007-01-20 21:11:31

Run

apt-get install console-data console-tools debconf
dpkg-reconfigure console-data

or connect ot the virtual machine with an SSH client such as PuTTY. In PuTTY you use your client machine's keymap.

From: tonyg at: 2009-12-06 05:23:18

I just wanted to say THANK YOU for this resource.  I've been referring back to this article for the past 2 years now, it's saved my butt, and my data, a few times now.  Thanks!!!

From: Sun_Blood at: 2011-02-16 18:36:25

Just one word. GREAT!

This was a perfect start for me to learn on hot to use LVM. Now I'll setup my new NAS =)

From: Anonymous at: 2011-08-30 15:48:07

Out of the 6 drives on the image - drives 3 and 4 appear to be corrupt on my VM VirtualBox Manager.

From: Mark at: 2012-10-14 12:18:44

What a great introduction to LVM!  Thank you so much for taking the trouble to put all this together.

From: Ramesh at: 2013-11-06 13:29:05

Thank you very much for the Excellent article. I appreciate your effort. 

From: Anonymous at: 2013-11-13 23:14:48

Thank you for this guide.  I just ran into lvm at work and this is extremely helpful.
I am trying out the vm you provided for practice.  Login info in howtoforge is incorrect.

the user is: root
password : howtoforge

 

 

From: Anonymous at: 2014-01-22 16:20:13

I wanted to say thank you for the great and useful guide. On the internet we should find articles like this. Well done!!!

From: pointer2null at: 2014-12-21 22:17:39

I've just had a quick read of the tutorial and will run through it soon.

One thing I do notice is you give very clear instructions on how to execute each stage, but no explanation of why it is being done( and to a smaller degree, or what is accomplished in each step).

 

Still, it's a valuable resource. :)

From: Anonymous at: 2014-12-25 09:11:43

Try to use EasyRSH in Google  Play - it's quick reference guide for Solaris, HPUX, Redhat OSs

From: Anonymous at: 2010-08-18 19:50:40

If you get this error, you'll need to "deregister" the partition table from the kernel.

 kpartx -d /dev/fileserver/films

 lvremove /dev/fileserver/films

From: Andre de Araujo at: 2013-12-03 21:45:09

Correct is: #lvextend -L  +1.5G  /dev/fileserver/media

From: Adrian Varan at: 2014-02-08 21:00:40

"+" is optional (read the manual). If you use +1.5G then the 1.5G is added to the actual size (1.5+1=2.5G), without "+" the 1.5G represents the new absolute value of the logical volume.

From: Anonymous at: 2010-05-03 14:31:21

Great guide!

Thanks a lot - helped me out :-)

From: Navin Pathak at: 2011-03-17 12:55:15

Dear Freinds,

I have started learn linux from few days and now days I am learning LVM I have search a lot of document and finaly choose your site and start working today through your guide line for LVM I have completed today near entry to fstab of logical valume so I feel very well with your documents.

Thanks a lot you all who spend a time to cretae such a nice lvm real practical.

my one suggetion is that please explain the term of PE,LE and metadata.

again thanks.

 

Regards

Navin Pathak

TTSL India.

From: SN at: 2011-04-14 07:54:26

There's a Zimbra backup script based on LVM, I have no idea of LVM so I searched and found this amazing topic. Thanks so much for your work.

Regards,

SN

From: jonathan young at: 2012-01-29 05:39:55

This guide is so idiot-proofed and full of explanations.  Thank you so much, you saved my bacon. I am a beginning linux administrator (as a sideline to being a web architect) and LVM is so brand new to me, I was scared to resize lv's and now i'm like "wow, this is easy" 

 thank you so much!

 

 

From: acname at: 2012-12-09 09:26:45

perfect manual. thanx a lot

From: Anonymous at: 2014-05-05 00:16:53

need to not use if fdisk if drive is over 2 TB though.

From: Sebastian at: 2014-12-15 18:35:55

Thank you very much for this though tutorial. Helped alot!

From: Robert at: 2008-11-06 23:49:50

Bloody well excellent lvm2 guide.

Thank You.

From: Chris at: 2009-09-26 03:58:05

Hi

nice guide, and the vmware image is a great idea.

in your first RAID example, it looks like you've missed some of the pvmove arguments (it just has the source volume, not the dest volume).

cheers

 

 

From: Anonymous at: 2011-08-09 06:41:12

Apparently not - I was confused about that too at first, but actually working through the tutorial confirmed that this is not the case.

A quick check of the LVM docs reveals that pvmove with no arguments (other than the device) moves all the data on the device to free space in the volume group, wherever it can find it.

It's basically "move this data to anywhere else" as opposed to "move this data to this particular place" which is what we were doing with the previous uses of pvmove.

From: ilayaraja at: 2010-02-25 12:41:52

very very usefull for beginers

From: oldtimer_mando at: 2012-08-17 04:59:35

Awesome!  Thanks!

From: Imran at: 2013-06-04 12:49:54

really a very nice and useful guide for beginner, Thanks you so much

 

 

From: Sajid at: 2014-07-18 22:35:29

Excellent details and easy to follow, great work!

From: MTH at: 2009-10-14 02:40:08

Fantastic guide, covers many scenarios (adding drives, removing drives, resizing, etc).  I find myself always coming back to double check my LVM setups.  A+

From: at: 2007-01-19 06:16:45

Great howto, Falko.

I have needed this in the past and i have already bookmarked it for the next time.  I just don't work withthis stuff enough to memorize it.

You have a real talent for technical writing. 

Thanks,

From: at: 2008-09-13 06:30:55

This was exactly what I needed to get my home file server running on LVM.  I will need this again when I add disks and again when I move everything over to raid.

From: Tormod at: 2008-10-05 11:13:55

Excellent howto! I just noticed that the example fstab entries look wrong (in both examples): /dev/fileserver/share versus /dev/mapper/fileserver-share

From: Tormod at: 2008-10-06 19:35:07

Well, scratch that. It is correct anyway, silly me just had to try it out to see: /dev/fileserver/share is a soft link to /dev/mapper/fileserver-share

From: Anonymous at: 2008-10-05 07:34:30

I have about 2 years of experience using RAID and LVM, and I must say - in all of the literature and documentation I've ever encountered, _none_ of it ever came close to making things so simple and clear as you have just done. You've articulated the ideas of logical volumes, volume groups, and physical volumes well, and have provided concise examples.

Well done.

From: Craig at: 2011-05-26 16:39:00

One of the best howto I have come across -- wish they were all this good.

From: Gisli at: 2012-09-04 15:32:04

I agree with everyone here. Best howto I've come accross! Everything right to the point and with examples. Nice work!!

From: Vahid Pazirandeh at: 2011-07-01 18:50:57

Wow. Very well written howto. Well thought out examples. Thanks a lot to all who were involved.

I agree with an earlier comment - I have used Linux for many years and have read through lots of tutorials. This was so easy to read! :)

From: Anonymous at: 2011-06-30 04:11:11

In my 15 years with linux I have never, ever, seen such good howto. Very easy to follow and understand. In 30 mins my confidence level on LVM/RAID was boosted from 0 to 80.

 I wish there were more howtos like this!

From: csg at: 2011-09-16 12:46:18

Congratulations for this hard work, very clear and concise.

Looking forward to have it running.

Thanks for your work.

From: Gianluca at: 2012-01-01 21:15:07

Excellent HowTo, pls complete this guide with LVM snapshot examples.

From: Anonymous at: 2014-02-17 15:08:20

excellent tutorial that briefs how to manage disks in linux platforms..Thanks for your effort to have this tutorial get prepared..

From: albert gharbi at: 2015-03-18 14:52:08

Thank a lots. very excellent.that was very usefull and practical.

From: Rich at: 2015-03-31 07:33:42

Excellent tuorial.  There seems to be so little 'easy' documentation out there for managing disks and LVs in Linux. This tutorial is perfect for those that want to get started and understand the process rather than copy/paste snippets from other forums.  Great job!