Xen Live Migration Of An LVM-Based Virtual Machine With iSCSI On Debian Lenny - Page 2

4 Making The Shared Storage Available On server1 And server2

server1/server2:

On server1 and server2, we install the initiator:

aptitude install open-iscsi

Next we open /etc/iscsi/iscsid.conf...

vi /etc/iscsi/iscsid.conf

... and set node.startup to automatic:

[...]
node.startup = automatic
[...]

Then we restart the initiator:

/etc/init.d/open-iscsi restart

Now we connect to the target (iscsi.example.com) and check what storage devices it has to offer:

iscsiadm -m discovery -t st -p 192.168.0.102

server1:~# iscsiadm -m discovery -t st -p 192.168.0.102
192.168.0.102:3260,1 iqn.2001-04.com.example:storage.lun1
server1:~#

iscsiadm -m node

server1:~# iscsiadm -m node
192.168.0.102:3260,1 iqn.2001-04.com.example:storage.lun1
server1:~#

The settings for the storage device iqn.2001-04.com.example:storage.lun1 on 192.168.0.102:3260,1 are stored in the file /etc/iscsi/nodes/iqn.2001-04.com.example:storage.lun1/192.168.0.102,3260,1/default. We need to set the username and password for the target in that file; instead of editing that file manually, we can use the iscsiadm command to do this for us:

iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.102:3260" --op=update --name node.session.auth.authmethod --value=CHAP
iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.102:3260" --op=update --name node.session.auth.username --value=someuser
iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.102:3260" --op=update --name node.session.auth.password --value=secret

Now we can log in by running...

iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.102:3260" --login

In the output of

fdisk -l

you should now find a new hard drive; that's our iSCSI storage device (in this example, it's named /dev/sdf on server1 and /dev/sdc on server2):

server1 output:

server1:~# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00023cd1

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          62      497983+  83  Linux
/dev/sda2              63       60801   487886017+   5  Extended
/dev/sda5              63       60801   487885986   8e  Linux LVM

Disk /dev/dm-0: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table

Disk /dev/sdf: 21.4 GB, 21474836480 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Disk /dev/sdf doesn't contain a valid partition table
server1:~#

server2 output:

server2:~# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00036268

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          62      497983+  83  Linux
/dev/sda2              63       60801   487886017+   5  Extended
/dev/sda5              63       60801   487885986   8e  Linux LVM

Disk /dev/dm-0: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table

Disk /dev/sdc: 21.4 GB, 21474836480 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Disk /dev/sdc doesn't contain a valid partition table
server2:~#

To use that device, we must format it (I want to create/use LVM-based virtual machines, therefore I format it as LVM (type 8e)):

server1:

fdisk /dev/sdf

server1:~# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x353f5965.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.


The number of cylinders for this disk is set to 20480.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help):
 <-- n
Command action
   e   extended
   p   primary partition (1-4)

<-- p
Partition number (1-4): <-- 1
First cylinder (1-20480, default 1): <-- ENTER
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-20480, default 20480):
 <-- ENTER
Using default value 20480

Command (m for help):
 <-- t
Selected partition 1
Hex code (type L to list codes):
 <-- L

 0  Empty           1e  Hidden W95 FAT1 80  Old Minix       be  Solaris boot
 1  FAT12           24  NEC DOS         81  Minix / old Lin bf  Solaris
 2  XENIX root      39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 5  Extended        41  PPC PReP Boot   85  Linux extended  c7  Syrinx
 6  FAT16           42  SFS             86  NTFS volume set da  Non-FS data
 7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility
 9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt
 a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e1  DOS access
 b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O
 c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          e4  SpeedStor
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  EFI GPT
10  OPUS            55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f4  SpeedStor
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fd  Linux raid auto
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fe  LANstep
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid ff  BBT
1c  Hidden W95 FAT3 75  PC/IX
Hex code (type L to list codes):
 <-- 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help):
 <-- w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
server1:~#

Afterwards, the output of

fdisk -l

should look as follows:

server1:~# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00023cd1

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          62      497983+  83  Linux
/dev/sda2              63       60801   487886017+   5  Extended
/dev/sda5              63       60801   487885986   8e  Linux LVM

Disk /dev/dm-0: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table

Disk /dev/sdf: 21.4 GB, 21474836480 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x353f5965

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1       20480    20971504   8e  Linux LVM
server1:~#

Since this is a shared storage, /dev/sdc on server2 should now also contain an LVM partition, /dev/sdc1:

server2:

fdisk -l

server2:~# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00036268

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          62      497983+  83  Linux
/dev/sda2              63       60801   487886017+   5  Extended
/dev/sda5              63       60801   487885986   8e  Linux LVM

Disk /dev/dm-0: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table

Disk /dev/sdc: 21.4 GB, 21474836480 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x353f5965

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       20480    20971504   8e  Linux LVM
server2:~#

Now I initialize /dev/sdf1 on server1 for LVM usage and create the volume group vg_xen on it:

server1:

pvcreate /dev/sdf1

vgcreate vg_xen /dev/sdf1

In order to make the new volume group available on server2, we must first log out of iSCSI and then back in:

server2:

iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.102:3260" --logout
iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.102:3260" --login

Then run...

vgscan

server2:~# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "vg_xen" using metadata type lvm2
  Found volume group "vg0" using metadata type lvm2
server2:~#

... and...

vgchange -a y

server2:~# vgchange -a y
  0 logical volume(s) in volume group "vg_xen" now active
  2 logical volume(s) in volume group "vg0" now active
server2:~#

... to activate the vg_xen volume group on server2.

Share this page:

11 Comment(s)

Add comment

Comments

From: Anonymous at: 2009-10-07 02:14:02

http://www.kernsafe.com/Article_Product.aspx?id=5&&aid=22

This article demonstrate XEN Server how to work with windows iSCSI Target, may be helpful for us.

 

From: Anonymous at: 2010-07-03 18:33:07

KernSafe is not iSCSI target. It's complete JUNK :((

From: Ichiro Arai at: 2010-08-13 20:45:09

Free StarWind iSCSI Target is now Citrix Xen certified! 

http://hcl.xensource.com/BrowsableStorageList.aspx

and

http://www.starwindsoftware.com/news/36 

 Arigato!

 -ichiro

 

From: Anonymous at: 2010-08-27 10:41:54

starwind is not free, the stupid crook.

I've found a lot of bugs, it is very unstable.

From: Ichiro at: 2010-08-30 15:56:24

Oh, Mr. Aldrich from KernSafe :) Your self-promotion is very impressive :) Good luck selling your "Made in China" software. Hope your'll get prosperity.

 -ichiro

From: sam at: 2009-05-03 10:43:28

Hi there, thanks for such an excellent article falko. I wondered do you have an article on converting a physical Debian Etch machine, to a Xen VM – not necessarily a live migration, but perhaps a best practice guide for doing this. Am wanting to take an overspecd 1u server which is a LAMP server, and move this to a Xen VM on another machine altogether. I frequently look at this site, and have found it invaluable. kind regards Sam

From: falcon at: 2009-04-28 22:53:24

Hello Thanks for the howto

However the URL to xensource is no longer valid. xensource.com redirects you to the citrix website

From: Anonymous at: 2010-06-14 22:56:08

Hi There,

Is it safe to connect 2 clients to a single iSCSI target?

From: alxgomz at: 2010-07-17 00:52:16

I have almost the same set up with Fibre Channel instead of iSCSI and your question is a good question!

Connecting two hosts to a shared storage is not a problem by itself. You have to make sure you filesystem (if you use direct filesystem) is cluster aware.

With block devices (iSCSI, FC, AoE) it's the same thing.

Unless you use clvm, LVM is not cluster aware!

Using clvm instead is a mess it requires redhat clustering suite or openais (much more simple to install), but none of thoose 2 cluster interface are stable enough (at least their API with LVM) to allow efficient administration (often LV operations just hangs so you have to restart openais)... This is a pity but things are like this... Further more using cLVM forbids creation of snapshots which is a really usefull feature in virtualization environments!

 As a workaround you can do what Falko and I did... use non-cluster aware LVM in a clustered environment... but in this case you have to be reaaaaaaaaally carefull with what you or may easyly loose data!

From: Daniel Bojczuk at: 2010-11-10 19:56:20

Hi... I'm trying to use OCFS2 or GFS on my Gentto+Xen, but I'm having trouble with both o them.  I'm surprise that I can use LVM intead a clustered filesystem. alxgomz wrote: "... but in this case you have to be reaaaaaaaaally carefull with what you or may easyly loose data!"

 Can you explain more about this? What can I need to do to never loose data?

 Many thanks,

From: Wiebe Cazemier at: 2010-12-03 13:33:38

Doesn't that mean that if you don't make 100% sure all vg's and lv's are known on the machine you're going to perform an LVM command on (like lvcreate), you can mess up your volumes?

I mean, if server2 is not aware of the recenlty made LV made with server1, and you do lvcreate on server2, it can create it in used space in the VG, right?