E. iSCSI
iSCSI is a Storage Area Network protocol allowing shared storage going through an exising network infrastructure. In my setup, I used iscsitarget from http://iscsitarget.sourceforge.net.
1. iSCSI server installation and configuration
1.a compiling the iscsi application tarball
This needs to be done on the physical host.
- Get the tarball from SourceForge and put it in /usr/local/src.
- cd to /usr/local/src:
cd /usr/local/src
- Then extract the files:
tar xvf iscsitarget-0.4.16.tar.gz
cd iscsitarget-0.4.16 - Then run:
make
make install
1.b configuration needed
This is my ietd.conf configuration defining the "LUNs" to be allocated to the guests from the physical host's disks:
#/etc/ietd.conf # NOTE: the config files has more entries than what i'm showing here. # but i've commented out the original entries and made the following Target iqn.2008-07.NODE00:LUN01.NODE00 MaxConnections 2 Lun 1 Path=/dev/Virtual00VG/lvLUN01,Type=fileio Alias LUN01 Target iqn.2008-07.NODE00:LUN02.NODE00 MaxConnections 2 Lun 2 Path=/dev/Virtual00VG/lvLUN02,Type=fileio Alias LUN02 # end of ietd.conf
In my physical host system, I have created two logical volumes 50G each in size. You can also use files or disk partitions, just change the Path entries in the ietd.conf file.
1.c ACL
iscsitarget has /etc/initiators.allow and /etc/initiators.deny that work like hosts.allow and hosts.deny. In my setup, I will allow node01 and node02 to access the two LUNs defined in ietd.conf.
#/etc/initiators.allow #this should correspond to the definition in your /etc/ietd.conf iqn.2008-07.NODE00:LUN01.NODE00 192.168.100.10, 192.168.100.20 iqn.2008-07.NODE00:LUN02.NODE00 192.168.100.10, 192.168.100.20 # endof initiators.allow
- Start the iscsi-target service:
service iscsi-target start
- and make sure it starts during bootup:
chkconfig --add iscsi-target
chkconfig iscsi-target on
chkconfig --list iscsi-targetiscsi-target 0:off 1:off 2:on 3:on 4:on 5:on 6:off
2 Client Side
The package iscsi-initiator-utils-6.2.0.865-0.8.el5 should already be installed (as it is included in the kickstart file above).
2.a configuration
- Edit the file /etc/iscsi/initiatorname.iscsi to define the targets.
- My /etc/iscsi/initiatorname.iscsi is as follows:
#/etc/iscsi/initiatorname.iscsi InitiatorName=iqn.2008-07.NODE00:LUN01.NODE00 InitiatorName=iqn.2008-07.NODE00:LUN02.NODE00 # end of #/etc/iscsi/initiatorname.iscsi
- Run iscsid service and try to discover the LUNs:
service iscsid start
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
iscsiadm -m discovery -t st -p node00
192.168.222.1:3260,1 iqn.2008-07.NODE00:LUN01.NODE00
192.168.222.1:3260,1 iqn.2008-07.NODE00:LUN02.NODE00 - Then start the iscsi service. You'll then see the LUN definitions created earlier:
service iscsi start
will then show the following:
iscsid (pid 964 963) is running... Setting up iSCSI targets: Login session [iface: default, target: \ iqn.2008-07.NODE00:LUN02.NODE00, portal: 192.168.222.1,3260] Login session [iface: default, target: iqn.2008-07.NODE00:LUN01.\ NODE00, portal: 192.168.222.1,3260] [ OK ]
- Check system logs to see if the disks have been seen:
dmesg
scsi0 : iSCSI Initiator over TCP/IP
Vendor: IET Model: VIRTUAL-DISK Rev: 0 Type: Direct-Access ANSI SCSI revision: 04 scsi 0:0:0:2: Attached scsi generic sg0 type 0 SCSI device sda: 104857600 512-byte hdwr sectors (53687 MB) sda: Write Protect is off sda: Mode Sense: 77 00 00 08 SCSI device sda: drive cache: write through SCSI device sda: 104857600 512-byte hdwr sectors (53687 MB) sda: Write Protect is off sda: Mode Sense: 77 00 00 08 SCSI device sda: drive cache: write through sda: unknown partition table sd 0:0:0:2: Attached scsi disk sda scsi1 : iSCSI Initiator over TCP/IP Vendor: IET Model: VIRTUAL-DISK Rev: 0 Type: Direct-Access ANSI SCSI revision: 04 SCSI device sdb: 104857600 512-byte hdwr sectors (53687 MB) sdb: Write Protect is off sdb: Mode Sense: 77 00 00 08 SCSI device sdb: drive cache: write through SCSI device sdb: 104857600 512-byte hdwr sectors (53687 MB) sdb: Write Protect is off sdb: Mode Sense: 77 00 00 08 SCSI device sdb: drive cache: write through sdb: unknown partition table sd 1:0:0:1: Attached scsi disk sdb sd 1:0:0:1: Attached scsi generic sg1 type 0I now have sda and sdb, each with 53687 MB in size (results for your setup may be different.
- Running fdisk:
fdisk -l
Disk /dev/xvda: 32.2 GB, 32212254720 bytes 255 heads, 63 sectors/track, 3916 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/xvda1 * 1 13 104391 83 Linux /dev/xvda2 14 3916 31350847+ 8e Linux LVM Disk /dev/sda: 53.6 GB, 53687091200 bytes 64 heads, 32 sectors/track, 51200 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk /dev/sda doesn't contain a valid partition table Disk /dev/sdb: 53.6 GB, 53687091200 bytes 64 heads, 32 sectors/track, 51200 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk /dev/sdb doesn't contain a valid partition table
Now do the same for node02. Once the disks are seen by both guests, you can then start setting up a two-node cluster. I've used this configuration to test a two-node Oracle 10gR2 RAC setup with shared ASM storage and OCFS2 on a 64-bit system.
F. Conclusion
This kind of setup will help you to learn the basics of clustering without the need of acquiring additional hardware. In no way can this setup be used in a "live" environment. Once you have familiarized yourself with the concept of how a cluster is prepared, you can apply the concept when building real, physical setups that you need for your organization. I hope you'll find this useful.