How To Build A Low Cost SAN - Page 2
On this page
- 7 AoE TARGET SETUP
- 7.1 KVBLADE
- About Kvblade
- Compilation and Installation
- Creation of Block Devices
- Export those Block Devices by Kvblade
- 7.2 AOESERVER
- About Aoeserver
- Compilation and Installation
- Creation of Block Devices
- Export those Block Devices by Aoeserver
- 7.3 VBLADE-KERNEL
- About Vblade-kernel
- Compilation and Installation
- Creation of Block Devices
- Export those Block Devices by Vblade-kernel
- 7.4 VBLADE
- About Vblade
- Compilation and Installation
- Creation of Block Devices
- Export those block devices by Vblade
- 7.5 GGAOED
- 7.5.1 Ggaoed base version
- About it
- Compilation and Installation
- Creation of Block Devices
- Export those Block Devices by GGAOED
- 7.5.2 Updated Ggaoed
- About It
- Compilation and Installation
- Sample Configuration File(ggaoed.conf)
- Creation of Block Devices
- Export those Block Devices by GGAOED
- 7.6 QAOED
- 7.6.1 QAOED (base version)
- Compilation and Installation
- Creation of Block Devices
- Sample Configuration File
- Export those Block Devices by QAOED
- 7.6.2 SQAOED: Porting Qaoed on Solaris 10
- Compilation and Installation
- Creation of Block Devices
- Export those Block Devices by QAOED
7 AoE TARGET SETUP
The brief description of these targets and their setup procedure is described in this section. The objective of this setup is to export the block device on the network. If you don't have spare block device then you can treat your loop device as a block device using losetup command. In this section, all the experiments are done on loop devices. We can replace /dev/loop0 by /dev/sda2 if we have spare disks on our box.
7.1 KVBLADE
About Kvblade
Kvblade is a kernel module implementing the target side of the AoE protocol. Users can command the module through sysfs to export block devices on speci- fied network interfaces. The loopback device should be used as an intermediary for exporting regular files with kvblade.
Compilation and Installation
Download kvblade alpha3 :
[[email protected] ]# wget http://downloads.sourceforge.net/aoetools/kvblade-alpha- 3.tgz
Untar this package:
[[email protected] ]# tar -xzvf kvblade-alpha-3.tgz
[[email protected] ]# cd kvblade-alpha-3
Compilation of this kernel module was done on FC7 (fedora 7):
[[email protected] kvblade-alpha-3]# uname -r
2.6.21-7.fc7xen
Do the following manipulation in kvblade.c file at line no. 3 and at line no. 66 : at line no. 3 : change linux/config.h |> linux/autoconf.h at line no. 66: change ATA SERNO LEN |> ATA ID SERNO LEN Since, linux kernel always keep changes their file name, api and data structures. So, we have done above manipulation for successful compilation of this kernel module.
[[email protected] ]# make
[[email protected] kvblade-alpha-3]# make install
[[email protected] kvblade-alpha-3]# insmod kvblade.ko
Now your kvblade has cleanly compiled and installed on your system. Just run the following command to verify it:
[[email protected] kvblade-alpha-3]# lsmod | grep kvblade
kvblade 17992 0
Creation of Block Devices
If you don't have free devices like /dev/sda4 or /dev/sda5 then you can treat your loop device as your block device by using following command:
[[email protected] kvblade-alpha-3]# dd if=/dev/zero of=le.img bs=1M count=400
[[email protected] kvblade-alpha-3]# losetup /dev/loop0 le.img
Export those Block Devices by Kvblade
[[email protected] kvblade-alpha-3]# ./kvadd 0 0 eth0 /dev/loop0
Now you have exported your block device over network.
7.2 AOESERVER
About Aoeserver
Aoeserver is an in-kernel Ata Over Ethernet Storage target driver used to emu- late a Coraid EtherDriver Blade. It is partly based on vblade and the aoe-client from the Linux 2.6-kernel.
Compilation and Installation
Download the src:
[[email protected] ]# svn checkout http://aoeserver.googlecode.com/svn/trunk/ aoe-
server
[[email protected] ]# cd aoeserver/aoeserver
Compilation of this kernel module has done on FC10 (fedora10) having kernel
version 2.6.27.5-117.fc10.i686
Comment line no. 362 and 380 of /aoeserver/linux/drivers/block/aoeserver/aoeproc.c file :
at line 362: /* remove proc entry(PROCFSNAME, &proc root) */
at line 380: /* remove proc entry(PROCFSNAME, &proc root) */
Now run make command:
[[email protected] aoeserver]# make
[[email protected] aoeserver]# make install
Insert the module:
[[email protected] aoeserver]# sh load.sh
Creation of Block Devices
[[email protected] aoeserver]# dd if=/dev/zero of=file1.img bs=1M count=400
[[email protected] aoeserver]# losetup /dev/loop1 file1.img
Export those Block Devices by Aoeserver
[[email protected] aoeserver]# echo add /dev/loop1 0 1 eth0 > /proc/aoeserver
7.3 VBLADE-KERNEL
About Vblade-kernel
Vblade-kernel is an AoE target emulator implemented as a kernel module for Linux 2.6.* kernels.
Compilation and Installation
Download vblade kernel:
[[email protected] krishna]# wget http://lpk.com.price.ru ~lelik/AoE/vblade-kernel-
0.3.4.tar.gz
[[email protected] krishna]# tar -xzvf vblade-kernel-0.3.4.tar.gz
[[email protected] krishna]# cd vblade-kernel-0.3.4
This kernel module has compiled on FC7 having kernel version 2.6.21-7.fc7xen Do the following change in vblade.h file: at line no. 27 : linux/config.h {> linux/autoconf.h Do the following change in main.c file : (skb linearize(skb, GFP ATOMIC) < 0) {> (skb linearize(skb) Now run make command to compile:
[[email protected] vblade-kernel-0.3.4]# make
[[email protected] vblade-kernel-0.3.4]# insmod vb.ko
Creation of Block Devices
[[email protected] vblade-kernel-0.3.4]# dd if=/dev/zero of=file2.img bs=1M count=200
[[email protected] vblade-kernel-0.3.4]# losetup /dev/loop2 gile2.img
Export those Block Devices by Vblade-kernel
[[email protected] vblade-kernel-0.3.4]# echo add do /dev/loop2 > /sys/vblade/drives
[[email protected] vblade-kernel-0.3.4]# echo add eth0 0 2 32 8 > /sys/vblade/do/ports
Here eth0 is the interface, 0 is shelf, 2 is slot, 32 is the buffer(length of the request queue) and 8 is the maximum number of sectors per request.
7.4 VBLADE
About Vblade
Vblade is the virtual EtherDrive (R) blade, a program that makes a seekable file available over an ethernet local area network (LAN) via the ATA over Ethernet (AoE) protocol. Seekable file is typically a block device like /dev/md0 but even regular files will work. Sparse files can be especially convenient. When vblade exports the block storage over AoE it becomes a storage target. Another host on the same LAN can access the storage if it has a compatible aoe kernel driver.
Compilation and Installation
Download vblade:
[[email protected] krishna]# wget http://downloads.sourceforge.net/aoetools/vblade-
19.tgz
[[email protected] krishna]# tar -xzvf vblade-19.tgz
[[email protected] krishna]# cd vblade-19
[[email protected] krishna]# make
[[email protected] vblade-19]# make install
Creation of Block Devices
[[email protected] vblade-19]# dd if=/dev/zero of=newfile3.img bs=1M count=200
[[email protected] vblade-19]# losetup /dev/loop3 newfile3.img
Export those block devices by Vblade
[[email protected] vblade-19]# ./vbladed 0 3 eth0 /dev/loop3
7.5 GGAOED
Ggaoed is an AoE (ATA over Ethernet) target implementation for Linux. It utilizes Linux kernel AIO, memory mapped sockets and other Linux features to provide the best performance. GGAOED comes in two avour: Ggaoed base version and Updated Ggaoed.
7.5.1 Ggaoed base version
About it
This is the base version of ggaoed-0.9.tar.gz (release-0.9) released in July 2008. This can be downloaded from following site: http://code.google.com/p/ggaoed/downloads/list
Compilation and Installation
Running ggaoed requires Linux kernel 2.6.22 or later. So, Its better that to run ggaoed on your box you can have FC10 installed on your system.The following software is needed in order to build ggaoed:
- glibc 2.7 (built on Linux kernel 2.6.22 or later)
- libaio 0.3.107
- libatomic ops 1.2
- glib 2.12
- xsltproc and DocBook XML 4.5 stylesheets for building the documentation
Run following commands to compile GGAOED:
[[email protected] ggaoed-0.9]# ./configure
[[email protected] ggaoed-0.9]# make
[[email protected] ggaoed-0.9]# make install
Sample Configuration File(ggaoed.conf)
The format of sample configuration file is as follows:
example file: ggaoed.conf
[sdc] path = /dev/sda2 shelf = 0 slot = 0 broadcast = true read-only = true queue-length=64 direct-io=true
Creation of Block Devices
Here we have taken /dev/sda2 which is unmounted. If you don't have your available unmounted partition, then you can create your block device by using dd and losetup command.
Export those Block Devices by GGAOED
[[email protected] ggaoed-0.9]# ./ggaoed -c ggaoed.conf -d
7.5.2 Updated Ggaoed
Updated version of GGAOED has released in March 2009 according to the re- vised AOE protocol specification in Feb 2009.
About It
The new GGAOED has released with updtaed features of AOE protocol like mac-filtering and acl-listing.
Compilation and Installation
Download the source:
[[email protected] Desktop]# svn checkout http://ggaoed.googlecode.com/svn/trunk/
ggaoed
[[email protected] Desktop]# cd ggaoed/
[[email protected] ggaoed]# autoreconf - - install
Comment the following line in Makefile.am at line no. 26: #SUBDIRS += doc
[[email protected] ggaoed]# ./configure
[[email protected] ggaoed]# make
[[email protected] ggaoed]# make install
Sample Configuration File(ggaoed.conf)
First create the directory for config file by running following command:
[[email protected] ]# mkdir -p /usr/local/var/ggaoed
Format of config file:
# Sample ggaoed configuration file [defaults] pid-file = /var/run/ggaoed.pid [eth0] mtu = 1500 [testdev] path = /dev/loop0 direct-io = true queue-length = 16 shelf = 0 slot = 0 interfaces = eth0 broadcast = true read-only = true
Creation of Block Devices
[[email protected] ]# dd if=/dev/zero of=newfile.img bs=1M count=200
[[email protected] ]# losetup /dev/loop0 newfile.img
Export those Block Devices by GGAOED
[[email protected] ]# ./ggaoed -c ggaoed.conf -d
7.6 QAOED
Qaoed is a multithreaded ATA over Ethernet daemon that's easy to use, yet highly configurable.QAOED also comes in two avours: qaoed and sqaoed.
7.6.1 QAOED (base version)
Compilation and Installation
You can download qaoed by following command:
[[email protected] Desktop]# svn checkout http://qaoed.googlecode.com/svn/trunk/
qaoed
[[email protected] qaoed]# make
Creation of Block Devices
The block devices can be created as follows:
[[email protected] qaoed]#dd if=/dev/zero of=newfile6.img bs=1M count=200
[[email protected] qaoed]#losetup /dev/loop6 newfile6.img
Sample Configuration File
apisocket = /tmp/qaoedsocket; default { shelf = 0; /* Shelf */ slot = 6; /* Autoincremented slot numbers */ interface = eth0; device { target = /dev/loop6; }
Export those Block Devices by QAOED
[[email protected] qaoed]# ./qaoed -c qaoed.conf
7.6.2 SQAOED: Porting Qaoed on Solaris 10
Fubra decided to evaluate various platforms to help support our next genera- tion network infrastructure (http://code.fubra.com/wiki/PortingQaoed). One of the platforms they are evaluating is the Sun T5420, a high end Ultra Sparc, T2 dual processor with support for 64 threads per processors. It was decided to port the Qaoed daemon to the Solaris platform, on both x86 and Ultra Sparc. Sqoaed was the result of stripping out all the pthreads and reordering parts of the qaoed sources.
Compilation and Installation
Download the source from following site:
[[email protected] sqaoed]# svn co http://svn.fubra.com/storage/sqaoed/ trunk
[[email protected] sqaoed]# cd trunk/trunk
[[email protected] sqaoed]# make
[[email protected] sqaoed]# make install
Creation of Block Devices
[[email protected] trunk]# dd if=/dev/zero of=newfile7.img bs=1M count=200
[[email protected] trunk]# losetup /dev/loop7 newfile7.img
Export those Block Devices by QAOED
[[email protected] trunk]# ./sqaoed eth0 /dev/loop7 0 0