There is a new version of this tutorial available for Ubuntu 12.10 (Quantal Quetzal).

Using ATA Over Ethernet (AoE) On Ubuntu 12.04 (Initiator And Target) - Page 2

This tutorial exists for these OS versions

On this page

  1. 4 Setting Up The Initiator (server1)
  2. 5 Links

4 Setting Up The Initiator (server1)

server1:

On server1, we install the initiator:

apt-get install aoetools

Now we check what AoE storage devices are available:

aoe-discover

The command

aoe-stat

should now show the storage devices:

[email protected]:~# aoe-stat
      e0.1        21.474GB   eth0 up
[email protected]:~#

At this point we have a new block device available on the client box named /dev/etherd/e0.1. If we have a look at the /dev tree a new node appears:

ls -la /dev/etherd/

[email protected]:~# ls -la /dev/etherd/
total 0
drwxr-xr-x  2 root root     160 2012-10-01 16:24 .
drwxr-xr-x 16 root root    3800 2012-10-01 16:21 ..
c-w--w----  1 root disk 152,  3 2012-10-01 16:21 discover
brw-rw----  1 root disk 152, 16 2012-10-01 16:24 e0.1
cr--r-----  1 root disk 152,  2 2012-10-01 16:21 err
c-w--w----  1 root disk 152,  6 2012-10-01 16:21 flush
c-w--w----  1 root disk 152,  4 2012-10-01 16:21 interfaces
c-w--w----  1 root disk 152,  5 2012-10-01 16:21 revalidate
[email protected]:~#

In the output of

fdisk -l

you should now also find the new hard drive:

[email protected]:~# fdisk -l

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00016be9

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              32        3917    31205377    5  Extended
/dev/sda5              32        3917    31205376   8e  Linux LVM

Disk /dev/etherd/e0.1: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/etherd/e0.1 doesn't contain a valid partition table
[email protected]:~#

To use that device, we must format it:

fdisk /dev/etherd/e0.1

[email protected]:~# fdisk /dev/etherd/e0.1
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x3093ae28.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help):
 <-- n
Command action
   e   extended
   p   primary partition (1-4)

<-- p
Partition number (1-4): <-- 1
First cylinder (1-2610, default 1): <-- ENTER
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610):
 <-- ENTER
Using default value 2610

Command (m for help):
 <-- t
Selected partition 1
Hex code (type L to list codes):
 <-- 83

Command (m for help): <-- w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[email protected]:~#

Afterwards, the output of

fdisk -l

should look as follows:

[email protected]:~# fdisk -l

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00016be9

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      248832   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              32        3917    31205377    5  Extended
/dev/sda5              32        3917    31205376   8e  Linux LVM

Disk /dev/etherd/e0.1: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3093ae28

            Device Boot      Start         End      Blocks   Id  System
/dev/etherd/e0.1p1               1        2610    20964793+  83  Linux
[email protected]:~#

Now we create a filesystem on /dev/etherd/e0.1p1...

mkfs.ext4 /dev/etherd/e0.1p1

... and mount it for test purposes:

mount /dev/etherd/e0.1p1 /mnt

You should now see the new device in the outputs of...

mount

[email protected]:~# mount
/dev/mapper/server1-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
none on /var/lib/ureadahead/debugfs type debugfs (rw,relatime)
/dev/sda1 on /boot type ext2 (rw)
/dev/etherd/e0.1p1 on /mnt type ext4 (rw)
[email protected]:~#

... and

df -h

[email protected]:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/server1-root
                       18G  838M   16G   5% /
none                  243M  180K  242M   1% /dev
none                  247M     0  247M   0% /dev/shm
none                  247M   32K  247M   1% /var/run
none                  247M     0  247M   0% /var/lock
none                  247M     0  247M   0% /lib/init/rw
none                   18G  838M   16G   5% /var/lib/ureadahead/debugfs
/dev/sda1             228M   17M  199M   8% /boot
/dev/etherd/e0.1p1     20G  172M   19G   1% /mnt
[email protected]:~#

You can unmount it like this:

umount /mnt

To have the device mounted automatically at boot time, e.g. in the directory /storage, we create that directory...

mkdir /storage

... and add the following line to /etc/fstab:

vi /etc/fstab

[...]
/dev/etherd/e0.1p1       /storage        ext4    defaults,auto,_netdev 0 0

This alone isn't enough to have the device mounted at boot time because the AoE stuff gets loaded after /etc/fstab is read. Therefore we open /etc/rc.local...

vi /etc/rc.local

... and add the following lines to it (before the exit 0 line):

[...]
aoe-discover
sleep 5
mount -a
[...]

For test purposes, you can now reboot the system:

reboot

After the reboot, the device should be mounted:

mount

[email protected]:~# mount
/dev/mapper/server1-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
none on /var/lib/ureadahead/debugfs type debugfs (rw,relatime)
/dev/sda1 on /boot type ext2 (rw)
/dev/etherd/e0.1p1 on /storage type ext4 (rw,_netdev)
[email protected]:~#

df -h

[email protected]:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/server1-root
                       18G  838M   16G   5% /
none                  243M  180K  242M   1% /dev
none                  247M     0  247M   0% /dev/shm
none                  247M   32K  247M   1% /var/run
none                  247M     0  247M   0% /var/lock
none                  247M     0  247M   0% /lib/init/rw
none                   18G  838M   16G   5% /var/lib/ureadahead/debugfs
/dev/sda1             228M   17M  199M   8% /boot
/dev/etherd/e0.1p1     20G  172M   19G   1% /storage
[email protected]:~#

 

Falko Timme

About Falko Timme

Falko Timme is an experienced Linux administrator and founder of Timme Hosting, a leading nginx business hosting company in Germany. He is one of the most active authors on HowtoForge since 2005 and one of the core developers of ISPConfig since 2000. He has also contributed to the O'Reilly book "Linux System Administration".

Share this page:

Suggested articles

0 Comment(s)

Add comment

Comments

By: Falko Timme