PDA

View Full Version : Nic Bonding/Teaming Howto


themachine
2nd December 2005, 08:15
I know some of you have seen this howto:

http://www.howtoforge.com/nic_bonding


However, some people were having issues getting it working properly. So, I wanted to start the thread and see if this is working for people or not. I just updated it with some key changes (typos... left overs from Woody config) and I think it should work solid now.

Let me know if there are any questions maybe I could help.

lxtwin
5th February 2006, 01:36
Hi there,
Have you got this working on RHEL 4 where eth0 & eth1 = bond0 failover mode
and eth2 & eth3 = bond1 in failover mode. I have this working on SLES9 and RHEL3 but not on 4?


paul

themachine
6th February 2006, 08:58
What exactly isn't working? Are you getting errors, do the logs note anything...?

I don't exactly have a RHEL box available to me right now, but I might be able to make something happen.

william
8th February 2006, 17:42
I have three internet connections, dsl, satellite and a modem. If i bond all three connections ie eth0 eth1 eth2 then route bond1 to a local network, will the local network get a internet speed equal to the dsl + satellite + modem? And will it share equally to all three, because the dsl is faster than the modem?

modem ---|
dsl ---|---bond1(eth0,1,2)---linux---eth3---|---local network
kuband ---|

Many Thanks

William

themachine
8th February 2006, 20:48
I am not exactly sure what you are attempting to accomplish. The main goal of bonding is failover in the event that one of your network links fails. If all 3 nic are bonded, and connected to DSL... your bottleneck (speed limitation) will be that of the DSL link. You're not going to increase your DSL speed by bonding multiple NIC to the DSL connection.

Additionally, all links need to be a part of the same VLAN.

aixguy
21st April 2006, 21:58
I am working with a Ubuntu server install
2.6.12-10-686
My goal is to get NIC failover such as we have on our AIX servers. If on path fails the other takes over instantly.
I follow the instructions here:
http://www.howtoforge.com/nic_bonding

And I have had some success.
bond0 comes up and it functional, but I think this is a trunk set up rather than failover. All three adapter have the same MAC and I don't think that is correct. In the AIX would the new sudo adapter like bond0 has the mac of the first nic, and when it fails over to the other NIC keeps the address of the first. However, the 2 physical NICs keep there real MACs when you look at the devices. Also neither of the physical nics are up or show up in ifconfig..

When we pull the cable hooked to eth0 the only way we can get the server to fail over, is to initiate a connection from the server to someting .. ie ping some other box. If we don't do this, we can not get to the box from the rest of the network. The other issue is that this config does not come back after a reboot. After rebooting there is no bond0 and no network connection.
Thanks in advance for any insite, I am new to Linux and am getting frustrated with not being able to find the answers I need.
Thanks,
Doug

themachine
21st April 2006, 22:16
I am working with a Ubuntu server install
2.6.12-10-686
My goal is to get NIC failover such as we have on our AIX servers. If on path fails the other takes over instantly.
I follow the instructions here:
http://www.howtoforge.com/nic_bonding

And I have had some success.
bond0 comes up and it functional, but I think this is a trunk set up rather than failover. All three adapter have the same MAC and I don't think that is correct. In the AIX would the new sudo adapter like bond0 has the mac of the first nic, and when it fails over to the other NIC keeps the address of the first. However, the 2 physical NICs keep there real MACs when you look at the devices. Also neither of the physical nics are up or show up in ifconfig..

When we pull the cable hooked to eth0 the only way we can get the server to fail over, is to initiate a connection from the server to someting .. ie ping some other box. If we don't do this, we can not get to the box from the rest of the network. The other issue is that this config does not come back after a reboot. After rebooting there is no bond0 and no network connection.
Thanks in advance for any insite, I am new to Linux and am getting frustrated with not being able to find the answers I need.
Thanks,
Doug


It would sound to me, that if the 'bond0' device does not load on boot up then you have not configured the module properly in '/etc/modutils/aliases'. You need to add something like the following:

alias bond0 bonding


Once in '/etc/modutils/aliases', you need to run 'update-modules'. This is the Debian way of doing things, and I assume the Ubuntu way. If not, simply add 'alias bond0 bonding' to /etc/modules.conf. You also need the aliases for your physical NICs (I believe).

This type of configuration performs a 'round robin' type network connection. Both NICs are used to send traffic, therefore the traffic comes from the bond0 interface (appearing to coming from the MAC of the bond). If the MAC address changed when the NIC failed over, you would loose your ARP and therefore no connections would come to the box until you re-arped (i.e. ping from the other NIC). So it appears that the bond is infact not configured properly (or there is something different about Ubuntu which I wouldn't think is the case).

Can you paste your '/etc/network/interfaces' config?

Thanks.

vertical98
19th November 2006, 01:21
just to get it out of the way here is my pings

=================
64 bytes from 10.1.100.61: icmp_seq=25 ttl=64 time=0.060 ms
64 bytes from 10.1.100.61: icmp_seq=27 ttl=64 time=0.063 ms
64 bytes from 10.1.100.61: icmp_seq=29 ttl=64 time=0.062 ms
64 bytes from 10.1.100.61: icmp_seq=31 ttl=64 time=0.061 ms
64 bytes from 10.1.100.61: icmp_seq=33 ttl=64 time=0.064 ms

--- 10.1.100.61 ping statistics ---
34 packets transmitted, 17 received, 50% packet loss, time 33004ms
rtt min/avg/max/mdev = 0.060/0.075/0.285/0.053 ms
mail:~#
=================

I have two machines: mail and argon connected via 2 crossover cables

mail has 3 nics: eth0 = pcnet32 --> lan
eth1 = tulip --> X-over --> argon
eth2 = 8139too -> X-over -> argon

argon also has 3 nics: eth0 = e100 --> lan
eth1 = pcnet32 --> X-over --> mail
eth2 = 8139too --> X-over --> mail

aliases is correct and /etc/network/interfaces should be fine:

auto bond0
iface bond0 inet static
address 10.1.100.63
netmask 255.255.255.0
# hwaddress ether 00:02:B3:48:50:2C
# gateway 10.1.100.1
up ifenslave bond0 eth1 eth2
down ifenslave -d bond0 eth1 eth2

I just realized that the HWaddress was commented out. I removed the # and presto it works fine.

I'll leave the full comment, incase someone else runs into this problem.

Vertical

Djolliff
27th March 2007, 03:11
I'm using (K)Ubuntu 6.10 and followed the bonding howto, but the RealTek chipset cards don't seem to work -- Anyone have this working with these cheap cards?
They work really well so far stand alone.

babrew
15th November 2007, 21:52
if, on rhel4 update 4, you have working bond0 but have problem with setting up bond1 interface see
http://article.gmane.org/gmane.linux.network/77672

To load multiple instances of the module, it is necessary to
specify a different name for each instance (the module loading system
requires that every loaded module, even multiple instances of the same
module, have a unique name). This is accomplished by supplying multiple
sets of bonding options in /etc/modprobe.conf, for example:

alias bond0 bonding
options bond0 -o bond0 mode=balance-rr miimon=100

alias bond1 bonding
options bond1 -o bond1 mode=balance-alb miimon=50

will load the bonding module two times. The first instance is
named "bond0" and creates the bond0 device in balance-rr mode with an
miimon of 100. The second instance is named "bond1" and creates the
bond1 device in balance-alb mode with an miimon of 50.

In some circumstances (typically with older distributions),
the above does not work, and the second bonding instance never sees
its options. In that case, the second options line can be substituted
as follows:

install bond1 /sbin/modprobe --ignore-install bonding -o bond1 \
mode=balance-alb miimon=50

This may be repeated any number of times, specifying a new and
unique name in place of bond1 for each subsequent instance.

It has been observed that some Red Hat supplied kernels are unable
to rename modules at load time (the "-o bond1" part). Attempts to pass
that option to modprobe will produce an "Operation not permitted" error.
This has been reported on some Fedora Core kernels, and has been seen on
RHEL 4 as well. On kernels exhibiting this problem, it will be impossible
to configure multiple bonds with differing parameters (as they are older
kernels, and also lack sysfs support).

On my server I set up bonding in /etc/modprobe.conf in this way:
alias bond0 bonding
options bond0 -o bond0 miimon=100 mode=1
install bond1 /sbin/modprobe --ignore-install bonding -o bond1 mode=1 miimon=100
and now it works.

bartek

kizman
30th January 2008, 22:53
So the purpose of NIC Bonding/Teaming is making sure that your connection to the internet is not disrupted. (If one connection fails, the next one takes over)?

I have read that this method can be used to Bond multiple connections in a sense of speed.

For example:

DSL LINE NIC#1 --------------------\ ___> > Bond NIC#3(virtual NIC)
WIRELESS CONNECTION NIC#2------ /


A dsl line (NIC#1), and a Wireless Connection (NIC#2) are bonded to a virtual NIC#3. This virtual NIC#3 uses the full speeds of NIC#1+NIC#2 to give a combined download speed.

---------------------------------

I know this is possible. It's just that software may not have been programmed to do this. ( Atleast I haven't found anything, yet)

Does anybody know how I can accomplish the scenario?

If not exactly my method, then is there any way to combine two connections into one which accepts both NIC's packets to download the combined speed of the two connections.

I have two TWO NICS:

One is my DSL connection
The other is a wireless network.

bamatunz
13th September 2010, 23:06
I have a similar scenario on RHEL4 - I have successfully setup 2 sets of RAIN - bond0 and bond1. 1 is for the public IP and the other is for internal replication (DRBD) between 2 nodes in a HA cluster. For my purposes I need to try and reduce the number of switch ports required so I can deploy them with a minimal amount of specs (several clusters going to remote locations). My thought is to use crossover cables to connect the internal DRBD RAINs but I cannot get them to work properly. As you can see I'm using mode 5 which does not require additional switch support so in my head it should work with crossover cables directly. What am I missing?

Here is are some config settings so you can see what I have:

1) cat /etc/modprob.conf
alias eth0 e1000
alias eth1 e1000
alias eth2 tg3
alias eth3 tg3
alias scsi_hostadapter cciss
alias scsi_hostadapter1 ata_piix
alias bond0 bonding
options bond0 miimon=80 mode=5
alias bond1 bonding
options bond1 miimon=80 mode=5

2) cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=10.0.0.140
NETMASK=255.255.255.0
BROADCAST=10.0.0.255
GATEWAY=10.0.0.1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
TYPE=BOND

(the 2nd bond1 is identical except for the IP and device)

3) cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes
TYPE=Ethernet

Once again the other eth* are identical except for device name and master entries.

CURRENT RESULTS:
When I try to replace the switched cables with crossover cables OR standard Gigabyte Cat6e cables I loose connectivity on bond0 and even loose network all together on node-2 in the cluster.

NOTE!
I think it's pertinent to mention that 2 of the NICs on each server are onboard GB and the other two are GB on a dual-port PCI-x card. I have tried crossovers on both bond0 and bond1, onboard and PCI-x = no joy.

Does this have something to do with needing independent NIC cards with single ports on them in order to successfully cross them?

If you need more information please let me know.

Thanks in advance!

matty
16th September 2010, 08:41
I'm fairly certain a lot of the bonding modes use a gratuitous arp to advise the network which NIC is in control. This would require a switched environment where all NICs see each other. Connecting NIC to NIC removes that. I'm not certain that is the cause of your issues, but I'd be suspicious of it. Maybe you could try mode 1, which, IIRC, is a simple failover system.

To prove your hardware, you could revert to single ethX settings and see if your direct cable works in that configuration.

bamatunz
16th September 2010, 20:20
Thanks for the quick response...I will definitely try mode 1 just to prove your point which is a great idea - let's see if it works with 1 NIC straight through first, then introduce RAIN. (currently this cluster is in a holding pattern while we troubleshoot another issue).

However, the reason we wanted to use mode 5 or 6 is 2 fold:

1) to increase the throughput for the drbd resource.

2) to reduce the amount of switch ports required by our cluster for remote deployment situations where space is already at capacity.

I will give your suggestions a try and re-run the baseline tests we've developed to measure performance which is our deciding factor. Obviously with the introduction of RAIN we have failover which is great but only temporarily - we need the throughput to me at max capacity to be in a production environment.

capableson
3rd May 2011, 23:13
The term "network teaming / bonding" have been used to aggregate IP links for faster connection. In the early 90s, shotgun dial up modems were using bonding to double the network speed.

Today, there are really 2 main approaches. One is the bonding at layer1 or layer2 or just above layer2 (such as MLPPP). These require ISP to install MLPPP line-cards in their POP. The other more efficient approach is to do the bonding in higher layers and that is called "Broadband Bonding". Hope this helps :o