4 NICs - 2 in RAIN - crossover doesn't work
I have a similar scenario on RHEL4 - I have successfully setup 2 sets of RAIN - bond0 and bond1. 1 is for the public IP and the other is for internal replication (DRBD) between 2 nodes in a HA cluster. For my purposes I need to try and reduce the number of switch ports required so I can deploy them with a minimal amount of specs (several clusters going to remote locations). My thought is to use crossover cables to connect the internal DRBD RAINs but I cannot get them to work properly. As you can see I'm using mode 5 which does not require additional switch support so in my head it should work with crossover cables directly. What am I missing?
Here is are some config settings so you can see what I have:
1) cat /etc/modprob.conf
alias eth0 e1000
alias eth1 e1000
alias eth2 tg3
alias eth3 tg3
alias scsi_hostadapter cciss
alias scsi_hostadapter1 ata_piix
alias bond0 bonding
options bond0 miimon=80 mode=5
alias bond1 bonding
options bond1 miimon=80 mode=5
2) cat /etc/sysconfig/network-scripts/ifcfg-bond0
(the 2nd bond1 is identical except for the IP and device)
3) cat /etc/sysconfig/network-scripts/ifcfg-eth0
Once again the other eth* are identical except for device name and master entries.
When I try to replace the switched cables with crossover cables OR standard Gigabyte Cat6e cables I loose connectivity on bond0 and even loose network all together on node-2 in the cluster.
I think it's pertinent to mention that 2 of the NICs on each server are onboard GB and the other two are GB on a dual-port PCI-x card. I have tried crossovers on both bond0 and bond1, onboard and PCI-x = no joy.
Does this have something to do with needing independent NIC cards with single ports on them in order to successfully cross them?
If you need more information please let me know.
Thanks in advance!