Differences between revisions 3 and 4
Revision 3 as of 2010-08-30 13:00:29
Size: 3345
Comment:
Revision 4 as of 2010-08-30 14:05:40
Size: 4628
Comment:
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
Some machines, especially servers, are equipped with dual Ethernet ports
on the motherboard.
In order to use both ports for increased bandwidth and/or
redundancy, Linux must be configured appropriately.
.. Contents::

Linux port bonding
==================

Some machines, especially servers, are equipped with dual Ethernet ports on the motherboard.
In order to use both ports for increased bandwidth and/or redundancy, Linux must be configured appropriately.
Line 20: Line 24:
--------------------------------- =================================
Line 37: Line 41:
------------------------- =========================
Line 87: Line 91:
------------------------ ========================
Line 92: Line 96:

Port bonding troubleshooting
============================

No DHCP response for the bond0 device
-------------------------------------

If you've set up the *bond0* device for DHCP by ``BOOTPROTO=dhcp`` and you don't get a DHCP response from the server,
then it may be because *bond0* uses the **first Ethernet device** (usually *eth0*) for DHCP.
If your DHCP server is configured with the Ethernet MAC-address of another device (for example, *eth1*), then DHCP will fail.

This scenario happens when the Linux kernel has swapped around the Ethernet devices eth0 and eth1 opposite to what the hardware thinks.
Check this by::

  ifconfig -a

to see the MAC-addresses of the network interfaces.

*SystemImager* can correct this problem by **explicit naming of network interfaces** as described in the *Troubleshooting* section
`A possible solution to fix network interface naming <http://wiki.systemimager.org/index.php/Troubleshooting#A_possible_solution_to_fix_network_interface_naming>`_.

You learn the PCI device names and their MAC-addresses by, for example::

  udevinfo -a -p /sys/class/net/eth0

and then add appropriate configuration lines to the file ``/etc/udev/rules.d/60-net.rules``.

Using Multiple Ethernet Cards

Linux port bonding

Some machines, especially servers, are equipped with dual Ethernet ports on the motherboard. In order to use both ports for increased bandwidth and/or redundancy, Linux must be configured appropriately.

You should consult this very nice overview of the Linux bonding driver and the Linux Ethernet Bonding Driver HOWTO. The kernel-doc RPM also documents port bonding in the file /usr/share/doc/kernel-doc-*/Documentation/networking/bonding.txt.

For Redhat RHEL 4 Linux this is documented in A.3.1. Using Multiple Ethernet Cards and 8.2.3. Channel Bonding Interfaces.

Loading the bonding kernel module

Read the A.3.2.1. bonding Module Directives manual.

Add this line to /etc/modprobe.conf (not /etc/modules.conf as documented):

alias bond0 bonding
options bond0 mode=6 miimon=100

with suitable parameters such as miimon as documented. The mode=6 refers to:

Sets an Active Load Balancing (ALB) policy for fault tolerance and load balancing.
Includes transmit and receive load balancing for IPV4 traffic.
Receive load balancing is achieved through ARP negotiation.

Modifying network scripts

In /etc/sysconfig/network-scripts/ new script files should be created:

  1. Create a new bonding device script file ifcfg-bond0 containing:

    DEVICE=bond0
    BOOTPROTO=dhcp
    ONBOOT=yes
    USERCTL=no
  2. The normal Ethernet interface scripts ifcfg-ethN should turn eth0 and eth1 into slave devices:

    DEVICE=eth0
    ONBOOT=yes
    BOOTPROTO=dhcp
    MASTER=bond0
    SLAVE=yes
    USERCTL=no
and similarly for eth1.

When using systemimager to clone the nodes these steps can be performed automatically using post-install scripts, e.g., /var/lib/systemimager/scripts/post-install/20q.eth_bonding_config script for the step 2.:

#!/bin/sh

# Get the Systemimager variables
. /tmp/post-install/variables.txt

# Name of the central server on this network
SERVER=audhumbla1
DOMAINNAME=dcsc.fysik.dtu.dk

# Correct the SystemImager eth0 config, turning eth0 into an Ethernet bonding device (bond0=eth0+eth1)
cp -p /etc/sysconfig/network-scripts/ifcfg-eth0 /tmp/ifcfg-eth0.BAK
cat <<EOF > /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=dhcp
MASTER=bond0
SLAVE=yes
USERCTL=no
EOF

# Finished
cd

Restart network services

At this stage the network should be restarted by service network restart, or the system should be rebooted, in order to activate the bond0 device in stead of the normal eth0 device.

Port bonding troubleshooting

No DHCP response for the bond0 device

If you've set up the bond0 device for DHCP by BOOTPROTO=dhcp and you don't get a DHCP response from the server, then it may be because bond0 uses the first Ethernet device (usually eth0) for DHCP. If your DHCP server is configured with the Ethernet MAC-address of another device (for example, eth1), then DHCP will fail.

This scenario happens when the Linux kernel has swapped around the Ethernet devices eth0 and eth1 opposite to what the hardware thinks. Check this by:

ifconfig -a

to see the MAC-addresses of the network interfaces.

SystemImager can correct this problem by explicit naming of network interfaces as described in the Troubleshooting section A possible solution to fix network interface naming.

You learn the PCI device names and their MAC-addresses by, for example:

udevinfo -a -p /sys/class/net/eth0

and then add appropriate configuration lines to the file /etc/udev/rules.d/60-net.rules.

Niflheim: MultipleEthernetCards (last edited 2017-01-06 09:15:53 by OleHolmNielsen)