bonding.txt

《嵌入式系统设计与实例开发实验教材二源码》Linux内核移植与编译实验

                   Linux Ethernet Bonding Driver mini-howto

Initial release : Thomas Davis <tadavis at lbl.gov>
Corrections, HA extensions : 2000/10/03-15 :
  - Willy Tarreau <willy at meta-x.org>
  - Constantine Gavrilov <const-g at xpert.com>
  - Chad N. Tindel <ctindel at ieee dot org>
  - Janice Girouard <girouard at us dot ibm dot com>

Note :
------
The bonding driver originally came from Donald Becker's beowulf patches for
kernel 2.0. It has changed quite a bit since, and the original tools from
extreme-linux and beowulf sites will not work with this version of the driver.

For new versions of the driver, patches for older kernels and the updated
userspace tools, please follow the links at the end of this file.

Installation
============

1) Build kernel with the bonding driver
---------------------------------------
For the latest version of the bonding driver, use kernel 2.4.12 or above
(otherwise you will need to apply a patch).

Configure kernel with `make menuconfig/xconfig/config', and select
"Bonding driver support" in the "Network device support" section. It is
recommended to configure the driver as module since it is currently the only way
to pass parameters to the driver and configure more than one bonding device.

Build and install the new kernel and modules.

2) Get and install the userspace tools
--------------------------------------
This version of the bonding driver requires updated ifenslave program. The
original one from extreme-linux and beowulf will not work. Kernels 2.4.12
and above include the updated version of ifenslave.c in Documentation/network
directory. For older kernels, please follow the links at the end of this file.

IMPORTANT!!!  If you are running on Redhat 7.1 or greater, you need
to be careful because /usr/include/linux is no longer a symbolic link
to /usr/src/linux/include/linux.  If you build ifenslave while this is
true, ifenslave will appear to succeed but your bond won't work.  The purpose
of the -I option on the ifenslave compile line is to make sure it uses
/usr/src/linux/include/linux/if_bonding.h instead of the version from
/usr/include/linux.

To install ifenslave.c, do:
    # gcc -Wall -Wstrict-prototypes -O -I/usr/src/linux/include ifenslave.c -o ifenslave 
    # cp ifenslave /sbin/ifenslave

3) Configure your system
------------------------
Also see the following section on the module parameters. You will need to add
at least the following line to /etc/conf.modules (or /etc/modules.conf):

	alias bond0 bonding

Use standard distribution techniques to define bond0 network interface. For
example, on modern RedHat distributions, create ifcfg-bond0 file in
/etc/sysconfig/network-scripts directory that looks like this:

DEVICE=bond0
IPADDR=192.168.1.1
NETMASK=255.255.255.0
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

(put the appropriate values for you network instead of 192.168.1).

All interfaces that are part of the trunk, should have SLAVE and MASTER
definitions. For example, in the case of RedHat, if you wish to make eth0 and
eth1 (or other interfaces) a part of the bonding interface bond0, their config
files (ifcfg-eth0, ifcfg-eth1, etc.) should look like this:

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

(use DEVICE=eth1 for eth1 and MASTER=bond1 for bond1 if you have configured
second bonding interface). 

Restart the networking subsystem or just bring up the bonding device if your
administration tools allow it. Otherwise, reboot. (For the case of RedHat
distros, you can do `ifup bond0' or `/etc/rc.d/init.d/network restart'.)

If the administration tools of your distribution do not support master/slave
notation in configuration of network interfaces, you will need to configure
the bonding device with the following commands manually:

    # /sbin/ifconfig bond0 192.168.1.1 up
    # /sbin/ifenslave bond0 eth0
    # /sbin/ifenslave bond0 eth1

(substitute 192.168.1.1 with your IP address and add custom network and custom
netmask to the arguments of ifconfig if required).

You can then create a script with these commands and put it into the appropriate
rc directory.

If you specifically need that all your network drivers are loaded before the
bonding driver, use one of modutils' powerful features : in your modules.conf,
tell that when asked for bond0, modprobe should first load all your interfaces :

probeall bond0 eth0 eth1 bonding

Be careful not to reference bond0 itself at the end of the line, or modprobe will
die in an endless recursive loop.

4) Module parameters.
---------------------
The following module parameters can be passed:

    mode=

Possible values are 0 (round robin policy, default) and 1 (active backup
policy), and 2 (XOR).  See question 9 and the HA section for additional info.

    miimon=

Use integer value for the frequency (in ms) of MII link monitoring. Zero value
is default and means the link monitoring will be disabled. A good value is 100
if you wish to use link monitoring. See HA section for additional info.

    downdelay=

Use integer value for delaying disabling a link by this number (in ms) after
the link failure has been detected. Must be a multiple of miimon. Default
value is zero. See HA section for additional info.

    updelay=

Use integer value for delaying enabling a link by this number (in ms) after
the "link up" status has been detected. Must be a multiple of miimon. Default
value is zero. See HA section for additional info.

    arp_interval=

Use integer value for the frequency (in ms) of arp monitoring.  Zero value 
is default and means the arp monitoring will be disabled.  See HA section
for additional info.   This field is value in active_backup mode only.

    arp_ip_target=

An ip address to use when arp_interval is > 0.  This is the target of the
arp request sent to determine the health of the link to the target.  
Specify this value in ddd.ddd.ddd.ddd format.

If you need to configure several bonding devices, the driver must be loaded
several times. I.e. for two bonding devices, your /etc/conf.modules must look
like this:

alias bond0 bonding
alias bond1 bonding

options bond0 miimon=100
options bond1 -o bonding1 miimon=100

5) Testing configuration
------------------------
You can test the configuration and transmit policy with ifconfig. For example,
for round robin policy, you should get something like this:

[root]# /sbin/ifconfig
bond0     Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4  
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:7224794 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3286647 errors:1 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:0 

eth0      Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4  
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3573025 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1643167 errors:1 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:100 
          Interrupt:10 Base address:0x1080 

eth1      Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4  
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3651769 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1643480 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          Interrupt:9 Base address:0x1400 

Questions :
===========

1.  Is it SMP safe?

	Yes.  The old 2.0.xx channel bonding patch was not SMP safe.
	The new driver was designed to be SMP safe from the start.

2.  What type of cards will work with it?

	Any Ethernet type cards (you can even mix cards - a Intel
	EtherExpress PRO/100 and a 3com 3c905b, for example).
	You can even bond together Gigabit Ethernet cards!

3.  How many bonding devices can I have?

	One for each module you load. See section on module parameters for how
	to accomplish this.

4.  How many slaves can a bonding device have?

	Limited by the number of network interfaces Linux supports and the
	number of cards you can place in your system.

5.  What happens when a slave link dies?

	If your ethernet cards support MII status monitoring and the MII
	monitoring has been enabled in the driver (see description of module
	parameters), there will be no adverse consequences. This release
	of the bonding driver knows how to get the MII information and
	enables or disables its slaves according to their link status.
	See section on HA for additional information.

	For ethernet cards not supporting MII status, or if you wish to 
	verify that packets have been both send and received, you may
	configure the arp_interval and arp_ip_target.  If packets have
	not been sent or received during this interval, an arp request
	is sent to the target to generate send and receive traffic.  
	If after this interval, either the successful send and/or 
	receive count has not incremented, the next slave in the sequence
	will become the active slave.

	If neither mii_monitor and arp_interval is configured, the bonding
	driver will not handle this situation very well. The driver will 
	continue to send packets but some packets will be lost. Retransmits 
	will cause serious degradation of performance (in the case when one
	of two slave links fails, 50% packets will be lost, which is a serious
	problem for both TCP and UDP).

6.  Can bonding be used for High Availability?

	Yes, if you use MII monitoring and ALL your cards support MII link
	status reporting. See section on HA for more information.

7.  Which switches/systems does it work with?

	In round-robin mode, it works with systems that support trunking:
	
	* Cisco 5500 series (look for EtherChannel support).
	* SunTrunking software.
	* Alteon AceDirector switches / WebOS (use Trunks).
	* BayStack Switches (trunks must be explicitly configured). Stackable
	  models (450) can define trunks between ports on different physical
	  units.
	* Linux bonding, of course !
	
	In Active-backup mode, it should work with any Layer-II switches.