NIC Teaming Overview

NIC Teaming, also known as load balancing and failover (LBFO), allows multiple network adapters on a computer to be placed into a team for the following purposes:

• Bandwidth aggregation
  
• Traffic failover to prevent connectivity loss in the event of a network component failure
  

This feature has been a requirement for independent hardware vendors (IHVs) to enter the server network adapter market, but until now NIC Teaming has not been included in Windows Server operating systems.

Requirements

NIC Teaming requires the presence of a single Ethernet network adapter, which can be used for separating traffic that is using VLANs. All modes that provide fault protection through failover require at least two Ethernet network adapters. Windows Server® 2012 supports up to 32 network adapters in a team.

Technical overview

This guide includes the following sections that describe the NIC Teaming technology in Windows Server 2012:

• NIC Teaming architecture
  
• Bandwidth aggregation (also known as load balancing) mechanisms
  
• Traffic distribution algorithms
  
• NIC Teaming in virtual machines
  
• Incompatibilities
  
• Requirements
  

NIC Teaming architecture

Today, all NIC Teaming solutions on the market have a similar architecture, as shown in Figure 1.
Figure 1   Teaming solution architecture
Two or more physical network adapters are connected to the NIC Teaming solution multiplexing unit, which then presents one or more virtual adapters (also known as team network adapters) to the operating system. There are several different algorithms that distribute inbound and outbound traffic between the physical network adapters. In current non-Microsoft NIC Teaming solutions, the team network adapters divide traffic by virtual LAN (VLAN) so that applications can connect to different VLANs at the same time. Technically, this separation of traffic is not part of NIC Teaming. However, because other commercial implementations of NIC Teaming have this capability, the Windows Server 2012 implementation also includes it.

NIC Teaming configurations

There are two basic sets of algorithms that are used for NIC Teaming:

• Algorithms that require the switch to participate in the teaming, also known as switch-dependent modes. These algorithms usually require all the network adapters of the team to be connected to the same switch.
  
• Algorithms that do not require the switch to participate in the teaming, also referred to as switch-independent modes. Because the switch does not know that the network adapter is part of a team, the team network adapters can be connected to different switches. Switch-independent modes do not require that the team members connect to different switches, they merely make it possible.
  

There are two common choices for switch-dependent modes of NIC Teaming:

• Generic or static teaming (IEEE 802.3ad draft v1). This mode requires configuration on the switch and the computer to identify which links form the team. Because this is a statically configured solution, no additional protocol assists the switch and the computer to identify incorrectly plugged cables or other errors that could cause the team to fail. This mode is typically supported by server-class switches.
  
• Dynamic teaming (IEEE 802.1ax, LACP). IEEE 802.1ax uses the Link Aggregation Control Protocol (LACP) to dynamically identify links between the computer and a specific switch. This enables the automatic creation of a team and, in theory, the expansion and reduction of a team simply by the transmission or receipt of LACP from the peer network adapter. Typical server-class switches support IEEE 802.1ax, but most switches require manual administration to enable LACP on the port.
  
  

  Note
  IEEE 802.1ax is also commonly known as IEEE 802.3ad because it was developed by the IEEE 802.3ad committee before being published as IEEE 802.1ax.

Both modes should result in inbound and outbound traffic approaching the practical limits of the aggregated bandwidth because the pool of links in the team functions as a single pipeline.

Traffic distribution algorithms

Outbound traffic can be distributed among the available links in many ways. Try to keep all packets that are associated with a single flow (TCP stream) on a single network adapter. This is needed to minimize out-of-order packet arrival scenarios.
NIC Teaming in Windows Server 2012 supports the following traffic distribution methods:

• Hyper-V switch port. In cases where virtual machines have independent media access control (MAC) addresses, the MAC address of the virtual machine can provide the basis for dividing traffic. There is an advantage in using this scheme in virtualization. Because the adjacent switch can determine that specific source MAC addresses are on only one connected network adapter, the switch will balance the egress load (the traffic from the switch to the computer) on multiple links, based on the destination MAC address for the virtual machine. This is particularly helpful when used with virtual machine queue. However, this mode might not be specific enough to get a well-balanced distribution, and it limits a single virtual machine to the bandwidth that is available on a single network adapter.
  
  

  Note
  Windows Server 2012 uses the Hyper-V switch port as the identifier rather than the source MAC address, because in some instances, a virtual machine might be using more than one MAC address on a switch port.

• Hashing. This algorithm creates a hash based on components of the packet, and then it assigns packets that have that hash value to one of the available network adapters. This keeps all packets from the same TCP stream on the same network adapter. Hashing alone usually creates balance across the available network adapters. Some NIC Teaming solutions that are available on the market monitor the distribution of the traffic and reassign specific hash values to different network adapters in an attempt to better balance the traffic. The dynamic redistribution is known as smart load balancing or adaptive load balancing.
  

The components that can be used as inputs to the hashing function include:

• Source and destination MAC addresses
  
• Source and destination IP addresses, with or without considering the MAC addresses (2-tuple hash)
  
• Source and destination TCP ports, usually used along with the IP addresses (4-tuple hash)
  

The quadruple hash creates a finer distribution of traffic streams, which results in smaller streams that can be independently moved between network adapters. However, it cannot be used for traffic that is not TCP or UDP traffic or where the TCP and UDP ports are hidden from the stack, such as traffic that is protected by Internet Protocol security (IPsec). In these cases, the hash falls back to a 2-tuple hash. If the traffic is not IP traffic, the hash generator will use the source and destination MAC addresses.

NIC Teaming in virtual machines

NIC Teaming in Windows Server 2012 also works within a virtual machine. This allows a virtual machine to have virtual network adapters that are connected to more than one Hyper-V switch and still have connectivity even if the network adapter under that switch gets disconnected. This is particularly important when working with features such as Single Root I/O Virtualization (SR-IOV) because SR-IOV traffic does not go through the Hyper-V switch. Thus, it cannot be protected by a team that is under a Hyper-V switch. With the virtual machine teaming option, an administrator can set up two Hyper-V switches, each connected to its own SR-IOV-capable network adapter. At that point:

• Each virtual machine can then install a virtual function from one or both SR-IOV network adapters. Then, in the event of a network adapter disconnect, the virtual machine can fail over from the primary virtual function to the backup virtual function.
  
• Alternately, the virtual machine might have a virtual function from one network adapter and a non-virtual function network adapter to the other switch. If the network adapter that is associated with the virtual function gets disconnected, the traffic can fail over to the other switch without loss of connectivity.
  

Note
Because failover between network adapters in a virtual machine might result in traffic being sent with the MAC address of the other network adapter, each Hyper-V switch port that is associated with a virtual machine that is using NIC Teaming must be set to allow MAC spoofing or must have the “AllowTeaming=On” parameter set using the Set-VmNetworkAdapter PowerShell cmdlet.

Incompatibilities

NIC Teaming is compatible with all networking capabilities in Windows Server 2012 with three exceptions: SR-IOV, remote direct memory access (RDMA), and TCP Chimney. For SR-IOV and remote direct memory access (RDMA), data is delivered directly to the network adapter without passing through the networking stack. Therefore, it is not possible for the network adapter team to look at or redirect the data to another path in the team. TCP Chimney is not supported with NIC Teaming in Windows Server 2012.

Requirements

NIC Teaming requires the presence of a single Ethernet network adapter, which can be used for separation of traffic using VLANs. All modes that provide fault protection through failover require at least two Ethernet network adapters. The Windows Server 2012 implementation supports up to 32 NICs in a team.

Configuring NIC Teaming

You can configure NIC Teaming in Windows Server 2012 through the NIC Teaming management interface or through the rich PowerShell interface. The NIC Teaming management UI is a simple interface that uses the PowerShell cmdlets to manage teams. See below for the complete guide to NIC Teaming PowerShell cmdlets. There are multiple ways to invoke the NIC Teaming management interface.

From Server Manager:

1. Open Server Manager.
   
2. In the console tree, click Local Server.
   
3. In the details pane, in the Properties section, click NIC Teaming Administration under Remote Desktop.
   

From the desktop:

1. Click Start, type lbfoadmin, and then press Enter. Alternately, click Start, click Run, type lbfoadmin, and then press Enter.
   

To manage multiple servers at a time you can add servers through the “Add Server” task item, or you can use the following command:
lbfoadmin /servers servername1 servername2 …
This will run the NIC Teaming management interface with the list of specified server names.

To configure NIC Teaming on a server

1. Click the server name in the list of servers (even if there is only a single server).
   
2. From the Tasks drop-down menu in the Teams section, click New Team.
   
3. In the Add Team dialog box, type a team name and select the network adapters in the team.
   
4. If you are using VLANs, clear the Default check box and specify the VLAN ID. By default, the team will deliver up all traffic received regardless of VLAN ID (though the VLAN ID is passed with the packet so that other components in the stack can sort them appropriately).
   
5. If you want to use a mode other than the default, click Advanced and specify the teaming mode (Switch Independent, Static Teaming, or LACP) and load distribution mode (Address Hash) as needed.
   
6. Click OK to create the team.
   

   Note
   The Address Hash option for the load distribution mode is the 4-tuple hash previously described. PowerShell cmdlets for NIC Teaming allow you to configure other address hashing modes.

   
   
   To modify the settings of a network adapter team, click the team name in the list of teams, and then click Modify team in the list of tasks.
   

NIC Teaming PowerShell commands

Windows Server 2012 NIC Teaming has PowerShell cmdlets that operate on the following object types:

 

Objects to be managed	Supported operations on the object
1. NetLbfoTeam	Get, New, Remove, Rename, Set
2. NetLbfoTeamMember	Add, Get, Remove, Set
3. NetLbfoTeamNic	Get, New, Remove, Set

For further information and complete descriptions of the NIC Teaming PowerShell cmdlets, see the PowerShell cmdlet reference for NIC Teaming.