vSphere Overview Video
What is vCenter (Watch the first two minutes)
When they understand basic vSphere terms like vCenter and ESXi we can start talking about virtual networking.
First thing first, VMware vSwitch is not a switch. Let me repeat it again ...
VMware vSwitch is not a typical ethernet switch.It is not a typical network (ethernet) switch because not all switch ports are equal. In VMware vSwitch you have to configure switch uplinks (physical NICs) and internal switch ports (software constructs). If the ethernet frame is coming from the physical network via uplink, vSwitch will never forward such frame to any other uplink but only to internal switch ports, where virtual machines are connected. This behavior guarantees that vSwitch will never cause the L2 loop problem. It also means that vSwitch does not need to implement and participate in spanning tree protocol (STP) usually running in your physical network. Another different vSwitch behavior compared to traditional ethernet switch is that vSwitch does not learn external MAC addresses. It only knows about MAC addresses of virtual machines running on particular ESXi host (hypervisor). Such devices are often called port extenders. For example, CISCO FEX (fabric extender) is a physical device having the same behavior.
Now let's talk about network redundancy. In production environments, we usually have a redundant network where multiple NICs are connected to different physical switches.
|Each NIC connected to different physical switch|
- VMware vSphere standard switch (aka vSwitch or vSS)
- VMware vSphere distributed virtual switch (aka dvSwitch or vDS)
VMware vSphere standard switch (vSS)
VMware vSphere standard switch supports multiple switch independent active/active and active/standby teaming methods and also one switch dependent active/active teaming method.
The standard switch can use following switch independent load balancing algorithms:
- Route based on originating virtual port - (default) switch independent active/active teaming where the traffic is load balanced in round-robin fashion across all active network adapters (NICs) based on internal vSwitch port id where virtual machine vNIC's or ESXi vmKernel ports are connected.
- Route based on source MAC hash - switch independent active/active teaming where the traffic is load balanced in round-robin fashion across all active network adapters (NICs) based on source MAC address identified in standard vSwitch.
- Use explicit failover order - is another switch independent teaming but active/passive. Only one adapter from all active adapters is used and if it fails the next one is used. In other words, it always uses the highest order uplink from the list of Active adapters which passes failover detection criteria.
- Route based on IP hash - switch dependent active/active teaming where the traffic is load balanced based on a hash of the source and destination IP addresses of each packet. For non-IP packets, whatever is at those offsets is used to compute the hash. This is switch dependent teaming, therefore, the static port-channel (aka ether-channel) has to be configured on the physical switch side otherwise, it will not work.
VMware vSphere distributed switch (vDS)
If you have vSphere Enterprise Plus license or VSAN license you are eligible to use VMware vSphere distributed switch. VMware distributed switch key advantages are
- centralized management
- advanced enterprise functionality
- NIOC (Network I/O Control) which allows QoS and marking (802.1p tagging, DSCP)
- LBT (Load Based Teaming) - official name is "Route based on physical NIC load"
- LACP - dynamic switch dependent teaming
- Route based on physical NIC load - another switch independent teaming with optimized load balancing
- ACLs - Access Control Lists
- Port mirroring
- Configuration backup and restore
- and more
- Switch Independent - LBT
- Switch Dependent - LACP
LBT is proprietary VMware solution where vmkernel checks every 30sec the stats from the relevant pNIC's and the calculation use the stats for the 30sec interval and get an average over the 30sec (to normalize the average and to eliminate spikes). If the bandwidth is above 75%, it will mark the pNIC as saturated. If a pNIC have been marked as saturated, the vmkernel will not move any more traffic onto the saturated pNIC. Read blog post "LBT (Load Based Teaming) explained" for more details (see links to references below this article).
LACP is standardized link aggregation protocol. It is worth to mention, that when LACP is used you can leverage significantly enhanced load balancing algorithms to more optimal bandwidth usage of physical NICs. Theoretically, you can use for single VM more bandwidth than is the bandwidth of single pNIC. But to see it in the real world, multiple flows must be initiated from that single VM and LACP the result of the hash algorithm must route it across multiple links bundled in a LAG.
vSphere 6.0 LACP supports following twenty (20) hash algorithms:
- Destination IP address
- Destination IP address and TCP/UDP port
- Destination IP address and VLAN
- Destination IP address, TCP/UDP port and VLAN
- Destination MAC address
- Destination TCP/UDP port
- Source IP address
- Source IP address and TCP/UDP port
- Source IP address and VLAN
- Source IP address, TCP/UDP port and VLAN
- Source MAC address
- Source TCP/UDP port
- Source and destination IP address
- Source and destination IP address and TCP/UDP port
- Source and destination IP address and VLAN
- Source and destination IP address, TCP/UDP port and VLAN
- Source and destination MAC address
- Source and destination TCP/UDP port
- Source port ID
esxcli network vswitch dvs vmware lacp
esxcli network vswitch dvs vmware lacp config get
esxcli network vswitch dvs vmware lacp status get
esxcli network vswitch dvs vmware lacp timeout set
Hope this was informative and useful.
References to other useful resources
- VMware : vSphere Networking (official documentation)
- James Green : LAG vs. LBT for vSwitch Uplinks in vSphere
- Hugo Strydom : LBT (Load Based Teaming) explained - Part 1, Part 2, Part 3
- Chris Wahl : Exploring Enhanced LACP Support with vSphere 5.5
- Steven Kang : ADVANCED LACP CONFIGURATION USING ESXCLI