Achieving Deterministic Peering using NSX-T Named Teaming Policies

NSX-T supports two type of Teaming Policies – Default Teaming Policy and Named Teaming Policy. Default Teaming Policy is created by default and applies to both Overlay and VLAN logical Segments when we associate an uplink profile to an NSX-T transport node (hosts or edges). Named Teaming Policies are custom created and are available for the VLAN Logical Segments to override the default teaming policy to achieve better traffic steering by binding the VLAN segments to specific pNIC uplinks in an Active-passive way (or Active only). Note that Named Teaming Policies are available only for VLAN Logical Segments, Overlay Logical Segments always uses the Default Teaming Policy. Named Teaming policies can be used to achieve deterministic peering in the below scenarios:

When the Transport host networking is completely decoupled from vSphere DVS to N-VDS and we need the infrastructure VLAN logical segments (Management, vSAN, vMotion etc) to be bound to specific Uplink interfaces (pNICs). In a 2 pNIC host networking ( 2 X 10G or 2 X 25G) we might see this scenario.
For single N-VDS Multi-TEP Edge nodes (from Version 2.5 onwards) where we need to eBGP peer T0 VLAN Uplinks over specific pNICs to the L3 Leaf Switches. This is applicable for both VM and Baremetal form factors.

There could be few other scenarios as well, but for this blog post lets focus on the above two.

Scenario 1 : Deterministic Uplink Peering on Edge nodes

Let’s revisit the Single-NVDS Multi-TEP Edge architecture in my previous post – https://vxplanet.com/2019/09/23/nsx-t-single-nvds-multi-tep-edge-vm-deployment-configuration-on-vsphere-dvs/

This is a Single NVDS Multi-TEP Edge VM design leveraging vSphere DVS Port Groups. Without deterministic peering for the Edge VLAN Uplinks, this is how the uplink traffic pattern looks like. VLAN 60 & VLAN 70 T0 Uplinks leverage both of the Edge Uplinks (and subsequently DVS Trunk Port Groups as well) for Northbound connectivity.

[Click here for HQ Image]

Let’s apply some traffic steering here so that VLAN 60 always uses Edge Uplink1 (and subsequently DVS Trunk PG 1) and VLAN 70 uses Edge Uplink2 (and subsequently DVS Trunk PG 1) for eBGP peering. To achieve this end to end, we have to do this both on the Edge VM NVDS as well as on the vSphere DVS Trunk Port Groups where the Edge Uplinks are connected. Once done, this is how the T0 VLAN Uplink Traffic pattern looks like:

T0 Uplink VLAN 60 – Edge Uplink1 -> DVS PG Trunk01 -> ESXi host vmnic0 -> L3 Leaf Switch1 and establishes eBGP peering over VLAN 60

T0 Uplink VLAN 70 – Edge Uplink2 -> DVS PG Trunk02 -> ESXi host vmnic1 -> L3 Leaf Switch2 and establishes eBGP peering over VLAN 70

[Click here for HQ Image]

First, let’s configure the Edge nodes to use Named Teaming Policies.

We will edit the Uplink profile for the Edges to include 2 Named Teaming Policies – one for Uplink VLAN 60 and other for Uplink VLAN 70. We choose the Teaming mode as Failover with alternating Edge Uplinks as Active.

T0 Uplink VLAN 60 – Active: Edge Uplink1

T0 Uplink VLAN 70 – Active: Edge Uplink2

We don’t need to include a Standby Edge Uplink here as the failover is already taken care by the other T0 VLAN Uplink. As mentioned earlier, Overlay traffic always uses the default Teaming policy which is ‘Load Balance Source’ over the two Active-Active Uplinks

We will now associate the Named Teaming Policies to the VLAN Transport Zone on which the Edges are a part of.

We will now configure the Uplink VLAN Segments to override the default Teaming Policy with the Named Teaming Policy. This needs to be done from the “Advanced Options”

Now the Edge Uplinks are configured with Deterministic peering using Named Teaming Policies. Now let’s configure the vSphere DVS Trunk Port Groups where the Edge Uplinks are connected.

DVS Trunk Port Group 1 – Active: Uplink1 (vmnic0) & Standby: Uplink2 (vmnic1)

DVS Trunk Port Group 2 – Active: Uplink2 (vmnic1) & Standby: Uplink1 (vmnic0)

We have now achieved deterministic steering for Edges end to end.

Scenario 2 : Deterministic Steering for Infrastructure VLANs on Host Networking

Let’s look at a 2 X 10G (2 pNIC) host networking which leverage N-VDS for both Overlay and infrastructure VLANs. It is completely decoupled from vSphere DVS.

Without any Named Policies applied, all the Logical Segments (including the vmkernels) will use both host uplinks (Active-Active) for outbound connectivity.

To achieve deterministic steering for the infrastructure VLAN Logical Segments, let’s create Named Teaming Policies with alternating Active and Passive Uplinks.

Management vmk0 – Active: Uplink1 (vmnic0) and Standby: Uplink2 (vmnic1)

vSAN vmk1 – Active: Uplink1 (vmnic0) and Standby: Uplink2 (vmnic1)

vMotion vmk2 – Active: Uplink2 (vmnic1) and Standby: Uplink1 (vmnic0)

We will now associate the Named Teaming Policies to the VLAN Transport Zone on which the Transport Hosts are a part of.

We will now configure the Infrastructure VLAN Segments to override the default Teaming Policy with the Named Teaming Policy. This needs to be done from the “Advanced Options”

Finally use the NVDS Visualization option for the host to confirm that the Traffic pinning is configured as expected.

As an additional note, we could use this at the Transport Node profile to apply the Policy Settings at the vCenter Cluster level so that all the hosts are consistent according to the cluster specifications.