Deploying and Configuring NSX-T Edges on N-VDS Networking

NSX-T Edges have flexible deployment options based on the host networking that is used. We can deploy NSX-T Edges on Distributed vSwitches (which is managed by vCenter Server) or on host N-VDS (which is managed by NSX-T). NSX-T Edges deployment and configuration on DvSwitches has been covered in my previous post, you can find it here:

https://vxplanet.com/2019/05/23/deploying-the-nsx-t-edge-vm-cluster-leveraging-vsphere-dvs-portgroups/

Whether to use DvSwitch or N-VDS or both depends upon the number of pNICs available on the host networking. If the vSphere environment is based on a 2 pNIC host configuration, we could migrate all the host networking to N-VDS and disassociate the DVS from the ESXi hosts. I have covered the migration from DVS to N-VDS in my earlier article, you can find it here:

https://vxplanet.com/2019/05/27/nsx-t-n-vds-topologies-and-migrating-host-networking-between-vsphere-dvs-n-vds/

In this article, we will walk though the steps to deploy and configure the NSX-T Edges on N-VDS host networking. This approach is suitable if you are deploying on a 2 pNIC host platform. This also works on an NSX-T based Workload domain in VMware Cloud Foundation. Let’s get started.

Environment Details

  • 4 x Dell EMC PE R640 nodes as ESXi hosts
  • Collapsed Management, Compute and Edge vSphere 6.7U2 vSAN cluster
  • 2 X 25G host networking connected to Dell Networking L3 ToR switches in VLT
  • NSX-T 2.4.1 with a 3 node management cluster. All the 4 ESXi hosts are configured as NSX-T Transport nodes.
  • Host networking is completely decoupled from vCenter DVS and migrated to N-VDS

Current State

This is the current state of the NSX-T platform. The hosts are configured with 2 Transport Zones – One Overlay and another VLAN, both leveraging the same N-VDS.

[Click here for HQ Image]

2

[Click here for HQ Image]

3

The host Uplink Profile configures the host TEP VLAN and the teaming policies. We use VLAN 40 for TEP encapsulation and Load balancing as the Teaming policy.

[Click here for HQ Image]

4

We create 4 VLAN Segments on the N-VDS which is used for host networking (management, vSAN, vMotion, VMNetwork etc)

[Click here for HQ Image]

5

The N-VDS is visible to all the ESXi hosts and the host networking (vmk ports, virtual machines etc) are migrated to N-VDS. vCenter DVS is disassociated from the ESXi hosts.

[Click here for HQ Image]

1

Creating VLAN Transport Zones for the Edge Uplinks

We have to create VLAN Transport Zones based on the Uplink Interfaces that we use on the Edge nodes. We have two uplink VLANs for the Edges – VLAN 60 and VLAN 70, so we create two Transport zones. These Transport zones are not a part of the ESXi Transport nodes, they are only available to the Edge nodes. Each Uplink Transport zone will have a separate N-VDS on the Edges.

[Click here for HQ Image]

10

11

12

[Click for HQ Image]

13

 

Since NSX-T Edges are deployed on host N-VDS, the VLAN for the Edge TEP should be different from the ESXi host TEP. The host TEPs are on VLAN 40, so we choose the Edge TEPs to be in VLAN 80. We create 3 Logical Segments for the Edge Connectivity to the host N-VDS.

  • One for the Edge TEP – VLAN 80
  • Another for the Edge Uplink 1 – VLAN 60
  • Another for the Edge Uplink 2 – VLAN 70

20

[Click here for HQ Image]

21

[Click here for HQ Image]

22

[Click here for HQ Image]

23

[Click here for HQ Image]

These VLAN Logical Segments should now reflect on the host N-VDS.

[Click here for HQ Image]

24

Creating the Edge Uplink Profile

We could use the pre-created single nic Edge uplink profile. Make sure NOT to put VLAN tag on the Edge Uplink Profile. The Tag for the Edge TEP is applied by the host N-VDS.

[Click here for HQ Image]

30

Deploying the first Edge Node

We will use the Edge ova file to deploy the Edge nodes. I’ve already downloaded the ova from my VMware account.

4041

42

43

I’ve used Medium form factor for the deployment.

44

Select the datastore. Since our cluster is vSAN enabled, let’s place it on the vSAN datastore.

45

Configure the Networking. Edges are deployed with 4 vnics. We have to map the vnics to the appropriate Logical segments on the host N-VDS. Just in case, we use only one edge uplink, disconnect the 4th vnic. This is how the networks are attached:

  • Network 0 → Management VLAN Logical segment (VLAN 10)
  • Network 1 → Edge TEP (VLAN 80)
  • Network 2 → Uplink 1 Logical Segment (VLAN 60)
  • Network 2 → Uplink 2 Logical Segment (VLAN 70)

46

Configure the Management network, passwords, DNS, NTP and other basic settings. 

4748

Review the settings and click Finish to start the deployment of the first Edge node VM.

49

Power on the Edge VM and wait for it to initialize.

50

SSH to the Edge VM and perform basic connectivity checks.

5152

Joining Edge to the NSX-T Management Plane

Generate the Certificate Thumbprint from the NSX-T manager.

60

Join Edge to the management plane. [Click here for HQ Image]

61

The Edge should now appear under the “Edge Transport Nodes” section in the NSX-T Manager UI.[Click here for HQ Image]

62

Configuring the Edge as an NSX-T Transport Node

The NSX-T Edge VM will be a part of minimum 2 transport zones – one will be the overlay Transport zone and the other one will be the Uplink VLAN Transport zone. In our case, we have 2 Edge Uplinks – each on separate VLANs and with separate ports, hence we configure the Edges as a Transport node for the 3 Transport zones.

70

 

We have to configure 3 N-VDS here:

  • Overlay N-VDS is where the Edge TEP is configured. It’s uplink maps to fp-eth0 (This is the second adapter on the Edge VM)
  • Uplink1 VLAN N-VDS – It maps to the third interface fp-eth1
  • Uplink2 VLAN N-VDS – It maps to the forth interface fp-eth2

72

We could either use a Static entry or use IP Pool for the Edge TEPs. We can create an IP Pool directly from this configuration box.

71

7374

Once configured, verify the Edge transport node status.

[Click here for HQ Image]

75

Deploying the second NSX-T Edge and configuring as a Transport node

The procedure is exactly similar to the above except that management IP of 192.168.10.172/24 is used for the second node.

Once deployed and configured as a transport node, both Edges should display a healthy status.

[Click here for HQ Image]

80

Configuring the Edge Cluster

We need to create an Edge cluster and add both Edge nodes as its members.

[Click here for HQ Image]

90

91

We need two VLAN Logical segments created on the Edge Uplink Transport Zones, so that the Tier 0 Gateway can attach to the Edge Uplinks. 

Note that we need not have to put a VLAN tag here, as the tagging is applied at the host N-VDS level ie, on the VLAN Uplink Logical Segments that we created earlier.

[Click here for HQ Image]

100

[Click here for HQ Image]

101

Creating Tier 0 Gateway

We will now create a Tier 0 gateway on the Edge Cluster.

110

[Click here for HQ Image]

111

[Click here for HQ Image]

112

We will create two uplinks for the T0 gateway – One on VLAN 60 via the first Edge node and second on VLAN 70 via the second Edge node.

113

114

Confirm that the Uplink interfaces are initialized and are up.

115

 

Validating the External Connectivity

We will ssh to the Edge nodes and perform a connectivity test to the external ToR switches.

116

SUCCESS!!! Tier 0 gateway can establish communication to the ToRs via Edge node 1 over VLAN 60. Lets try the other Edge node.

117

SUCCESS!!! Tier 0 gateway can establish communication to the ToRs via Edge node 2 over VLAN 70.

We are now good to deploy the overlay logical segments, Tenant T1 routers and BGP peering with the L3 ToRs. 

I hope the post was informative. Thanks for reading

nsxrun

2 thoughts on “Deploying and Configuring NSX-T Edges on N-VDS Networking

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s