NSX-V to NSX-T Migration using Layer 2 Bridging

by Farhan Patel, posted in nsx-t, VMware NSX

NSX-V to NSX-T Migration using Layer 2 Bridging

This blog will explore how we can migrate workloads which are on hosts prepared for NSX-V to hosts prepared for NSX-T using NSX-T Layer 2 Bridging.

 

Cluster Setup

In the lab setup, four hosts ESXi 1 up to 4 are prepared for NSX-T and the remaining four hosts ESXi5 up to ESXi 8 are prepared for NSX-V

NSX-T edges used for Layer 2 bridging are on NSX-V prepared hosts.

 

Logical Setup

The above is the logical setup used in this lab.

 

IP Addressing

 

Above shows the IP addressing used in the lab.

 

BGP AS Numbering

 

The above picture shows the BGP AS numbering used.

 

BGP peerings

 

The above diagram shows the BGP peerings.

e-BGP peerings between NSX and the physical network.

i BGP between NSX-V edges and Distributed Logical Router.

There is no routing protocol between Tier 1 Gateway of NSX-T and Tier 0 Gateway upstream.

During migration, traffic flow will be through NSX-V edges which means that:

1. You can prefer not to advertise connected subnets on the Tier 1 Gateway

2. Or to keep BGP disabled on Tier 0 Gateway.

 

NSX-V Setup

NSX-V Prepared Cluster

 

Above picture shows the four hosts prepared for NSX-V
 

NSX-V Edges and DLR

 

The required NSX-V edges and DLR have been deployed.
 

Workloads hosted on both clusters
 
 

VM on VXLAN
 
One VM Windows 10-2 is hosted on this NSX-V prepared cluster.

 

We need to make sure that security settings of the port group (corresponding to the VXLAN being bridged) are set accordingly.

  • Set promiscuous mode on the portgroup.
  • Allow forged transmit on the portgroup.

https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.5/administration/GUID-F133B293-5DEA-4DC8-99DB-6EF004C8D8D7.html

Security settings of VXLAN backed port group

NSX-T Setup

You must change the default MAC address of the NSX-T virtual distributed router so that it does not use the same MAC address that is used by the Distributed Logical Router (DLR) of NSX-V.

The virtual distributed routers (VDR) in all the transport nodes of an NSX-T environment use the default global MAC address. You can change the global MAC address of the NSX-T VDR by updating the global gateway configuration with the following PUT API:

PUT https://{policy-manager}/policy/api/v1/infra/global-config

Note: 
Always refer to NSX-T Data Center Migration Coordinator Guide on VMware Docs for the latest updates. The objective of this post is to familiarise you with how NSX-T Layer 2 bridging works in conjunction with NSX-V to migrate workloads from NSX-V prepared hosts to NSX-T prepared hosts.
Also this post does not cover the micro-segmentation scenario.
 
Compute host transport nodes prepared for NSX-T

 

NSX-T Edges

 

nsx-edge-1 and nsx-edge-2 are NSX-T edges which are used for Layer 2 bridging.
These edges are placed on cluster prepared for NSX-V.
 
The remaining two edges nsx-edge-3 & nsx-edge-4 are used for Tier 0 Gateway.
 
Edge used for L2 Bridging
Fast-path interfaces fp-eth0 and fp-eth1 on the edges used for Layer 2 bridging are used for Geneve traffic, they are uplinked to a trunk port group on VDS used for NSX-V preparation. This way all NSX traffic stays on this VDS which is also used for NSX-V host preparation.
 

NSX-T Edge Clusters

 

Tier 0 Gateway

 

Tier 1 Gateway

 

NSX-T Segment connected to Tier 1 Gateway

 

Gateway set on NSX-T Segment
 

Validation of the setup

VM on NSX-T Segment
 
 
 
 
Reach ability between physical router loopback and both VMs

 

The above picture shows BGP peerings between the router and NSX-V edges.
At this stage, the BGP peerings between the physical routers and NSX-T edges are down/disabled.
Reason being that all workloads from NSX-V prepared hosts are not yet on NSX-T prepared hosts.
 
Reach ability between VM on VXLAN to loopback of physical router

 

Reach ability between VM on NSX-T segment to loopback of physical router

 

At this point, we know that Layer 2 bridging is working as intended and that the layer 2 bridge is forwarding traffic upstream.
 
 
Traffic flow from VM on NSX-T Segment to loopback interface of physical router
 

Migration

Now we will migrate the VM which is on NSX-V prepared cluster to NSX-T prepared cluster.

With this, both the workloads will then be on NSX-T prepared cluster.

At this point of time, we need to ensure that workloads have Tier 1 Gateway as their gateway.

We will ensure BGP peerings between physical routers and NSX-T edges are now all up.

And disable the BGP peerings between physical routers and NSX-V edges

 

Workloads migrated to NSX-T prepared cluster

 

Above picture shows that workloads have moved to NSX-T prepared cluster.

 

After migration, traffic flow from physical router to VMs on NSX-T segment

 

From the physical router, we validate that BGP peerings with NSX-T edges are now up and those with NSX-V edges are down.

Traffic now starts flowing through NSX-T edges.

 

VM on NSX-T Segment to loopback IP of physical router

 

VM on NSX-T segment to loopback IP of physical router

 

Traffic flow after migration

 

Further reading:

NSX Techzone – Techzone covers lot of guidance on NSX-V to NSX-T migration.

2 thoughts on “NSX-V to NSX-T Migration using Layer 2 Bridging”

  1. This is a great blog! Is there anyone to accomplish this if your V infrastructure is using OSPF, and of course our new T will be BGP?

    Like

    Reply

    1. The real answer would rely on the RP in the Core. This is bridging so its all L2. The routing would still from the core have to see your NSX-V OSPF area as the preferred site of ingress/egress. Then, once you cut over the gateway to NSX-T the (U)DLR will stop advertising that subnet and it will start advertising out in BGP. Even if you have that site as the preferred route it is no longer being advertised from the first site so you would then be flowing through your BGP site. Again, this is the Routing Protocol at the core (or wherever the redistribution is taking place), that you would have to play with metrics to ensure symmetric data flow.

      Like

      Reply

Leave a comment