NSX-T Federation provides networking and security across multiple locations.
With NSX Federation, you can manage multiple NSX-T Data Center environments with a single pane of glass view, create gateways and segments that span one or more locations, and configure and enforce firewall rules consistently across locations.
NSX segments (overlay networks) can span multiple locations.
Span of NSX segment is equal to the span of the attached NSX gateway. These overlay networks of NSX can be connected to Tier 0 Gateway or Tier 1 Gateway ( in case of multi tier routing topology)
Global Manager – which provides a centralized management for networking and security services of multiple locations.
Global Manager Active instance (made of 3 Global Manager VMs) is placed in one Location.
And the Standby Global Manager instance (made of 3 Global Manager VMs) is placed in another location.
In addition to Global Managers, there are NSX-T Local Managers which are used to:
- Configure Transport Zones in a location
- Configure Transport Node Profiles which are used to install NSX on servers
- Install NSX on servers in a location
- Deploy edges and configure NSX on those NSX edges
NSX Gateways in the case of Federation can have a span which covers one location or can be stretched where the gateway spans multiple locations.
Tier-0 gateways can have one of the following configurations:
- Non-stretched tier-0 gateway.
- Stretched active-active with primary and secondary locations.
- Stretched active-active with all primary locations.
- Stretched active-standby with primary and secondary locations
The different topologies possible are documented here
In the case of NSX Federation, NSX edges (which mainly handle north-south traffic in the software defined data center and also provide services like NAT, gateway firewall) have Remote Tunnel Endpont Interface RTEP. This interface handles cross location traffic between different data centers. Essentially the cross site tunnels are established by using this RTEP interface.
Edge TEP interface will have tunnels between itself and TEP interfaces on hosts/servers within the same location. With NSX Federation, there will be no Geneve tunnels between two servers which are in different locations.
In one of my previous blogs, I have covered a topology where stretched Tier 0 Gateway is in Active-Active availability mode with locations as Primary & Secondary.
This blog covers
- routing for stretched Tier 0 Gateway which spans two locations.
- Two locations are Mumbai and Bangalore
- Availability mode on Tier 0 Gateway is Active & Standby
- Stretched Tier 0 Gateway has Mumbai as Primary Location and Bangalore as Secondary Location
In this lab setup, I have prepared NSX Fabric in both locations. There is no standby Global Manager in this lab. For production setup, it is a must to deploy standby Global Manager.
There are site local subnets for host TEP, edge TEP, edge RTEP, edge uplinks
There is NSX local manager in both the locations.
We need to define IP pools on the local NSX manager for host TEP, edge TEP and edge RTEP interfaces.
Besides the default transport zones, edge uplink transport zone has been configured. This transport zone is used to configure uplinks on NSX edges.
We need to configure uplink profiles for:
- Hosts
- Edges
Each server has 4 pnics and there are two VDS’ per cluster.
- One VDS for handling ESXi management, vmotion, NSX traffic
- Second VDS for storage traffic.
Next configure Transport Node Profile which will be attached to cluster. You can have Transport Node Profile on a cluster basis.
Using this transport node profile, servers are prepared for NSX.
Next edge VMs are deployed and configured for NSX.
Two edges have been deployed and edge cluster is created out of those edges.
With this, NSX fabric in Mumbai has been configured.
The above steps are to be followed to prepare NSX fabric in Bangalore location as well.
Next deploy NSX-T Global Managers, add locations and configure RTEP interfaces on edges.
RTEP config has to be done on edges of both locations.
Next, we will login to NSX-T Active Global Manager and configure segments to be used for stretched Tier 0 Gateway uplink interfaces.
We will configure stretched Tier 0 Gateway and specify
- Mumbai as Primary Location
- Bangalore as Secondary Location
- Availability mode as Active-Standby
- Specify edge cluster for each location
Configure Layer 3 interfaces on stretched Tier 0 Gateway
For each interface specify:
- Location
- IP Address
- Connected segment
- Edge Node
Configure BGP
- Specify local AS number for the stretched Tier 0 Gateway
Configure BGP peers. For each peer specify
- Peer IP address
- Location
- Remote BGP AS number
- Source addresses for BGP peering
Enable redistribution on Tier 0 Gateway for each location.
Redistribute connected interface.
Next configure stretched T1, connect it to Tier 0 Gateway and advertise connected networks.
Here Tier 1 Gateway is DR only Tier 1 Gateway which is not connected to any edge cluster.
In this lab, there are no stateful services running on Tier 1 Gateway but they can be enabled based on specific requirement.
Next configure overlay network / segment which is attached to Tier 1 Gateway.
Next from vsphere client, use this overlay network on appropriate VMs.
Validation
Traffic flows
We will validate the HA status of edge in Bangalore and ensure it is in Active state.
BGP routes on edges
Here the key point is that edge in Bangalore location does not advertise routes nor does it receive BGP routes via e-BGP sessions.
Now we will check BGP route learning on active edge in Mumbai location which is Primary location for the stretched Tier 0 Gateway.
BGP paths on router in Bangalore location are the IP addresses belonging to routers in Mumbai location.
Networking & Security 
Excellent!
LikeLike