BGP EVPN with VXLAN

Technical references:

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html

About VXLAN:

VXLAN is a network virtualization technology that allows for the extension of Layer 2 networks over a Layer 3 physical infrastructure.

It helps overcome the limitations of traditional VLANs, especially in large data centers.

VXLAN allows much better scalability as compared to traditional VLANs with the ability to provision many VXLANs (around 16 million) as compared to the 4,096 VLAN limit. It also overcomes the limitations of spanning tree protocol in a traditional layer 2 network whereby redundant layer 2 trunk interface on the access layer 2 switch towards the core network is in blocking state and the other L2 link is in forwarding state, thus the north-south bandwidth is reduced in a network that runs Spanning Tree Protocol even though the access switch has dual high capacity uplinks.

In a VXLAN overlay network, a VXLAN network identifier (VNI) uniquely identifies each Layer 2 subnet or segment.

With reference to encapsulation, the original IP packet is encapsulated in a new UDP packet with the outer header containing source and destination IPs of Tunnel End Point IP addresses.

Also note that VXLANs provide the same kind of segmentation and isolation that is achieved using VLANs.

When can one reap the benefits of VXLANs or overlay networking?

1. When you need to provision many networks and the VLAN limit of 4096 VLANs will easily be reached. This can easily happen in environments where multi-tenant architecture is required and you need to provision multiple customer networks on top of a shared networking infrastructure. The objective here is to scale data center networking to support multi-tenancy and isolation.
2. VXLAN can allow multi-site connectivity between data centers where it is possible to stretch network between two different data centers.
3. Overcome the limitations of physical server racks and geographical location boundaries and achieve flexibility for workload placement within a data center or between different data centers.

Do I need to go VXLAN always? Or VLANs will be sufficient for me? You will be okay with traditional VLANs in case:

1. The networking requirements in your data center are static, where you don’t need to provision new networks on a daily basis.
2. Multi-tenancy is not needed and there is no need to provision networks belonging to different customers on a shared networking infrastructure.
3. There are limited number of applications and new applications are not being deployed rapidly. You are anticipating very slow growth for applications.
4. Where exponential growth in terms of networks is not expected.

Why BGP EVPN along with VXLAN and why VXLAN alone was not sufficient?

1. VXLAN alone requires manual configuration of routes and MAC addresses, which can be time-consuming, will require a lot of administrative effort and is error-prone in large-scale networks. BGP EVPN provides a scalable and dynamic routing protocol that can handle large-scale VXLAN networks.
2. BGP EVPN enables seamless connectivity between multiple VXLAN sites.
3. Limitations of flood and learn mechanism of multicast: Flood and Learn (F&L) is an component of VXLAN networks, and it is used for distributing MAC addresses among the leaf switches (assume here a leaf and spine CLOS fabric). In large-scale networks, flooding multicast packets can consume significant network bandwidth from the underlay physical network and there could be network links which could easily get congested. Multicast based flood and learn VXLAN meant configuring multicast in the underlay physical network which again is administrative overhead, could be error prone.

BGP EVPN:

To overcome the above challenges, BGP EVPN came into the picture as control plane for VXLAN.

1. BGP EVPN advertises both layer 2 and layer 3 reachability information for end hosts that reside in EVPN VXLAN overlay network.
2. Built-in multi tenancy support is an advantage of MP-BGP EVPN VXLAN compared to multicast-based flood-and-learn VXLAN and other Layer-2 extension technologies without multi-tenancy capabilities
3. It is important to note that BGP EVPN does not serve as a control plane for VXLAN alone but it also has support for MPLS and NVGRE too.
4. It minimizes network flooding through MAC and IP route distribution and through ARP suppression.
5. It supports workload mobility. In a multi site or multi region architecture, workload mobility can be used to facilitate disaster recovery by allowing workloads to be moved to backup sites in case of a failure. When a workload is created or moved, the leaf switch where it is located advertises its MAC address to the other leaf switches in the VXLAN fabric using BGP EVPN. BGP EVPN also advertises the IP address of the workload’s virtual interface to the other leaf switches.
6. It gets rid of Spanning Tree Protocol STP limitation in a traditional layer 2 network which potentially introduces inefficient bandwidth usage of switch uplinks where traffic is pinned to one of the uplinks of access L2 switch. With BGP EVPN, active-active redundant links achieve full utilization of network bandwidth.

Use of route distinguisher RD and route target RT in the case of BGP EVPN:

Route distinguisher: An RD acts as a unique identifier for a VRF within a tenant. This ensures that routes from different VRFs do not conflict with each other. RDs help isolate the routing domains of different VRFs, preventing unauthorized traffic flow between them.

Route target RT: RDs are used in conjunction with Route Targets (RTs) to control which routes are imported and exported between VRFs.

Example of RD and RT in VRF based BGP EVPN

Scenario:

• A data center has two VRFs: VRF A and VRF B.
• VRF A represents the production network, while VRF B represents the management network.

Configuration:

VRF A:

• RD: 65000:100
• RT: 65000:100 (import and export)

VRF B:

• RD: 65000:200
• RT: 65000:200 (import and export)

Configuration on Leaf Switches:

• Configure the RDs and RTs for each VRF on the leaf switches of CLOS fabric.
• Advertise routes for the respective VRFs using BGP EVPN.

Key Points:

• The RDs (65000:100 and 65000:200) differentiate the routes belonging to VRF A and VRF B.
• The RTs (65000:100 and 65000:200) control which routes are imported and exported between the VRFs.
• This configuration ensures that traffic from VRF A cannot directly reach devices in VRF B, providing isolation and security.

By using RDs and RTs in this manner, organizations can effectively manage and control routing within VRF-based BGP EVPN networks, ensuring that traffic flows correctly and securely between different VRFs.