Worldwide Locations
Amr Ibrahim Enan is a Global Knowledge instructor who teaches and blogs from Global Knowledge Egypt.
In the previous post, we discussed the need for VXLAN in the cloud along with the issues it solves. In this post, we will focus more on how VXLAN works.
As you can see in the figure above, the packets exchanged between VXLAN enabled devices have four headers that encapsulate the Layer 2 frame. Those headers are:
- Ethernet header
- IP header
- UDP header
- VXLAN header
The first question that might come to your mind is why we need all of those headers? Why do we not just add the VXLAN header? In order to understand this, we need to understand what is VXLAN, and how does it work?
VXLAN
VXLAN (Virtual eXtensible Local Area Network) addresses the requirements of Layer 2 and Layer 3 data center network infrastructure in the presence of VMs in a multitenant environment. It runs across the existing networking infrastructure and provides a means to "stretch" a Layer 2 network. In short, VXLAN is a Layer 2 overlay scheme over a Layer 3 network
Only VM’s within the same VXLAN segment can communicate with each other. Each VXLAN segment is scoped through a 24 bit segment ID hereafter termed the VXLAN Network Identifier (VNI). This allows up to 16M VXLAN segments to coexist within the same administrative domain. Hence we have the usage of the VXLAN header in the figure.
The VNI scopes the inner MAC frame originated by the individual VM. Thus, you could have overlapping MAC addresses across segments but never have traffic "crossover" since the traffic is isolated using the VNI qualifier. This qualifier is in an outer header envelope over the inner MAC frame originated by the VM.
Due to this encapsulation, VXLAN could also be termed a tunneling scheme to overlay Layer 2 networks on top of Layer 3 networks. The tunnels are stateless, so each frame is encapsulated according to a set of rules. The end point of the tunnel (VTEP) is located within the hypervisor on the server which houses the VM. Thus, the VNI and VXLAN related tunnel/outer header encapsulation are known only to the VTEP— the VM never sees it. The VTEP we are talking about here is the Nexus 1000V. Nexus 1000V now fully supports the VXLAN technology. For more information, on that visit: www.cisco.com/go/nexus1000v .
VXLAN In Action
Consider a VM within a VXLAN overlay network. This VM is unaware of VXLAN. To communicate with a VM on a different host, it sends a MAC frame destined to the target as before. The VTEP on the physical host looks up the VNI to which this VM is associated. It then determines if the destination MAC is on the same segment. If so, an outer header comprising an outer MAC, outer IP address UDP address, and VXLAN header are inserted in front of the original MAC frame.
Now you might ask yourself why UDP and not TCP — or even why UDP in the first place? Well, the outer UDP header with a source port is provided by the VTEP, and the destination port is a well-known UDP port obtained by IANA assignment. It is recommended that the source port be a hash of the inner Ethernet frame's headers to obtain a level of entropy for ECMP/load balancing of the VM to VM traffic across the VXLAN overlay, which, as we discussed earlier, will now use VPC or VSS instead of STP which relies mainly on Portchannels.
The final packet is transmitted out to the destination, which is the IP address of the remote VTEP that connects the destination VM addressed by the inner MAC destination address.
Upon receipt, the remote VTEP verifies that the VNI is a valid one and is used by the destination VM. If so, the packet is stripped of its outer header and passed on to the destination VM. The destination VM never knows about the VNI or that the frame was transported with a VXLAN encapsulation.
In addition to forwarding the packet to the destination VM, the remote VTEP learns the Inner Source MAC to outer Source IP address mapping. It stores this mapping in a table so that when the destination VM sends a response packet, there is no need for an "unknown destination" flooding of the response packet.
So to summarize, VXLAN will allow you to increase the number of available layer 2 domains by adding the new VXLAN header. Using this header will allow you to have up to 16 M layer 2 domains, and if two devices hosted on the same physical infrastructure have the same address, it will not be a problem as long as it is configured in a different VXLAN.
Also your VXLAN members can be in the same layer 2 domains or in different layer 2 domains since VXLAN can over-relay over layer 3 domains. Devices with the same VXLAN still can communicate with each other.
Also, your traffic will be effectively load balanced over the over-relay network as the UDP source port will be different for each VM starting a new connection.