the_information_nexus/docs/tech_docs/linux/vxlan.md

Certainly! Setting up a bridge-to-bridge connection across different hosts using tunneling technologies like VXLAN is an advanced networking topic that's particularly useful in modern data centers and cloud environments. Here, we'll cover this topic in detail, focusing on VXLAN as a popular choice due to its scalability, flexibility, and support across various networking hardware and software.

Understanding VXLAN

VXLAN (Virtual Extensible LAN) is a network overlay technology designed to provide the same services as VLAN but with greater extensibility and flexibility. It encapsulates Ethernet frames in UDP packets and uses a 24-bit VXLAN Network Identifier (VNI) to allow for about 16 million isolated Layer 2 networks within a common Layer 3 infrastructure.

Why Use VXLAN?

1. Scalability: Overcomes the 4096 VLAN ID limit, supporting up to 16 million virtual networks.
2. Flexibility: Can be used over any IP network and across different data centers or cloud environments.
3. Compatibility: Works with existing virtualization technologies and can be implemented in software or supported by physical network hardware.

Setting Up VXLAN for Bridge-to-Bridge Communication

Prerequisites:

• Two hosts, each with at least one network interface.
• IP connectivity between the hosts.
• Kernel support for VXLAN (common in modern Linux distributions).

Configuration Steps:

Step 1: Install Necessary Tools Ensure iproute2 is updated as it contains necessary tools for managing VXLAN interfaces.

sudo apt-get update
sudo apt-get install iproute2

Step 2: Create Bridges on Each Host First, you need to set up a bridge on each host. Here's how you might set up a bridge on both Host A and Host B:

# On Host A
sudo ip link add br0 type bridge
sudo ip link set br0 up

# On Host B
sudo ip link add br0 type bridge
sudo ip link set br0 up

Step 3: Create VXLAN Interface On each host, create a VXLAN interface. This example uses VXLAN ID 42 and assumes the source IP addresses are static and known.

# On Host A
sudo ip link add vxlan42 type vxlan id 42 dev eth0 dstport 4789 remote <IP_OF_HOST_B> local <IP_OF_HOST_A> nolearning
sudo ip link set vxlan42 up
sudo ip link set vxlan42 master br0

# On Host B
sudo ip link add vxlan42 type vxlan id 42 dev eth0 dstport 4789 remote <IP_OF_HOST_A> local <IP_OF_HOST_B> nolearning
sudo ip link set vxlan42 up
sudo ip link set vxlan42 master br0

Replace <IP_OF_HOST_A> and <IP_OF_HOST_B> with the respective IP addresses of your hosts.

Step 4: Assign IP Addresses (Optional) For management or testing, you might want to assign IP addresses to the bridge or to virtual interfaces attached to the bridge.

# On Host A
sudo ip addr add 192.168.1.1/24 dev br0

# On Host B
sudo ip addr add 192.168.1.2/24 dev br0

Step 5: Testing Connectivity Use ping or other network tools to test connectivity between the hosts.

# On Host A
ping 192.168.1.2

Advanced Topics

• Security: Consider using IPsec to secure VXLAN traffic, especially when traversing untrusted networks.
• Dynamic VXLAN Setup: For dynamic environments (like those managed by Kubernetes or OpenStack), look into automating VXLAN setup with network controllers or using protocols like EVPN.
• Performance: Monitoring and tuning the performance of VXLAN tunnels is crucial, especially in high-throughput environments. Techniques include offloading VXLAN processing to network hardware, tuning MTU settings, and using jumbo frames.

Conclusion

VXLAN provides a robust method for extending Layer 2 networks over Layer 3 infrastructures. When properly configured, it enables flexible, scalable, and secure network designs across geographically dispersed locations. This setup is especially beneficial in environments where virtualization and containerization are heavily used, allowing seamless connectivity across various hosts and clusters.

graph TD;
    subgraph Site A
        A_OPNsense[OPNsense Gateway A<br>192.168.10.1] --> A_Debian[Debian A<br>10.0.0.1<br>VXLAN ID 100]
    end
    subgraph Site B
        B_OPNsense[OPNsense Gateway B<br>192.168.20.1] --> B_Debian[Debian B<br>10.0.0.2<br>VXLAN ID 100]
    end
    subgraph Site C
        C_OPNsense[OPNsense Gateway C<br>192.168.30.1] --> C_Debian[Debian C<br>10.0.0.3<br>VXLAN ID 100]
    end
    A_Debian --- B_Debian
    B_Debian --- C_Debian
    C_Debian --- A_Debian