Multi-tenant private cloud networks: VLAN limits, VXLAN and EVPN, and tenant isolation

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

A VLAN gives you 12 bits of tenant identifier. VXLAN gives you 24 bits and removes the need for one large Layer 2 network between hosts. BGP EVPN advertises learned destinations through a control plane, reducing reliance on flood-and-learn forwarding. Broadcast, unknown-unicast and multicast traffic still needs handling.

None of the three is a security boundary by itself. A VLAN tag or a VXLAN Network Identifier (VNI) is a label in a header. Tenant isolation holds only if nobody untrusted can write that label, and only if routing between tenants is denied by design.

The commands below were not run in a lab for this article. Interface names, addresses and VNI numbers are examples. Show commands are read-only; configuration changes on a live fabric can drop every tenant at once, so test them in an isolated environment first.

Recommendations and interpretations are editorial unless attributed; source research is not lab validation.

Why it matters

Tenant networks fail in two very different ways. The first is an outage: a tenant cannot reach its own second server. The second is a leak: a tenant reaches something belonging to someone else, and nothing alerts.

Outages get reported within minutes. Leaks may never be reported. That is why the design has to state where separation is enforced, and how you would prove it.

Scale forces the design decision. RFC 7348 describes the pressure directly: a 12-bit VLAN ID serves networks needing fewer than 4,094 VLANs, a provider may need several VLANs per tenant, and top-of-rack switches must learn the MAC address of every virtual machine that passes through them. If a table overflows, the switch may stop learning and flood frames for unknown destinations.

How it works

VLANs and QinQ

An 802.1Q tag carries a 12-bit ID. Each tag defines one broadcast domain, and the whole path between hosts must carry that VLAN.

QinQ stacks a service tag outside a customer tag. Proxmox VE's implementation uses the zone for the outer tag and the VNet for the inner one, and notes that you need four more bytes, or a guest MTU reduced to 1496 on a 1500-byte interface.

QinQ extends the numbering. It does not change the underlying model: the physical network is still one Layer 2 domain, which still needs a loop-free topology. RFC 7348 points out that spanning tree blocks links that you paid for, and that some sites limit VLAN counts because of it.

VXLAN: a 24-bit label inside UDP

VXLAN wraps the tenant's Ethernet frame in an outer IP and UDP packet. The endpoint that does this is a VXLAN Tunnel End Point, or VTEP, usually inside the hypervisor. RFC 7348 defines a 24-bit VNI, which allows about 16 million segments, and overlapping MAC addresses across segments are permitted because the VNI scopes the frame.

Four details matter when you operate it:

• Port. IANA assigned UDP 4789. The Linux kernel keeps an older default for backward compatibility, so state dstport 4789 explicitly on hand-built devices.
• Load sharing. The sender should derive the outer UDP source port from a hash of the inner frame. That gives the routed underlay something to spread traffic on across equal-cost paths.
• Fragmentation. VTEPs must not fragment VXLAN packets, and a receiving VTEP may silently discard fragments that a router created along the way.
• Inner VLAN tags. Decapsulated frames with an inner VLAN tag should be discarded unless configured otherwise, so a guest that tags its own traffic needs an explicit design.

The MTU arithmetic

Proxmox VE states the VXLAN overhead as 50 bytes and defaults EVPN zones to an MTU of 1450. The 50 bytes follow from the header layout in RFC 7348. An IP MTU is counted from the outer IP header inwards, so this is what sits on top of the guest's 1500-byte IP packet:

The outer Ethernet header is also 14 bytes and makes the frame longer on the wire, but like an outer VLAN tag (4 bytes) it is not part of the IP MTU. An outer IPv6 header is 20 bytes larger than a minimum IPv4 header. If your design lets guests send tagged frames, the optional inner tag adds another 4 bytes inside the encapsulation.

So there are two honest options. Either the guests run at 1450, or the underlay carries an IP MTU of at least 1550 so that guests keep 1500. RFC 7348 recommends sizing the physical network to fit the encapsulation.

Do not leave this half done. Small packets pass and large ones vanish, so ping works and the application hangs.

Why flood-and-learn does not suit tenants

RFC 7348's base scheme learns MAC-to-VTEP mappings from data traffic and uses IP multicast for broadcast, unknown unicast and multicast frames. It needs multicast in the underlay and gives you no central record of what should be where.

EVPN: a control plane for the overlay

RFC 7432 defines BGP EVPN, carried as address family L2VPN (25) with sub-family EVPN (70). RFC 8365 applies it to VXLAN-style overlays. Hosts advertise what they have instead of discovering it from traffic.

The route types you will read most often are:

Type 3 routes carry a tunnel attribute. With ingress replication, a broadcast packet is sent to every remote host in the instance, so the sending host makes the copies itself and the underlay does not need multicast. The cost lands on the sender, so a broadcast-heavy tenant consumes more host bandwidth as its segment spreads across hosts.

Route targets decide membership

This is the central point for isolation. FRRouting's EVPN documentation states it plainly: IP-VRFs and EVPN instances are identified by route targets, never by labels. The VNI carried in a route is data-plane information only.

It also gives the most useful diagnostic rule in EVPN work. A route target mismatch shows up as routes missing from the receiving VRF or instance. A label problem shows up as routes imported, but traffic dropped or misdelivered.

A tenant's isolation in the control plane is therefore a route target policy. In current FRRouting, automatic import targets are wildcards of the form *:VNI, and automatic export targets are (AS & 0xFFFF):VNI. Those defaults are convenient in a fabric with one VNI plan.

Our recommendation, not a vendor statement: if you peer two EVPN domains, or reuse VNI numbers between administrative domains, set route targets by hand. A wildcard import matches on the VNI value alone. RFC 8365 makes the related point for gateways: when targets are auto-derived from VNIs, there must be no conflict between the data centre and WAN target spaces, assuming both operate in the same Autonomous System.

Routing inside a tenant

RFC 9135 describes two ways to route between a tenant's subnets:

• Asymmetric routing does the routing lookup at the ingress side and bridges at the egress side. Each host must hold ARP entries for remote hosts and needs all of the tenant's subnets configured.
• Symmetric routing does a lookup on both sides between IP-VRFs. Each host only keeps ARP entries for locally connected hosts and only its locally configured subnets.

Symmetric routing uses a separate Layer 3 VNI per tenant VRF. In FRRouting, a VRF becomes L3 only through the vni statement under its vrf stanza; until then zebra treats the VNI as Layer 2. Proxmox VE calls it the VRF VXLAN ID and requires it to differ from the VNet IDs.

A Linux VRF gives each tenant its own routing table, which is what lets two tenants use the same private address range. RFC 7348 gives overlapping tenant addresses as a reason Layer 3 alone does not provide multi-tenancy.

Where isolation actually happens

Work through these layers, because each one fails separately.

The underlay. RFC 7348 is direct about this: VXLAN extends the attack surface, and the document incorporates no specific measures against spoofed traffic. Anyone who can send UDP 4789 to a VTEP can attempt to inject frames into a segment. Put VTEP addresses on a dedicated network. Use 5-tuple filters so that only known VTEPs can reach port 4789.

Encryption. VXLAN provides none. Proxmox VE warns to use a site-to-site VPN when joining sites, and RFC 7348 points to IPsec. Treat any link leaving your controlled facility as requiring this.

The VTEP's accepted VNIs. On Linux, a device in external mode with vnifilter receives only VNIs in its filtering table. That reduces exposure. It is filtering, not authentication.

The guest port. A guest with raw network access should not be able to claim another MAC or IP. Proxmox VE's firewall has per-interface ipfilter and macfilter options, and three conditions decide whether they work. Each virtual interface has its own firewall enable flag, required in addition to the general enable option. Once an ipfilter-net* set exists for an interface, outgoing traffic whose source IP is not in the set is dropped. And only containers with configured addresses get those addresses added automatically; for a VM the set implicitly holds just the link-local address. So enabling IP Filter on a VM with no addresses in its set can cut off its traffic, while leaving the interface flag off gives no protection. Fill the set for each VM interface first, then confirm the filter is in effect.

Inside one tenant network. A Linux bridge port flagged isolated can only talk to non-isolated ports. Proxmox VE's Isolate Ports option is local to each host, so two guests on different hosts are not separated by it. Proxmox VE directs you to the VNet firewall for that, and VNet-level and forward rules only work with the nftables-based proxmox-firewall. That documentation calls it a tech preview, warns of possible bugs and incompatibilities, and says: "It is currently not suited for production use." The stock pve-firewall ignores forward rules, so a rule can be present and do nothing.

That leaves a gap on this path, because the only documented cross-host control inside one tenant network is one the vendor does not recommend for production. Our recommendation, not a vendor statement: in production, do not rely on it. Put guests that must stay apart in separate VNets, with any permitted traffic passing through a firewall you already operate between them.

Between tenants. Separate VRFs with no shared route targets do not exchange routes through that fabric configuration. External routers, gateways and other policy paths can still connect them; inventory those as well as deliberate imports. Keep a list of them.

Exit points. In Proxmox VE, EVPN exit nodes announce a default route into the zone. Tenants meeting at a common exit gateway must be separated there too, because the shared router is the natural place for a leak.

Finding a fault: the safe order

Start at the bottom. Configuration queries inspect state; ping is an active probe that sends traffic.

1. Prove the underlay carries the full size. From one host to another VTEP address, with the Don't Fragment bit set using ping -M do:

# For an underlay IP MTU of 1550: 1550 - 20 (IP) - 8 (ICMP) = 1522
ping -M do -s 1522 -c 3 192.0.2.12

A failure here narrows the investigation to endpoint reachability, filtering and the underlay path. Compare small probes and ICMP errors; one failed large ping does not prove an MTU fault. Linux VXLAN devices do not set the DF bit by default, so an underlay router may fragment, which a receiving VTEP may then discard.

2. Read the device. Check the VNI, port, local address and whether it is in external mode:

ip -d link show vxlan100

3. Check what the host will do with a destination MAC. The bridge fdb table shows the VTEP a MAC maps to:

bridge fdb show dev vxlan100

The extern_learn flag marks an entry learned by hardware or a user-space controller, and the kernel does not age such entries. On a host where a routing daemon installs remote MACs, that is the flag to expect. An expected remote MAC that is absent points back to the control plane.

4. Read the control plane. These commands are in the FRRouting BGP and EVPN documentation. Those pages are the unversioned "latest" edition, so output and options may differ on your release:

show evpn vni
show vrf all vni
show evpn mac vni 100 detail
show bgp l2vpn evpn route type macip
show bgp l2vpn evpn route type multicast
show bgp vni 100 type macip

show evpn vni tells you whether a VNI is considered L2 or L3. show vrf all vni shows the VRF-to-L3VNI mapping and the router MAC.

5. Apply the diagnostic rule. These are likely readings drawn from the documentation above, not a vendor's fault table:

6. Watch the wire only if needed. A capture using the filter udp port 4789 on the underlay interface shows whether encapsulated traffic leaves and arrives. Tenant payload will be visible in that capture, so handle it under the access rules that apply to tenant data.

Duplicate MAC behaviour

EVPN uses a sequence number to track moves. RFC 7432 defines a default detection window of N=5 moves within M=180 seconds, after which a host must alert the operator and stop sending and processing routes for that MAC until someone corrects it.

Expect this when a tenant clones a VM without changing its MAC. Both values are required to be configurable, so read your implementation's settings.

NIC offloads

The kernel documentation names the features that indicate tunnel offload support, including tx-udp_tnl-segmentation for TCP segmentation of UDP-encapsulated frames, and ethtool --show-tunnels for listing offloaded UDP ports. Support is specific to each NIC. If throughput is poor while everything else is correct, this is where to look next.

What to choose and why

These are our recommendations.

Before you commit to EVPN, check the support status for your platform. As of the 9.2.13 documentation, Proxmox VE lists core SDN as fully supported, but complex routing via FRRouting and controller integration as technology preview. EVPN zones depend on that controller.

Allocate numbers as a plan, not per request. Decide the VNI ranges for L2 and L3 use, and treat VNIs as globally unique inside a data centre network unless you deliberately translate them at the boundary.

Risks and limits

• Do not run VXLAN across a network you do not control without encryption.
• Do not rely on host-local port isolation for separation across hosts.
• Do not assume silent guests are reachable. Proxmox VE notes that guests the anycast gateway has not seen may be unreachable inside EVPN unless you advertise the subnet.
• Floating IPs need care. Proxmox VE says ARP and ND suppression must be disabled if addresses move between systems.
• A BGP session is also an attack surface. RFC 8365 warns that altered encapsulation information can misroute or misdeliver user data, so protect peering and restrict who can peer.
• This article does not cover hardware VTEP table limits, which vary by switch model. Check your switch vendor's scale figures.
• Separation tests belong in your acceptance plan. Test from a tenant guest, including attempts to reach another tenant's addresses, after any change to route targets or exit nodes.

Sources

Opened 10 November 2026.

• RFC 7348: Virtual eXtensible Local Area Network
• RFC 7432: BGP MPLS-Based Ethernet VPN
• RFC 8365: A Network Virtualization Overlay Solution Using EVPN
• RFC 9135: Integrated Routing and Bridging in EVPN
• RFC 9136: IP Prefix Advertisement in EVPN
• Linux kernel: VXLAN
• ip-link(8)
• bridge(8)
• ping(8)
• FRRouting: EVPN
• FRRouting: BGP
• Proxmox VE: Software-Defined Network
• Proxmox VE: Firewall
• Linux kernel: Virtual Routing and Forwarding
• RFC 8200: Internet Protocol, Version 6
• pcap-filter(7)

This article was originally published on 10 November 2026.