One company in fourteen countries, with separate networks by region and business unit and zero downtime

The company had grown by acquisition, and its network showed it. Plants, warehouses and offices in 14 countries sat on one large, loosely connected routed network. Address ranges overlapped between acquired companies and were patched with NAT, internet breakouts differed in every country, and an infected machine in one plant could, in principle, reach a database in another region. The goal was separation: a network for each region and each business unit, connected only where the business needs a connection, and every crossing visible.

The architecture starts from a clean idea: separate routing tables. Each region and business unit gets its own VRF, so address space can overlap without conflict and traffic moves between VRFs only through a firewall policy someone has approved. Inside a site, access switches carry IEEE 802.1Q VLANs; where a carrier delivers several of the company's VLANs over a single tagged circuit, QinQ (IEEE 802.1ad) stacks a service tag on top of the customer tags so they survive the handoff intact. Datacenters use a spine-leaf fabric: SFP+ 10G for older servers and out-of-band, SFP28 25G for newer hosts, and QSFP28 100 Gb/s leaf-to-spine uplinks with equal-cost multipath, so capacity grows by adding leaves and a link failure costs a fraction of capacity rather than an outage.

At the edge, BIRD 2 runs BGP with every transit and peering partner. Incoming routes are checked against RPKI route-origin validation (RFC 6811) and invalid announcements are rejected; communities tag where routes were learned and let the team steer traffic, prefer a carrier or signal a partner to blackhole a target; BFD detects a dead link in a fraction of a second instead of waiting for BGP hold timers. Between company sites, a WireGuard mesh (Noise IK handshake, ChaCha20‑Poly1305, over UDP) provides simple, fast encrypted paths, while IPsec with IKEv2 and AES-GCM remains the language spoken to partners and suppliers who require it. The most important design choice was to stretch Layer 2 only where an application truly needs it: VXLAN with an EVPN control plane (RFC 7432) carries a VLAN across two datacenters or across a carrier network as a VXLAN overlay on UDP 4789, with the underlay run as routed Layer 3 and jumbo frames to absorb the overhead. Servers keep their addresses and MAC mobility handles a moving virtual machine.

The migration was planned like a flight, not a weekend. A lab replica of the target design was built first and every cutover step was rehearsed against it. Sites moved one at a time, each in two stages: bring the new fabric up in parallel with the old one, then move VLANs and routes across with BGP preference changes, so that going back is a configuration change rather than a rebuild. Stretched VLANs let applications move between datacenters while their IP addresses stayed the same, and no production line, warehouse scanner or ordering system was taken offline. Every window ended with a verification checklist before the old path was retired.

Before trusting the edge, we tested it. In a controlled flood test with the company's carriers informed in advance, synthetic attack traffic in the order of 180 Gb/s and tens of millions of packets per second was aimed at a protected prefix. XDP programs at the edge dropped spoofed and malformed packets before the kernel network stack saw them, FlowSpec rules pushed rate limits to the carrier-facing routers, and when the attack exceeded what the local links could absorb, the prefix was diverted to an upstream scrubbing service over BGP, with RTBH as the last resort that sacrifices one address to save the rest. Legitimate sessions on neighbouring prefixes were unaffected. The result: a network where a problem in one plant stays in one plant, a datacenter move is a routing change rather than a project, and the company can show its auditors exactly which segments talk to which.