Datacenter fabric & optics
A modern datacenter network is a leaf-spine fabric: every leaf switch connects to every spine, so any two servers are the same number of hops apart. Layer 2 and layer 3 segments are carried over that routed fabric with VXLAN (RFC 7348, UDP port 4789, 24‑bit VNI) and a BGP EVPN control plane (RFC 7432, RFC 9136). Our team designs and builds these fabrics, and the optics and cabling under them.
What we do
- Leaf-spine fabric design with BGP as the routing protocol
- VXLAN-EVPN to stretch VLANs and tenants across racks and datacenters
- MLAG and dual-homed server uplinks
- Optics and cabling plans from SFP+ 10G to QSFP-DD 400G
- DAC and AOC for in-rack and row runs, fiber for longer distance
- Datacenter interconnect over dark fiber or coherent DWDM
- Cumulus Linux, SONiC and EOS templated configuration
- Capacity planning, oversubscription review and migration plans
- Transceiver compatibility and coding checks before purchase
We work across the speed ladder. SFP+ carries 10 Gb/s, SFP28 25 Gb/s, QSFP28 100 Gb/s and QSFP-DD 400 Gb/s on eight lanes, with DAC and AOC cables for short runs and fiber optics for longer ones. Between datacenters we plan DCI over dark fiber, wavelength services or coherent DWDM, and MLAG pairs where servers need dual-homed uplinks.
The fabric stays boring on purpose: BGP as the only routing protocol as described in RFC 7938, templated configuration, automated validation and a rollback plan for each change.
Leading platforms we work with
- Arista Networks
- NVIDIA Spectrum
- Broadcom Tomahawk & Trident
- Cisco
- Juniper Networks
- Dell PowerSwitch
- Extreme Networks
- Nokia SR Linux
Platforms and practices
- Spine-leaf design
- VXLAN-EVPN
- MLAG
- SFP+ / SFP28
- QSFP28 / QSFP-DD
- DAC & AOC cabling
- Coherent DWDM data center interconnect
- Cumulus Linux & SONiC
- Fabric automation & validation
- Transceiver coding and compatibility
References
- Arista EOS (vendor page)
- NVIDIA Ethernet switching: Spectrum, Spectrum-X, Cumulus Linux and SONiC (vendor page)
- Broadcom StrataXGS BCM56990 series (vendor page)
- RFC 7348: VXLAN
- RFC 7432: BGP MPLS-Based Ethernet VPN
- RFC 9136: IP Prefix Advertisement in EVPN
- RFC 7938: Use of BGP for Routing in Large-Scale Data Centers
- Small Form-factor Pluggable: SFP, SFP+, SFP28, QSFP28, QSFP-DD, OSFP (reference article)
- OIF 400ZR Implementation Agreement OIF-400ZR-01.0 (PDF: DP-16QAM, 48 x 100 GHz DWDM channels, no restriction on pluggable form factor incl. QSFP-DD and OSFP)
- OIF implementation agreements list (400ZR versions 01.0, 02.0, 03.0)
Sources last checked .
Product names are trademarks of their owners and are listed to describe our experience; they do not imply endorsement or partnership.
A fabric with no single point of failure
Every leaf switch has a 100G QSFP28 uplink to every spine, servers attach at 25G or 10G through LACP pairs, and ECMP spreads flows across all spines. If a spine fails, the flows simply rehash onto the rest.
The right module for every reach
From 10G SFP+ to 400G QSFP-DD, the module is chosen by distance and fibre: DAC or AOC inside the rack, short-reach optics on OM4 multimode in the hall, long-reach on OS2 single-mode between buildings and coherent 400ZR between datacenters. One 100G port can also break out into four 25G.
Per-rack bandwidth, 10G to 100G
Aggregate uplink capacity from one rack to the spine, with four uplinks per rack. 10G SFP+ (10GBASE-SR, IEEE 802.3ae) gives 40 Gb/s, 25G SFP28 (25GBASE-SR, 802.3by) gives 100 Gb/s, and 100G QSFP28 (100GBASE-SR4, 802.3bm) gives 400 Gb/s. Illustrative.
How we use Datacenter fabric & optics for clients
Managed IT Services
We run your IT day to day so it stays secure: help desk, devices, Microsoft 365, Exchange, Active Directory, backups, networks and monitoring.
Learn moreCloud Management
Cloud cost review and management, without compromising on quality or security.
Learn moreCybersecurity
We find out whether everything you run is safe from attackers, then we fix it.
Learn moreFrequently asked questions
Direct answers to the questions we hear most. Something else on your mind?
Send us a messageWhat is a spine-leaf fabric and why use it?
Every leaf switch connects to every spine switch, so any two servers on different leaves are the same number of hops apart and traffic can use all paths with ECMP. You scale by adding leaves for ports or spines for bandwidth, without redesign. It replaces the older three-tier tree that was limited by oversubscribed uplinks and spanning tree.
Why VXLAN with EVPN instead of stretched VLANs?
VXLAN (RFC 7348) carries Layer 2 segments over a routed Layer 3 network, using a 24-bit VNI, so there are no spanning-tree loops across the fabric. EVPN (RFC 7432, RFC 9136) is the BGP control plane that distributes MAC and IP reachability instead of flooding. The result is VLANs that can span racks or datacenters while the underlay stays routed and stable.
How do we choose between DAC, AOC and fiber?
Direct-attach copper is cheapest for very short runs inside a rack, typically a few meters. Active optical cables suit row runs, and structured fiber with separate optics is right for longer distances and for patching that may change. We pick by distance, power and cost, and check that the switches accept the optics chosen.
Can we upgrade from 10G to 25G or 100G without replacing everything?
Often, if planned. SFP28 ports also accept SFP+ modules at 10G, and QSFP28 ports accept QSFP+ at 40G or can break out to four 25G links. Fiber type, switch support and server NICs decide it, so we check the existing cabling and optics before quoting. Moving to 400G generally means new optics and often new switches.
What is MLAG and when is it the right choice?
MLAG presents two switches to a server or another switch as one link aggregate, so either switch can fail without dropping connectivity. It suits server uplinks and small designs. In larger fabrics, EVPN multihoming provides the same function without a proprietary peer link between two vendor switches.
How do you connect two datacenters over EVPN?
Over dark fiber, a carrier circuit or coherent DWDM, we extend the fabric with VXLAN-EVPN, so a VLAN can exist in both sites and a server can move without changing its address. We design the interconnect for latency and failure: the link between sites becomes a shared fate, so we limit what stretches and keep routing local where possible.