Two colocation sites from empty cages: 40 racks, GPU, storage and compute, built, burned in and handed over

The company ran a growing share of its work in the public cloud: steady databases, a training cluster and trading-adjacent services that needed to sit near their counterparties. The cloud bill for work that never changed shape had passed the cost of owning it, and a commitment was about to end. The plan was two owned footprints in different metros, each built from an empty cage and running the same design, and without hiring a facilities team, so our team took the build from the first site visit to handover. The site survey came first. We visited five candidate facilities and scored them on what the sales sheet leaves out: the power each cabinet can really draw, whether the electrical and cooling paths are redundant all the way to the cage, which carriers are in the building, what cross-connects cost and how long they take, who may enter and when, and how the building fares in a flood or a hurricane. Two facilities won, and we read each contract for expansion rights, because a cage that cannot grow becomes a second project.

The design started with a rack elevation for each type. Each site has 20 racks in cold-aisle containment: 7 GPU racks, 4 compute racks, 4 storage racks, 2 network racks and 3 racks for operations and growth, for roughly 300 kW of IT load. Typical draw guided the layout: storage 5 to 8 kW per rack, general compute 10 to 15 kW, and GPU racks 20 to 40 kW. An 8‑GPU training node draws around 10 kW, so a GPU rack holds three and needs rear-door heat exchangers, which we agreed with each facility before ordering. Heavy servers go low, switches sit in fixed positions, and a blanking panel fills every empty unit so air cannot recirculate. Inlet air stays inside the ASHRAE recommended band of 18 to 27 degrees C.

Power was designed in pairs. Every server's two power supplies go to an A and a B feed from separate paths, through metered three-phase PDUs, and each feed must carry the whole rack if the other fails. A 32 A, 400 V three-phase feed is 22.2 kVA, and loaded to 80% it gives 17.7 kW usable per A/B pair, which suits compute and storage racks. GPU racks use 63 A feeds. The network is a spine-leaf fabric: 400G spine links, 100G uplinks to storage and GPU servers, 25G to compute, BGP with EVPN-VXLAN, and two 100G waves between the sites for replication. Short runs use direct-attach copper, rack-to-rack runs use active optical cables or 100GBASE-SR4 over OM4, and room and inter-site uplinks use OS2 single-mode.

Cabling and servers were built in the same weeks. Each site followed a 16‑week plan: survey and design in the first four weeks, procurement as the longest phase because switch, GPU and optic lead times set the pace, racking and cabling overlapping, then burn-in, acceptance and a scheduled cutover. We fixed MPO-12 trunks between the network racks and each row, ended them in patch panels, checked polarity with a tester before any switch was connected, and routed Cat6A for management in its own managers. Every cable and port carries a label to the TIA-606 scheme, so about 2,000 cables per site can be traced from either end. Our team assembled 200 servers across the two sites, pinned firmware to one baseline per model, configured BMCs for Redfish and IPMI, and burned each one in for 48 to 72 hours with memory pattern tests, CPU and GPU stress, NVMe write and verify, and line-rate network tests. Burn-in found 21 faulty parts out of thousands: 7 DIMMs, 1 CPU, 4 GPUs and 9 NVMe drives, all replaced under warranty before any workload arrived.

Acceptance was a written checklist, not a feeling. We pulled each A feed and each B feed in turn and confirmed that nothing dropped, failed a link on every spine, cut the primary uplink to prove the out-of-band path and watched inlet temperatures during a deliberate reduction in cooling. Only then did workloads arrive, in waves, each behind a method of procedure with a rollback. The DCIM records every device, rack unit, port, cable and outlet, and a change is not closed until the record matches the room. Handover decides whether the build lasts: the company's own team received the elevations, power budgets, cable schedules, runbooks and the baseline firmware set, and a remote-hands contract covers drive swaps and re-seating when nobody is on site. Decommissioned gear leaves under a media-sanitization procedure based on NIST SP 800-88. This case is illustrative: the power, rack, server and failure figures are modeled for a build of this kind. Our guide to optics from 10 to 400 Gb/s and our article on separating networks across datacenters explain the network choices.