AI data centers hit triple bottleneck: power, cooling, connectivity

By October 2026, the industry was no longer describing AI infrastructure limits as isolated problems but as a single design constraint spanning power, cooling, and connectivity. Semiconductor Engineering reported that the “1-megawatt rack” push had become a coordinated standards effort among hyperscalers, explicitly tying rack-scale design to simultaneous constraints across power delivery, cooling, and connectivity, while separate September analysis showed why: rack power requirements are moving from roughly 5 to 10 kilowatts to 100, 250 kilowatts, blowing past the assumptions embedded in earlier data center designs.

What made the constraint binding was that operators were now saying infrastructure readiness determined whether AI capacity could be used at all. In SiliconANGLE theCUBE, Supermicro described a GB300 example in which “A single rack goes in excess of $4 million” and warned that “if the liquid cooling is not ready, if the power delivery is not ready, if the cooling is not there, so then that entire infrastructure is sitting there unused,” a blunt illustration of why October’s OCP-era collaborations mattered: they surfaced that AI growth was being capped by three interlocked physical bottlenecks at once.

The first hard limit is no longer chip supply but the electrical system feeding AI clusters. On Catalyst, Shayle Kann said AI load “is very volatile… It has milliseconds, seconds and minutes variation that go from 10% of a load to 100% of the load,” so a “100 megawatt data center” can swing from 10 to 100 megawatts, stressing conversion equipment and creating harmonics; because “copper gets very large if you keep it at low voltage” and “you have to convert them all,” operators are shifting toward an 800 volt DC bus that “can natively provide 800 volt DC going to the grid.”

Those power-delivery changes are being forced by the scale of AI systems now entering production. Quantum Zeitgeist reported that “NVIDIA has achieved the largest-scale training demonstrated in MLPerf Training 6.0, utilizing systems with 8,192 GPUs,” while the GB300 NVL72 delivered up to 1.6 times faster training than GB200 NVL72 “driven by higher compute density, expanded memory, and increased power capacity”; NVIDIA also “leveraged its fifth-generation NVLink Switches, connecting 72 GPUs within each rack-scale system,” and the same architecture is already deployed by hyperscalers, with “Microsoft Azure scaled Llama 3.1 405B training to 8,192 GPUs” and “CoreWeave achieved the fastest time for DeepSeek-V3 671B… at 8,192-GPU scale.”

The thermal break with older data center design is now explicit: rack power and compute density have risen beyond what fans and room-level airflow can practically remove. One August analysis put it bluntly, saying “rack power requirements are moving from roughly 5 to 10 kilowatts to 100 to 50 kilowatts” while “Compute density is climbing something like 70x,” followed by the conclusion that “Cooling is moving from air to liquid as a requirement,” a shift that increasingly points toward direct-to-chip loops, immersion-style systems, and other high-density thermal architectures rather than conventional air-cooled racks.

Meta’s October disclosures show what that shift looks like in practice as Prometheus turns a former web-serving campus into an AI facility: Meta reports the New Albany, Ohio site “Two years ago... was a conventional data center serving web traffic,” but is now “being scaled toward roughly 1 million GPUs.” In that buildout, the company cites “~120 kW total system power across the two-rack pair,” says its GB300 NVL72 platform Clemente “deploys with either Meta’s modular air-assisted liquid cooling or facility liquid cooling,” and describes San Miguel as “fully liquid-cooled compute trays at more than 170kW per rack,” explicitly pairing higher-density NVL72 rack designs with liquid cooling options and indicating air cooling alone is no longer sufficient for gigawatt- and megawatt-class operation as cooling evolves with density.

GPU scaling is now constrained not just by compute but by the fabric connecting accelerators, as Vik's Newsletter argues that interconnects bottleneck efforts to make AI systems bigger and that “Nature limits copper’s reach as data rates get faster.” At the bandwidth targets now in view, that becomes a hard architectural limit: “As we move into 1.6 Tbps and 3.2 Tbps interconnect speeds, optics increasingly looks like the only option,” a shift made tangible by Lumentum saying demand for lasers “far exceeds even as we continue to add laser capacity,” with demand “far outstripping our ability to supply.” The article ties that directly to scaling optical interconnects, noting “The 200G version… is starting to ramp for 1.6”.

The response is no longer theoretical: NVIDIA’s GB300 NVL72 uses a high-bandwidth, non-blocking in-rack fabric, with fifth-generation NVLink delivering 1.8 TB/s per GPU and 130 TB/s of all-to-all bandwidth across 72 GPUs so added accelerators still raise throughput. But SemiAnalysis shows why copper-era scale-out gets messy beyond that domain, explaining that 800G CX-8 NICs are “multiplanar,” where “each NIC splits its bandwidth across independent switch fabrics, called planes,” and “Every NIC lane must land on the right plane,” while Light Matter pitches photonics that send 16 colors over one fiber and scale to 24 lanes at 114 terabits per second.

October 2026 was the moment the industry stopped talking about AI infrastructure limits as separate engineering annoyances and started presenting them as one coordinated design problem. At the 2026 Open Compute Project Global Summit from October 12–15, xFusion said it was showing “latest AI computing infrastructure — spanning liquid-cooled rack-scale systems” and next-generation high-voltage DC power, then made the synthesis explicit: the goal was “to solve three infrastructure challenges at once — cooling, power, and interconnect,” turning the summit itself into a public marker of convergence.

The substance of that convergence was unusually concrete. Under “Power,” xFusion said it was “advancing 800V high-voltage DC (HVDC) architecture,” including an “800V HVDC Sidecar” for 300 kW to 1.2 MW configurations; under “Cooling,” it argued that “100% direct liquid cooling (DLC) is becoming the baseline for AI infrastructure,” while Arthur Wang’s OCP keynote, “Next-Generation Power & Cooling Solutions,” reinforced the message in public. PR Newswire’s summit materials also tied interconnect to optical links and CPO, completing the three-part response.

Independent analysis after the event showed that this was not just one vendor’s framing. Morgan Stanley, after participating in the 2026 OCP Global Summit in San Jose, wrote that AI data center expansion “remains constrained by power, network, and infrastructure efficiency” even as hyperscaler demand for “power, chips, network, and optical communication equipment” stayed strong; it also highlighted the rising importance of optical interconnects and technology routes such as NPO and CPO. That made the October summit look less like scattered product marketing and more like a visible industry alignment.