What is one specific practice or choice that has been most important for scaling AI data center or compute infrastructure, and why?
The answers split between optimizing hardware, software, cooling, and operational strategies, with some emphasizing technical specialization and others focusing on efficiency, planning, or infrastructure reuse.
15 answers
Answers


Chris Lattner
EVP of Advanced AI Software and Platforms · Qualcomm
Unified software stack
Chris Lattner says the key practice is a unified software stack that spans different chips, because with no one-size-fits-all chip, software should let people avoid rewriting the same code for each chip.
The TWIML AI Podcast with Sam Charrington · Watch on YouTube

Sam Altman
Chief Executive Officer · OpenAI
Clear Goal & Plan
Sam Altman says a clear goal and a clear plan to get there, including how decisions will be made along the way, is very important, citing OpenAI's decision to bet on scaling deep learning as an example of that clarity.
The answer starts at 10:29.
Stanford Online: University Courses & Lectures · Listen on Apple Podcasts

Mark Papermaster
CTO and EVP · AMD
Full stack optimization
Mark Papermaster says full-stack optimization is required, arguing that you cannot achieve the latency and reflex response needed in physical AI without it.
Imagination in Action · Watch on YouTube

Naveen Rao
[un]CEO · Unconventional Ai
Smaller Data Types
Naveen Rao says most efficiency gains in recent chip generations have come from smaller data types, going from 32-bit to 16-bit to 8-bit to 4-bit, because you move and compute with fewer bits.
20VC with Harry Stebbings · Watch on YouTube
Rob Ferguson
VP Technology and Strategy · Fireworks Ai
Keep GPUs Busy
Rob Ferguson says the key is to keep GPUs busy, because idle time on GPUs drives up costs for everybody.
Mind the Product · Watch on YouTube

Eugene Cheah
CEO / Co-Founder · Featherless Ai
Dedicated Base Instances
Eugene Cheah says the biggest cost savings in the market come from deploying a dedicated instance when you know your base workload, since full use of it yields savings, though he notes it depends on traffic patterns.
AI Engineer · Watch on YouTubeIlya Sutskever
Cofounder and CEO
Faster Processors
Ilya Sutskever says you always want faster processors and more of them, and that faster processors are wanted "any day," though he frames the cost as justified only if what comes out of it outweighs the cost.
Eye on AI · Watch on YouTube

Shayle Kann
Managing Partner, Frontier · Energy Impact Partners
Rear-Door Heat Exchangers
Shayle Kann says his team used rear-door heat exchangers to bring cooling closer to the heat source, which saves fan energy by only moving heat near the compute, though he notes they stop working well beyond about 80kW.
The answer starts at 10:26.
Catalyst with Shayle Kann · Listen on Apple Podcasts

Amin Vahdat
Chief Technologist, AI Infrastructure and SVP · Google
Specialize Hardware
Amin Vahdat says the more you specialize hardware to a particular workload, the faster and more power-efficient it is, though less flexible, so the workload must be durable enough to justify it.
Sequoia Capital · Watch on YouTube

Gilad Shainer
Senior Vice President Networking · Nvidia
Scale-Out GPU Networking
Gilad Shainer says the industry invested in scale-out networking so all GPUs stay fully synchronized, avoiding jitter where one GPU's data-delivery hiccup makes the others wait.
SiliconANGLE theCUBE · Watch on YouTube
Andrew Feldman
Founder and CEO · Cerebras
Data Center Availability
Andrew Feldman says the whole industry is currently constrained by data center availability, which he says is holding everyone back because data centers move like real estate while the industry tries to move at the speed of software.
Imagination in Action · Watch on YouTube

Austin Lyons
Senior Analyst · Creative Strategies, Inc.
Fine-Tune Models
Austin Lyons says fine-tuning models is the practice, citing an insurance company example where humans correct agentic claims processing and that labeled data is kept internally to fine-tune a model so it gets things right in the future.
Chipstrat · Chipstrat · Read
Rodrigo Liang
Co-Founder & CEO · Sambanova
Reuse Infrastructure
Rodrigo Liang says reusing existing infrastructure where energy and space are already allocated lets state-of-the-art AI run at a much lower cost, making AI available to everybody.
SiliconANGLE theCUBE · Watch on YouTube
Samuel Rodriguez
Senior Product Manager - Industrial Solutions · Chatsworth Products
Adopt Liquid Cooling
Samuel Rodriguez says the biggest driver in thermal management will be the adoption of liquid cooling, because more devices will require it or perform better with it, forcing it into the data center.
The Data Center Frontier Show · The Data Center Frontier Show · Listen at 8:39

Matt Vincent
Editor-in-Chief/Head of Content (Event) · Data Center Frontier
Liquid cooling
Matt Vincent says the solution right now for higher-density chips is liquid cooling, though he notes there is currently room for both liquid and air cooling.
The Data Center Frontier Show · The Data Center Frontier Show · Listen at 7:12



