What is the biggest challenge AI companies face in sustaining margins as compute costs and competition increase, and why?
Most answers highlight rising compute and hardware costs as the main threat to AI margins, though some focus on capacity constraints, price competition, and the challenge of recouping investments.
10 answers
Answers


Gary D. Cohn
Lead Director · Apollo Global Management, Inc.
Finding ROI on AI spend
Gary D. Cohn says the challenge is finding return on AI spend, since companies are now forced to evaluate whether the value they get back from tokens matches what they paid.
The answer starts at 6:54.
Squawk on the Street · Listen on Apple Podcasts

Harry Stebbings
Founder · 20vc
Multiple Stack Layers
Harry Stebbings says the challenge is that margins are split across many stack layers, since a dollar paid at the top flows through applications, models, infrastructure, cloud and Nvidia, each taking 20-30% gross margins.
20VC with Harry Stebbings · Watch on YouTube

Sam Altman
Chief Executive Officer · OpenAI
Compute Shortages
Sam Altman says the "invisible hand" is failing to resolve compute shortages because memory makers, scarred by past boom-bust cycles, stay conservative on capex despite large profits, and he says the solution requires new entrants and supply-chain innovation.
TMT Breakout · TMT Breakout · Read
Raoul Pal
Co-Founder and CEO · Real Vision
Capacity Constraints
Raoul Pal says Anthropic has been capacity constrained since June, citing its IPO prospectus that it struggles to get the capacity it needs and calls this the biggest issue for its business.
The answer starts at 11:13.
Raoul Pal: The Journey Man · Listen on Apple Podcasts

Naveen Rao
[un]CEO · Unconventional Ai
Squeezing Margin from Hardware
Naveen Rao says inference providers must squeeze more margin out of hardware as the price per token drops, and he expects that layer to yield lower margins.
20VC with Harry Stebbings · Watch on YouTube
James Altucher
Founder
Data Center Interconnects
James Altucher says the main bottleneck for AI is in the data center, specifically the interconnects that connect GPUs, and suggests looking at companies making the hardware between two GPUs.
The answer starts at 7:55.
DisrupTV · Listen on Apple Podcasts

Jason Calacanis
Bestie · All-In Podcast
Compute Costs
Jason Calacanis says serving customers has real costs amid a compute crunch, and that every CPU or GPU cycle consumed carries a real cost, which he says is probably why companies are monetizing faster than in the web 2.0 era.
This Week in Startups · Watch on YouTube

Lin Qiao
CEO and cofounder · Fireworks Ai
Cost of Serving Exceeds Income
Lin Qiao says AI companies can have customers who love the product while the cost of serving them exceeds their income, so scaling literally scales into bankruptcy.
New Economies · New Economies · Read

Anthony Pompliano
Founder & CEO · Professional Capital Management
Price Cuts
Anthony Pompliano says the decline in AI spending is driven by model wars, citing price cuts and a shift toward cheaper standard and lite models over frontier ones.
The Pomp Letter · The Pomp Letter · Read

Eric Vishria
General Partner · Benchmark
Controlling Costs
Eric Vishria says cost matters a lot more for databases that must scale from zero to infinity with fast iteration, and that ultimately cost becomes the arbiter of who wins and loses.
Invest Like The Best · Watch on YouTube
More Generative AI questions
View all →- What design choice improves AI agent privacy?5 answers
- What architectural choices enable reliable agentic AI systems?8 answers
- What responsibilities should AI creators have for their models?12 answers
- How does your team create feedback loops for AI models?12 answers
- What architecture optimizes compute for AI workloads?13 answers


















