What is the biggest challenge AI companies face in sustaining margins as compute costs and competition increase, and why?

Most answers highlight rising compute and hardware costs as the main threat to AI margins, though some focus on capacity constraints, price competition, and the challenge of recouping investments.

10 answers

Answers

  • Gary D. Cohn

    Lead Director · Apollo Global Management, Inc.

    Finding ROI on AI spend

    Gary D. Cohn says the challenge is finding return on AI spend, since companies are now forced to evaluate whether the value they get back from tokens matches what they paid.

    The answer starts at 6:54.

    Squawk on the Street · Listen on Apple Podcasts
  • Harry Stebbings

    Founder · 20vc

    Multiple Stack Layers

    Harry Stebbings says the challenge is that margins are split across many stack layers, since a dollar paid at the top flows through applications, models, infrastructure, cloud and Nvidia, each taking 20-30% gross margins.

    20VC with Harry Stebbings · Watch on YouTube
  • Sam Altman

    Chief Executive Officer · OpenAI

    Compute Shortages

    Sam Altman says the "invisible hand" is failing to resolve compute shortages because memory makers, scarred by past boom-bust cycles, stay conservative on capex despite large profits, and he says the solution requires new entrants and supply-chain innovation.

    TMT Breakout · TMT Breakout · Read
  • Raoul Pal

    Co-Founder and CEO · Real Vision

    Capacity Constraints

    Raoul Pal says Anthropic has been capacity constrained since June, citing its IPO prospectus that it struggles to get the capacity it needs and calls this the biggest issue for its business.

    The answer starts at 11:13.

    Raoul Pal: The Journey Man · Listen on Apple Podcasts
  • Naveen Rao

    [un]CEO · Unconventional Ai

    Squeezing Margin from Hardware

    Naveen Rao says inference providers must squeeze more margin out of hardware as the price per token drops, and he expects that layer to yield lower margins.

    20VC with Harry Stebbings · Watch on YouTube
  • James Altucher

    Founder

    Data Center Interconnects

    James Altucher says the main bottleneck for AI is in the data center, specifically the interconnects that connect GPUs, and suggests looking at companies making the hardware between two GPUs.

    The answer starts at 7:55.

    DisrupTV · Listen on Apple Podcasts
  • Jason Calacanis

    Bestie · All-In Podcast

    Compute Costs

    Jason Calacanis says serving customers has real costs amid a compute crunch, and that every CPU or GPU cycle consumed carries a real cost, which he says is probably why companies are monetizing faster than in the web 2.0 era.

    This Week in Startups · Watch on YouTube
  • Lin Qiao

    CEO and cofounder · Fireworks Ai

    Cost of Serving Exceeds Income

    Lin Qiao says AI companies can have customers who love the product while the cost of serving them exceeds their income, so scaling literally scales into bankruptcy.

    New Economies · New Economies · Read
  • Anthony Pompliano

    Founder & CEO · Professional Capital Management

    Price Cuts

    Anthony Pompliano says the decline in AI spending is driven by model wars, citing price cuts and a shift toward cheaper standard and lite models over frontier ones.

    The Pomp Letter · The Pomp Letter · Read
  • Eric Vishria

    General Partner · Benchmark

    Controlling Costs

    Eric Vishria says cost matters a lot more for databases that must scale from zero to infinity with fast iteration, and that ultimately cost becomes the arbiter of who wins and loses.

    Invest Like The Best · Watch on YouTube

More Generative AI questions

View all →