AI token prices plunge, but total costs soar in 'inference paradox'

Major AI labs such as OpenAI, Anthropic, Google, and Meta have aggressively slashed prices for high-volume, commodity-tier AI usage, with token costs plummeting by approximately 95% over three years—from $60 per million tokens in early 2023 to as low as $0.10 to $3.30 by mid-2026. This dramatic price compression has fueled explosive growth in demand, exemplified by OpenAI’s weekly token usage soaring from 2.25 trillion in mid-2025 to over 45 trillion by mid-2026, and Google’s monthly tokens surging 330-fold between 2024 and 2026, illustrating a robust demand elasticity where lower prices lead to disproportionately higher consumption.

This aggressive price competition is part of a deliberate dual-track pricing strategy where commodity-tier AI models are commoditized through steep price cuts to capture volume and maximize GPU utilization, while frontier models with advanced capabilities maintain premium pricing and restricted access to protect margins. For instance, OpenAI’s flagship GPT-4 and GPT-5 models retain high price points ($30 and $5 per million tokens, respectively), and Anthropic gates its Mythos 5.1 model behind specialized programs, reflecting a balance between scaling usage and preserving valuation as both companies prepare for public offerings.

The plunge in commodity-tier prices has not only driven higher query volumes but also encouraged users to engage in more complex, token-dense tasks, with median prompt lengths quadrupling and programming workloads expanding from 11% to over 50% of usage between 2023 and 2025. However, corporate customers show caution in upgrading to pricier frontier models, favoring cost-effective alternatives that deliver sufficient performance, as seen with Anthropic’s Claude Opus 5 gaining traction over the more expensive Claude Fable 5, highlighting a pragmatic approach to AI adoption amid economic uncertainty.

The relentless price cuts for commodity-tier AI APIs also exert pressure on app-layer companies, prompting a phenomenon known as 'sherlocking,' where AI model providers integrate popular app features directly into their models to capture more value. This dynamic underscores the strategic imperative for labs to dominate high-volume usage by aggressively lowering prices to saturate GPU capacity, thereby consolidating control over both infrastructure and end-user applications in an increasingly competitive market.

Despite dramatic reductions in per-token prices—dropping by as much as 60-70% annually—total AI inference costs per agentic workflow are projected to increase more than fivefold by 2028. This counterintuitive trend, termed the Inference Paradox by Gartner's Will Sommer, arises because advanced AI applications increasingly rely on complex, multistep reasoning and negotiation tasks that consume exponentially more tokens than simple chatbots. For example, routing a task to an agentic reasoning model can inflate inference costs by at least five times, underscoring how growing workflow sophistication drives up total expenses even as unit costs fall.

The Inference Paradox reflects a broader dynamic akin to Jevons Paradox, where improvements in AI unit economics paradoxically fuel greater token consumption and more elaborate workflows, ultimately escalating total costs faster than price reductions can offset. Gartner and Goldman Sachs research highlight that token usage by AI agents is expected to surge 24-fold between 2026 and 2030, reaching an astounding 120 quadrillion tokens per month. This surge is propelled by enterprises deploying more powerful models and intricate multimodel ecosystems, which require extensive compute resources and complex orchestration to manage escalating inference demands efficiently.

Underlying these rising costs are structural factors that complicate cost management: significant asymmetries between input and output token pricing, hidden token consumption from internal reasoning processes, and tokenmaxxing behaviors that inflate bills beyond headline rates. For instance, Claude Sonnet 4.6 charges five times more for output tokens than input, while GPT-5.4 applies a sixfold multiplier, and reasoning models generate numerous unseen 'thinking tokens' that count toward usage. Furthermore, enterprises like Uber have exhausted annual token budgets in mere months, illustrating how subscription fees often mask the true scale of token consumption and inference expenses.

The economics of AI inference are further strained by the fixed costs of GPU infrastructure and the autocatalytic nature of AI scaling, where AI systems recursively generate more AI workloads. Providers face a delicate balancing act: overprovisioning GPUs leads to wasted resources, while underprovisioning risks throttling customer usage. As one expert notes, "You have to pay for the hourly GPU use regardless," highlighting the inflexibility of infrastructure costs. Meanwhile, massive capital investments—such as $3 billion clusters—enable rapid scaling beyond traditional engineering constraints, fueling exponential increases in inference demand and costs that defy conventional limits like the 'mythical man month.'

Despite the steep deflation in commodity-tier AI inference costs, profit and value capture have increasingly concentrated at the frontier AI capabilities and infrastructure layers that resist price erosion. Companies like OpenAI and Anthropic exemplify this trend, with OpenAI’s revenue soaring from $3.7 billion in 2024 to over $40 billion in 2026, and Anthropic’s annualized revenue reaching $65 billion alongside margin expansion from 38% to 70%. This underscores a structural shift where undifferentiated, lower-tier AI offerings face margin compression, while cutting-edge models and the underlying infrastructure—such as data centers and specialized hardware—remain the primary profit engines.

Frontier AI models maintain premium pricing and robust enterprise demand even as lower-tier models undergo aggressive price cuts and commoditization. For instance, flagship pricing dropped from $30 to $1.25 but rebounded to around $5 with GPT-5.5, reflecting sustained willingness among enterprises to pay for top-tier intelligence. This dual-track pricing strategy enables labs like Anthropic to grow rapidly and achieve profitability despite subsidizing cheaper plans, highlighting that the discount never truly applies to frontier capabilities, which continue to command a pricing premium due to their superior performance and brand strength.

Infrastructure providers such as Nvidia have become pivotal beneficiaries of AI’s deflationary pricing environment by supplying high-margin hardware critical to sustaining frontier AI development. Nvidia’s disclosure of over $1 trillion in orders at a 74.9% gross margin illustrates how value capture has shifted toward scarce, non-deflatable resources like specialized chips and power capacity, which have even seen price increases in some markets. This dynamic reinforces the notion that while AI token costs plummet, the physical and energy infrastructure underpinning frontier models remains a lucrative bottleneck.

Looking ahead, AI companies are refining cost management and pricing strategies to emulate SaaS-like economics by leveraging scale, amortizing content costs, and innovating flexible customer commitments. Executives like Rahul envision a future dominated by one or a few premium model providers wielding significant pricing power, supported by sub-linear scaling of embedded support and advisory services. This strategic focus on infrastructure efficiency and bespoke, high-value AI offerings is essential to sustaining profitability amid spiraling costs and the commoditization of token consumption.