AI token bills soar: routing tools become cost-saving lifeline

By mid-2026, enterprises were learning that AI’s real budget problem was not training but inference, as usage scaled faster than pricing relief. FinOps X 2026 opened with a chart showing “token consumption climbing from single-digit trillions to tens of trillions in under a year,” while TechTarget noted that “it’s the inference costs that typically put the most long-term pressure on AI operational budgets”; even a modest workload at $5 per million tokens and 20 million tokens a day already runs $3,000 a month before output tokens and other costs, making token consumption the cost center executives could no longer ignore. The budgeting challenge was sharpened by wide tiered pricing bands—budget models at $0.14 to $1 per million tokens, midrange models at $2 to $15, and high-end models at $20 to $75—making model selection and routing a direct financial control.

By late summer and into October, the pressure had become structural: Accenture warned that “while token prices are falling, usage is rising,” citing Goldman Sachs projections that token use could grow 24x by 2030 even as cost per token falls 60–70% annually. That mismatch was reinforced when AMD CEO Lisa Su said that in 2026 “for the first time, the world is using more AI compute to run models than to train them,” while Diginomica reported enterprises actively governing against token-cost “explosion” and BigDATAwire pointed to escalating costs and unclear value as a reason many agentic AI projects were being canceled. In that environment, the spread between budget, midrange, and high-end token pricing made tiering and routing increasingly urgent as enterprises tried to keep growing agent usage within budget.

Falling token prices no longer protect enterprise margins because agentic workflows multiply token use faster than unit costs decline. Forbes notes Goldman Sachs forecasts token consumption will multiply 24 times between 2026 and 2030, reaching 120 quadrillion tokens per month, and gives the core arithmetic: if token draw for a task rises 20-fold while unit price falls 75%, total model charges still rise fivefold; that is why The AI Frontier warned that “throwing your largest model at every task” burns budget, after one RunLLM investigation mistakenly cost $63.

The pressure intensifies because reasoning and multi-agent execution add hidden and repeated token overhead, making model choice inseparable from workflow design. Forbes notes Muse Spark 1.1 bills internal thinking tokens at the output rate and GPT-5.6’s ultra mode coordinates four agents in parallel, while Authority Hacker Podcast warns that if “you spawn a 100 sub agents,” you pay to write all those prompts and reread the same file “100 times”; keeping the orchestrator lean instead enables “very long running tasks” because, rather than becoming saturated and compacting, “it doesn't read all that stuff” and “the sub agent goes and work for 20 minutes,” preserving quality while cheaper models handle the dirty work.

The fix is increasingly straightforward: stop treating the frontier model as the default and instead route each request to the cheapest model that still clears the quality bar. OnlyCFO’s Newsletter put the economics bluntly: the “smartest” AI model is 5x+ more expensive than the cheapest model, meaning companies can be paying up to 5x more than necessary for routine work; in its example, a company spending $100K a month on Claude could save nearly 50%—$576K a year—by shifting appropriate tasks from Opus 4.8 to Sonnet and Haiku. That logic is showing up even at the individual-user level: one speaker said, “I spend maybe $2,000 or so per week on on tokens… [and] that is already influencing my behavior,” adding that they do not currently use fast mode for these models for that reason.

What makes that routing credible is that “cheaper” no longer automatically means unusable: Ravi on Product argued the premium model should be an escalation path, not a default, because mid-tier often means last generation’s frontier. The same piece pointed to Anthropic’s curve as the tell—“Take Sonnet 4.5… Two months later, Opus 4.5 moved the frontier again — but the comparison is the tell: at medium effort, Opus 4.5 matched Sonnet 4.5’s best SWE-bench Verified score using 76% fewer output tokens”—while The Digital Creator noted GPT-5.5 costs $5 per million input tokens and $30 per million output tokens, versus $0.094 per million for DeepSeek V4 on agentic tasks. In other words, rising token bills are already changing behavior before quality collapses, reinforcing the case for reserving premium models for escalation rather than default use.

This is no longer a theory test. ByteByteGo reported that Kilo “published figures from its own production traffic over the first quarter of 2026,” with a chart labeled “Auto-routing cuts cost by a third,” after sending all traffic through its gateway instead of letting users choose models manually. That result fits Kilo’s broader finding that “80 to 90% of requests do not need frontier models,” so a centralized routing layer can move routine work onto cheaper tiers at scale without making degraded output the default operating mode.

Enterprises are also layering spend controls and harness optimization on top of routing, and the savings are already large. BigGo Finance said model routers can “automatically schedule the most suitable AI model based on task complexity” and cut computing expense “by up to 97% without significantly sacrificing output quality”; McCarthy Building said Palantir’s Evolve drove quarterly token usage down 60%, and Palantir cited one switch from GPT-5.1 to GPT-5.4 Nano that cut cost 97%. At Coinbase, cache hit rate rose from 5% to 60% and spend was nearly halved while usage grew, while Plano’s model-affinity pinning avoids cache loss from model switching; VentureBeat likewise reported Writer’s harness cut token spend nearly 40%, with cost-per-successful-task down up to 61% and accuracy holding steady.

Competition is shifting into execution tooling because vendors now have to prove they can keep enterprises off the most expensive path by default. InfoQ reports Microsoft packaged a three-layer routing stack for agent workloads on AKS, where “the load-bearing number in the whole setup is RouteLLM's escalation threshold”; in testing, “the mf router achieved about 95% of GPT-4's MT-Bench quality” while “only sent around 26% of calls to GPT-4,” and Microsoft adds that “A simple round-robin load balancer makes this even worse,” such as when a 200-token completion sits behind a 100k-token prefill on a busy GPU while another pod is idle.

That same competitive logic is spreading beyond model choice into token-efficiency features, pricing segmentation, and spend controls, because buyers are rewarding vendors that reduce total cost per task. THE ELEC says providers now emphasize prompt caching and compression alongside API prices; it defines prompt caching as reusing “previously computed results, or the key-value (KV) cache,” and cites OpenAI saying it “can reduce input-token costs by as much as 90% and latency by up to 80%,” while Forbes-cited reporting notes Uber “exhausted its annual AI budget in April 2026,” and Venture Curator says “the market that actually matters is the middle tier,” with Tomasz Tunguz of Theory Ventures arguing the same as the article states “The mid tier already takes about 40% of AI spend and 30% of tokens,” while the frontier is “thin,” frontier token share among large accounts falling from 53% to 45%, and price cuts being matched within 90 minutes.