Evidence correction, 14 October 2026: earlier versions of this post attributed an agent token study to Microsoft Research and a token-waste figure to the wrong paper, misdescribed several Gartner, Epoch AI and vendor figures, cited cloud and FinOps market numbers we could not find, and contained arithmetic errors in the routing and 2030 examples. Those claims are removed or corrected below.
Gartner says inference gets 90% cheaper. That does not mean your bill shrinks.
In March 2026, Gartner published a prediction that by 2030, performing inference on a trillion-parameter LLM will cost providers over 90% less than it did in 2025. Gartner added that those falling provider costs will not be fully passed on to enterprise customers. Its May 2026 forecast put total worldwide AI spending at $2.59 trillion for the year, a 47% increase. In October 2025, Gartner forecast that inference-focused spending on AI-optimized IaaS would more than double, from $9.2 billion in 2025 to $20.6 billion in 2026, making inference 55% of all AI-optimized IaaS spending, a share it expects to pass 65% by 2029.
Source: Gartner, "Gartner Forecasts Worldwide AI Spending to Grow 47% in 2026," May 2026
A 90% cost drop sounds like it solves the AI cost problem. It does not. The reason is structural, and it is the reason routing becomes more valuable over time, not less.
Prices are falling faster than ever. Bills are rising faster than the prices fall.
Epoch AI measured inference price trends across six benchmarks and found that the price to reach a fixed level of performance fell between 9x and 900x per year, depending on the task and performance level, with a median of 50x per year. That median works out to prices halving roughly every two months. Using only data from January 2024 onward, the median rose to 200x per year.
Source: Epoch AI, "LLM inference prices have fallen rapidly but unequally across tasks," March 2025
At the same time, Goldman Sachs projects that token consumption will multiply 24 times between 2026 and 2030, reaching 120 quadrillion tokens per month. Spending is already rising: Menlo Ventures estimates enterprise generative AI spend rose from $11.5 billion in 2024 to $37 billion in 2025, even as per-token prices kept falling.
Source: Goldman Sachs, "AI Agents Forecast to Boost Tech Cash Flow as Usage Soars," May 2026
Source: Menlo Ventures, "2025: The State of Generative AI in the Enterprise," December 2025
The math is counterintuitive but consistent. If prices fall 10x and usage grows 24x, your total bill grows 2.4x (24 divided by 10). Gartner's 90% is a cost to providers, and Gartner says it will not be fully passed on to customers, so the customer-side price cut could be smaller and the bill could grow faster. Gartner itself expects overall inference costs to increase as token consumption rises faster than token costs fall. Cheaper tokens do not produce cheaper bills. They produce more usage, which produces larger bills at lower per-unit rates.
This is the same dynamic that played out in cloud computing. Cloud unit prices fell for years, yet Gartner forecast worldwide public cloud end-user spending of $675.4 billion for 2024, up 20.4% from $561 billion in 2023. Cheaper did not mean less spending. It meant more usage, more waste, and a larger optimization opportunity.
The tier spread is structural. It does not close as prices fall.
The second reason routing stays valuable is that the price gap between model tiers persists across price generations.
Here is a selection of list prices as of May 2026:
| Model | Input ($/M tokens) | Output ($/M tokens) |
|---|---|---|
| Claude Opus 4.7 | $5.00 | $25.00 |
| GPT-5.5 | $5.00 | $30.00 |
| Claude Sonnet 4.6 | $3.00 | $15.00 |
| Claude Haiku 4.5 | $1.00 | $5.00 |
| Gemini 2.5 Flash | $0.30 | $2.50 |
The output price spread between the cheapest model in this table (Gemini 2.5 Flash at $2.50/M) and the most expensive (GPT-5.5 at $30/M) is 12x. Between Haiku and Opus, it is 5x.
Compare this to two years ago. In early 2024, GPT-4 cost roughly $30/M input. GPT-3.5 Turbo cost $0.50/M. The spread was 60x.
The absolute prices fell dramatically. Frontier models went from $30 to $5 per million input tokens. Commodity models went from $0.50 to $0.30. The spread between tiers narrowed from about 60x to between 5x and 12x as the market matured, but it has not converged to 1x, and it will not.
The reason is economic. Frontier models cost more to train and run. They represent the latest research, the largest parameter counts, and the most expensive infrastructure. Commodity models use older architectures, smaller parameters, and cheaper hardware. As long as there is a quality gap between model sizes, there will be a price gap. And as long as there is a price gap, routing captures the spread on every request.
Epoch AI's data also shows that price declines are uneven. The rate of decline ranges from 9x to 900x per year depending on the task and performance milestone, a 100x difference. Reaching GPT-4-level performance on PhD-level science questions got 40x cheaper per year.
Because declines are uneven across tasks, the cheapest model that can handle a given request keeps changing. Routing captures that difference request by request.
Agentic workloads make the case even stronger over time.
Gartner's inference spending data shows the shift toward production inference. The $20.6 billion figure is spending on inference, not on training. Agentic workflows run on that inference, and Gartner says agentic models require 5x to 30x more tokens per task than a standard chatbot.
Researchers from the University of Michigan and Stanford documented this in April 2026, analyzing runs by eight frontier models on SWE-bench Verified. Coding agents consumed about 1,000x more tokens than code reasoning and code chat tasks, and runs on the same task varied by up to 30x. A separate study, AgentDiet, cut agents' input tokens by 39.9% to 59.7% by trimming waste from their trajectories while keeping the same task performance.
As agentic workloads grow (Vercel reports that 22.2% of its AI Gateway requests ended with a tool call in April 2026, up from 11.4% in October 2025), the total token volume subject to routing grows with them. Each agentic session has dozens of turns, each of which is independently routable. A 30-turn session where 20 turns are low complexity and route to Haiku instead of Opus saves 80% on those 20 turns at list prices (Haiku 4.5 at $1/$5 versus Opus at $5/$25). The savings multiply across hundreds of sessions per day.
IDC predicts that by 2028, 70% of top AI-driven enterprises will use multi-tool architectures to dynamically manage model routing. That prediction is not about cost cutting alone. It is about managing the combinatorial complexity of multi-model portfolios (F5 reports that organizations now coordinate an average of seven AI models in production) and the volume explosion from agentic adoption.
Source: IDC, "The future of AI is model routing," November 2025
The 2030 math: cheaper tokens, more tokens, same spread.
Here is an illustrative projection, not a forecast. It applies Gartner's 90% provider-cost decline to today's list prices, which overstates the price cut customers are likely to see, since Gartner says provider savings will not be fully passed on. It combines that with Goldman Sachs' consumption forecast.
2026 baseline (current):
- Average blended cost on AI.cc's platform: $6.07/M tokens in April 2026, down from $18.40 a year earlier (vendor data)
- Monthly token consumption: about 5 quadrillion tokens (implied by Goldman Sachs' 24x growth to 120 quadrillion)
- Tier spread: 5x to 12x
2030 projection (applying a 90% price cut + Goldman's 24x consumption growth):
- Frontier model cost: ~$0.50/M input, ~$2.50/M output (90% cheaper than today's $5/$25)
- Commodity model cost: ~$0.03/M input, ~$0.25/M output (90% cheaper than Gemini 2.5 Flash's $0.30/$2.50)
- Monthly token consumption: ~120 quadrillion tokens (24x growth)
- Tier spread: about 10x on output ($2.50 vs $0.25) and about 17x on input ($0.50 vs $0.03)
Without routing in 2030: 120 quadrillion tokens a month at frontier rates, assuming an even input and output mix (an average of $1.50 per million tokens). That is 120 billion million-token units times $1.50, or roughly $180B a month industry-wide.
With routing in 2030: 60% of tokens go to commodity models at the same even mix ($0.14 per million tokens). That is 40% of $180B ($72B) plus 72 billion million-token units times $0.14 (about $10B), or roughly $82B a month. These figures show the shape of the math, not a market forecast.
The per-request savings in 2030 will be smaller in absolute dollars (cents instead of dollars). But the volume will be 24x larger. The total addressable savings, the number of dollars that routing can redirect, grows every year.
Cloud cost optimization is the precedent.
This pattern has played out before. AWS launched EC2 in 2006 with a single instance type. Within a decade it had added spot and reserved pricing and many more instance types, and a cottage industry of cloud cost optimization vendors had grown around it. Gartner forecasts worldwide IT spending of $6.31 trillion in 2026, with IT services the largest segment.
The FinOps Foundation's State of FinOps 2026 survey drew 1,192 respondents representing more than $83 billion in annual cloud spend. None of this happened because cloud got expensive. It happened because cloud got cheap enough that everyone used it, and the aggregate spend became large enough that optimization mattered.
AI inference is on the same curve, compressed into a shorter timeline. The FinOps Foundation's 2026 report already shows 98% of FinOps teams managing AI spend, up from 31% two years ago. The tools, practices, and market for AI cost optimization are forming right now. Routing is the foundational layer.
Source: FinOps Foundation, "State of FinOps 2026 Report"
Three things to do before prices drop further.
1. Start routing now, not later. Every month you wait is a month of overspending at current prices. AI.cc, an API aggregator, reports that accounts using three-tier routing on its platform paid a median $2.31 per million tokens in 2026, against $18.40 for frontier-only workloads a year earlier. That gap mixes a year of price cuts with routing, and the data is the vendor's own, so measure your own baseline before projecting savings.
2. Build the routing muscle before 2028. IDC predicts 70% of top AI-driven enterprises will use multi-tool architectures to manage model routing by 2028. That gives you two years to instrument your traffic, understand your complexity distribution, and tune routing thresholds on your workload. Teams that start now will have years of calibration data. Teams that wait will start cold.
3. Treat inference cost optimization as a permanent discipline, not a one-time project. Just as cloud cost optimization became a permanent function, AI inference optimization is here to stay. The tools will evolve. The models will change. The principle, route each request to the cheapest model that can handle it, will not.
Where Nadir fits.
Nadir is built for this trajectory. Its decision API recommends model, effort, cache, context, and policy without calling a provider, so teams can shadow it before changing execution. The older 60% / 98% RouterBench result is a reference-assisted research ceiling, not deployed performance.
New models launch and pricing shifts all the time. Nadir is building an outcome-conditioned retraining pipeline for its verifier, but it is not yet automatic. When the routing policy changes, your application code does not.
The integration is two lines: change the base URL, set model="auto". Complete non-streaming responses report model and nadir_metadata.cost.total_cost_usd; a configured, priced benchmark adds nadir_metadata.benchmark_comparison.savings_usd.
Inference costs will keep falling. Usage will keep growing. The spread between model tiers will persist. Routing captures that spread request by request, today and in 2030.
Sources: [Gartner, "Gartner Predicts That by 2030, Performing Inference on an LLM With 1 Trillion Parameters Will Cost GenAI Providers Over 90% Less Than in 2025"](https://www.gartner.com/en/newsroom/press-releases/2026-03-25-gartner-predicts-that-by-2030-performing-inference-on-an-llm-with-1-trillion-parameters-will-cost-genai-providers-over-90-percent-less-than-in-2025) (March 2026). [Gartner, "Gartner Forecasts Worldwide AI Spending to Grow 47% in 2026"](https://www.gartner.com/en/newsroom/press-releases/2026-05-19-gartner-forecasts-worldwide-ai-spending-to-grow-47-percent-in-2026) (May 2026). [Gartner, "Gartner Says AI-Optimized IaaS Is Poised to Become the Next Growth Engine for AI Infrastructure"](https://www.gartner.com/en/newsroom/press-releases/2025-10-15-gartner-says-artificial-intelligence-optimized-iaas-is-poised-to-become-the-next-growth-engine-for-artificial-intelligence-infrastructure) (October 2025). [Gartner, "Gartner Forecasts Worldwide IT Spending to Grow 13.5% in 2026, Totaling $6.31 Trillion"](https://www.gartner.com/en/newsroom/press-releases/2026-04-22-gartner-forecasts-worldwide-it-spending-to-grow-13-point-5-percent-in-2026-totaling-6-point-31-trillion-dollars) (April 2026). [Gartner, "Gartner Forecasts Worldwide Public Cloud End-User Spending to Surpass $675 Billion in 2024"](https://www.gartner.com/en/newsroom/press-releases/2024-05-20-gartner-forecasts-worldwide-public-cloud-end-user-spending-to-surpass-675-billion-in-2024) (May 2024). [Epoch AI, "LLM inference prices have fallen rapidly but unequally across tasks"](https://epoch.ai/data-insights/llm-inference-price-trends) (March 2025). [Goldman Sachs, "AI Agents Forecast to Boost Tech Cash Flow as Usage Soars"](https://www.goldmansachs.com/insights/articles/ai-agents-forecast-to-boost-tech-cash-flow-as-usage-soars) (May 2026). [Menlo Ventures, "2025: The State of Generative AI in the Enterprise"](https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/) (December 2025). [Bai et al., "How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks," arXiv:2604.22750](https://arxiv.org/abs/2604.22750) (April 2026). [Xiao et al., "Improving the Efficiency of LLM Agent Systems through Trajectory Reduction," arXiv:2509.23586](https://arxiv.org/abs/2509.23586) (September 2025). [Vercel, "AI Gateway production index"](https://vercel.com/blog/ai-gateway-production-index) (May 2026). [IDC, "The future of AI is model routing"](https://idc.com/resource-center/blog/the-future-of-ai-is-model-routing) (November 2025). [F5, "AI has left the lab: F5 report reveals 78% of enterprises now run AI inference as a core operation"](https://www.f5.com/company/news/press-releases/enterprises-now-run-ai-inference-as-core-operation) (May 2026). [AI.cc press release, "AICC Report: Enterprise Token Costs Drop 67% Year-Over-Year as Multi-Model AI Adoption Hits Record High"](https://www.einpresswire.com/article/911544568/aicc-report-enterprise-token-costs-drop-67-year-over-year-as-multi-model-ai-adoption-hits-record-high) (May 2026). [FinOps Foundation, "State of FinOps 2026 Report"](https://data.finops.org/) (2026). Anthropic, OpenAI and Google model pricing as of May 2026.