Evidence correction, 14 October 2026: earlier versions of this post presented an API reseller's data about its own customers as an industry average, attributed survey figures to sources that do not contain them, and cited a Microsoft Research result and several accuracy and latency figures we could not find. Those claims are removed or attributed to their real sources below.
The 67% drop, and whose number it is
AI.cc, a Singapore API aggregator, reported in May 2026 that the average cost per million tokens across its customer accounts fell from $18.40 in Q1 2025 to $6.07 in Q1 2026, a 67% drop. It is a vendor's data about its own platform, published as a press release, not an industry survey. It is still a useful picture of where the savings come from.
Token prices did fall. Anthropic cut Opus pricing from $15/$75 to $5/$25 per million tokens. OpenAI launched cheaper tiers. Google's Flash-Lite models already priced input near $0.075 per million tokens.
AI.cc attributes about half of the drop to multi-model routing, customers matching each request to the cheapest model that could handle it, and the rest to cheaper open-weight models and its own volume discounts.
The multi-model shift
Across AI.cc's accounts, the average number of models per account rose from 2.1 to 4.7 in a year. Independent surveys point the same way: in a16z's June 2025 survey of about 100 CIOs, 37% used five or more models, up from 29% a year earlier.
The catalyst was a combination of model proliferation (more good options at every price point), cost pressure, and tooling maturity.
AI.cc customers fully using tiered routing paid a median $2.31 per million tokens in 2026, against $18.40 for frontier-only workloads a year earlier. That gap mixes a year of price cuts with routing, so it is not a same-period saving.
Why single-model deployments are expensive
The logic is straightforward. Much enterprise API traffic is not complex reasoning. The share of requests a budget model can handle depends on your traffic and the quality bar you set.
When every request goes to a frontier model such as Opus 4.7, at $5 input and $25 output per million tokens, you are paying frontier rates for tasks like formatting output, parsing errors, answering FAQs, and generating boilerplate code. Cheaper models can often handle these tasks, which you should confirm on your own traffic.
The FinOps Foundation noticed
The FinOps Foundation's 2026 State of FinOps report found that 98% of respondents now manage AI spend, up from 31% two years earlier, and named FinOps for AI the top forward-looking priority.
This is driving a shift in how teams measure AI economics. The old metric, cost per token, is being replaced by cost per successful interaction. A cheap model that fails and retries three times can cost more than an expensive model that succeeds on the first try. Token cost alone does not capture this.
The teams with the best economics track three things:
- Cost per completed task, not cost per token
- Success rate per model per task type, to calibrate routing thresholds
- Retry cost, which compounds because each retry carries the full accumulated context
This is why static rules like "send everything under 100 tokens to Haiku" fall short. The routing decision depends on task complexity, not input length.
What a routing layer actually does
A routing layer sits between your application and the LLM providers. For each request, it:
- Classifies the request complexity
- Selects the cheapest model that can handle that complexity level
- Forwards the request to the selected provider
- Returns the response with cost metadata
The classifier is the critical piece. A trained classifier that evaluates semantic complexity, not just surface features like token count, is what makes the routing decision worth trusting.
The overhead matters too. Classification latency adds to every request, so it has to stay small next to the model call it routes.
The ROI math for a typical enterprise
Here is what the numbers look like for a team spending $10,000 per month on LLM APIs with a mixed workload:
| Request type | Share of traffic | Without routing | With routing | Monthly cost (before) | Monthly cost (after) |
|---|---|---|---|---|---|
| Simple (formatting, lookups, FAQ) | 50% | Opus ($5/MTok) | Haiku ($1/MTok) | $5,000 | $1,000 |
| Medium (code gen, explanations) | 35% | Opus ($5/MTok) | Sonnet ($3/MTok) | $3,500 | $2,100 |
| Complex (architecture, debugging) | 15% | Opus ($5/MTok) | Opus ($5/MTok) | $1,500 | $1,500 |
| Total | 100% | $10,000 | $4,600 |
That is 54% savings on this assumed mix. The 15% that genuinely needs frontier reasoning still gets it, so the number to measure is whether the 85% routed down holds its quality.
Gartner forecasts worldwide AI spending at $2.52 trillion in 2026. If even a fraction of that is inference spend that could be routed more efficiently, the aggregate savings run into billions.
How to evaluate routing for your workload
Not every workload benefits equally. Here is a quick diagnostic:
Routing helps most when:
- Your prompt mix includes 40%+ simple or medium-complexity requests
- You make more than a few hundred API calls per day
- You run agentic workflows (coding assistants, multi-step chains)
- Your monthly LLM spend exceeds $500
Routing helps least when:
- Nearly every request requires complex reasoning (legal analysis, medical diagnosis)
- You are already on the cheapest available model
- Your volume is too low for the savings to matter
The fastest way to check: pull a week of API logs, bucket each request by complexity, and calculate what the cost would have been if simple requests went to Haiku and medium requests went to Sonnet. If the theoretical savings exceed 30%, routing pays for itself immediately.
The market is moving fast
Intel Market Research projects the LLM middleware and gateway market will grow about 49.6% a year through 2034, from a small base of under $20 million. Routing is becoming standard infrastructure, not a nice-to-have optimization.
Nadir can shadow each request and return an auditable model-and-effort decision without calling a model provider; its Decision API targets about 100 ms. NadirClaw is source-available under PolyForm Noncommercial and runs locally. Hosted Route can execute and measure requests; its savings ledger uses the customer-declared baseline, while decision-only estimates remain advisory.
Sources: [AI.cc press release, "Enterprise Token Costs Drop 67% Year-Over-Year"](https://www.einpresswire.com/article/911544568/aicc-report-enterprise-token-costs-drop-67-year-over-year-as-multi-model-ai-adoption-hits-record-high) (May 2026). [a16z, "How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025"](https://a16z.com/ai-enterprise-2025/) (June 2025). [Intel Market Research, "LLM Middleware and Gateway Market"](https://www.intelmarketresearch.com/llm-middleware-gateway-market-30754) (2026). [FinOps Foundation, "State of FinOps 2026"](https://data.finops.org/) (2026). [Gartner, "Worldwide AI Spending Forecast"](https://www.gartner.com/en/newsroom/press-releases/2026-1-15-gartner-says-worldwide-ai-spending-will-total-2-point-5-trillion-dollars-in-2026) (January 2026). Anthropic, OpenAI, Google model pricing as of May 2026.