Two numbers from the same report that shouldn't both be true
On August 17, 2026, Gartner published a prediction with two halves that seem to contradict each other. Token prices will fall roughly 95% by 2030. And inference costs per agentic workflow will increase more than fivefold through 2028. Source: Gartner, "Predicts AI Inference Costs Per Agentic Workflow Will Increase More Than Fivefold Through 2028," August 17, 2026.
Both are true at once, and the reason they can coexist is the whole story. "Product leaders cannot rely on more efficient token economics to rationalize AI costs," said Will Sommer, Gartner Senior Director Analyst. "Each successive generation of AI capability will necessitate more, and often more expensive, tokens." Gartner's own name for this is the inference paradox: prices fall, bills rise, and the two trends do not cancel out. Sommer and fellow analyst Sabine Zimmerhansl put it more bluntly in coverage of the report: the market is captured by a "token-deflation illusion." Source: Computerworld, "AI inference is getting cheaper, but your agents are getting more expensive," August 2026.
This blog has tracked pieces of this pattern before: why 280x cheaper tokens didn't shrink anyone's bill, and why July's Forbes and Fortune pieces both concluded that cheaper tokens don't mean cheaper agents. What's new in the August report is a number attached to where this goes next, a named mechanism, and Gartner's own prescription for fixing it. That prescription is routing, and it's worth walking through exactly what they mean by it.
Why the two numbers don't cancel out
A simple chatbot reads a query and responds. An agent reasons about the query, decides whether it needs a tool, calls the tool, reads the result, decides whether that was enough, and sometimes repeats the loop. "Where a simple chatbot must read and interpret a query and quickly respond," Sommer explained, "an AI agent must constantly reason, negotiate, and question itself." Each of those extra steps is a separate model call, and Gartner's data shows the multiplier isn't small:
| Task type | Gartner's per-task cost estimate |
|---|---|
| Basic workflow | ~$0.05 |
| Summarization / knowledge retrieval | ~$0.10 |
| Complex, multi-step workflow | ~$0.30 |
| Planning and learning | ~$0.40 |
That's an 8x spread between the cheapest and most expensive task category, before accounting for which model answers each one. Layer in model choice and the gap widens further: routing a task to a frontier reasoning model instead of a standard chatbot model raises inference cost by a factor of five on Gartner's own numbers, with more complex scenarios running higher, up to 150x per task versus a basic chatbot call in the report's high end. Training and serving infrastructure for a mid-sized agentic model also runs about 2.5x the hardware cost of a simple chatbot deployment. None of that shows up in a per-token price chart. It shows up in how many tokens, and which tier of model, a single task now touches.
That's the mechanism behind the fivefold number. Per-token prices really are falling on the schedule this blog has documented before: roughly 90% cheaper inference by 2030 on Gartner's earlier March 2026 forecast, consistent with the 95% figure in the August report. But the thing being priced per token is changing faster than the price is falling. A task that took one chatbot call in 2025 now runs a five-step agent loop, each step billed separately, several of them landing on a more expensive model tier than the chatbot ever touched. Multiply a falling per-unit price by a rapidly rising unit count and the total can go either direction. Gartner's forecast says it goes up, by more than 5x, through 2028.
What this costs beyond the invoice
The report attaches a second, harder number to what happens when teams don't see this coming: more than 40% of agentic AI initiatives are on pace to be scaled back or shelved due to unmanaged costs and unclear ROI, and over half of generative AI projects are forecast to exceed budget from architecture choices made before anyone measured cost per completed task. Source: The Register, "Agentic AI costs set to balloon fivefold by 2028," August 17, 2026. A project that gets cancelled for going over budget rarely fails on capability. It fails because nobody could show the CFO what a completed task actually cost, until the bill made the question unavoidable.
Gartner's own fix is a routing problem
The recommendation in the report isn't vague. It's a named practice: inference tiering. Route each task to the most cost-efficient model capable of handling it, and structurally block agents from defaulting to a frontier reasoning model when a smaller one would do. "Defaulting to generic autonomous intelligence will result in unbounded costs orders of magnitude higher than those of optimized product ecosystems," Sommer said. Gartner pairs that with three more concrete practices: shift from flat-fee to usage-based, tiered pricing internally so cost visibility matches actual consumption; treat each model release like a depreciating asset and refresh on a cycle rather than standardizing on one model indefinitely; and define a success threshold per workload, then stress-test it against token-price and token-volume swings before they hit production.
That first practice, tiering by task, is a routing decision made at every single request. Here's what it looks like built by hand, and what it looks like when the routing layer owns it.
Built by hand
def pick_model(task):
# Someone has to write this, keep it in sync with pricing,
# and update it every time a model is deprecated or repriced.
if task.complexity == "planning":
return "claude-opus-4-8"
elif task.complexity == "complex":
return "claude-sonnet-5"
elif task.complexity == "summarization":
return "claude-haiku-4-5"
else:
return "claude-haiku-4-5"
model = pick_model(task)
resp = client.chat.completions.create(model=model, messages=task.messages)This works until a model gets deprecated, a price changes, or a new task type shows up that the if/else chain never anticipated. Someone owns that file forever, and Gartner's "continuous refresh cycle" recommendation means it needs updating on a schedule, not just when something breaks.
Routed automatically
from openai import OpenAI
client = OpenAI(base_url="https://api.getnadir.com/v1", api_key="YOUR_NADIR_KEY")
resp = client.chat.completions.create(
model="auto", # tiering happens per request, not in your code
messages=task.messages,
)The tiering logic moves out of application code and into infrastructure that gets versioned and updated independently. When a provider deprecates a model or ships a cheaper one, the routing layer adapts without a pull request against your agent's codebase. The dashboard shows cost per request and per task, which is the same "value-per-outcome tracking" Gartner recommends teams build before the 2028 forecast catches up with them, not after.
Generic intelligence versus an optimized ecosystem, at Gartner's own numbers
Take a workload that mixes Gartner's four task tiers evenly, one basic, one retrieval, one complex, one planning task, and price it two ways.
| Route everything to one frontier reasoning model | Tier by task, route each to the cheapest capable model | |
|---|---|---|
| Basic workflow | Frontier rate | ~$0.05 |
| Summarization / retrieval | Frontier rate | ~$0.10 |
| Complex workflow | Frontier rate | ~$0.30 |
| Planning / learning | Frontier rate (this is the one task that needs it) | ~$0.40 |
| Blended cost per task | Every task priced at the planning tier or close to it | $0.05 to $0.40, weighted by what each task actually requires |
The left column is what Gartner calls defaulting to generic autonomous intelligence. The right column is what they call an optimized product ecosystem. The report's fivefold cost increase describes what happens to the left column between now and 2028 as agent capability keeps expanding. The right column is the one lever in the report that doesn't scale the same way, because it prices each task at what it needed, not at what the most capable model in the fleet costs.
The part worth acting on before the forecast catches up
A forecast that a cost multiplies by 2028 is a two-year runway, not an emergency. That's exactly the window Gartner's report is aimed at: the teams that build inference tiering into their architecture now are the ones whose costs grow with usage instead of with model capability. The teams that don't are the ones supplying next year's 40% and 50% statistics.
Nadir is the managed version of the tiering Gartner is describing: an OpenAI-compatible endpoint, model="auto", and a dashboard that shows cost per request the way "value-per-outcome tracking" requires. It doesn't require standing up the classifier and refresh cycle yourself. Start free and see what your own basic, retrieval, complex, and planning tasks actually cost when they stop all paying the planning-tier rate.
Sources: [Gartner, "Predicts AI Inference Costs Per Agentic Workflow Will Increase More Than Fivefold Through 2028," August 17, 2026](https://www.gartner.com/en/newsroom/press-releases/2026-08-17-gartner-predicts-ai-inference-costs-per-agentic-workflow-will-increase-more-than-fivefold-through-2028). [Computerworld, "AI inference is getting cheaper, but your agents are getting more expensive," August 2026](https://www.computerworld.com/article/4210786/ai-inference-is-getting-cheaper-but-your-agents-are-getting-more-expensive.html). [The Register, "Agentic AI costs set to balloon fivefold by 2028," August 17, 2026](https://www.theregister.com/ai-and-ml/2026/08/17/agentic-ai-costs-set-to-balloon-fivefold-by-2028/5288363).