The Ten-Point Discount

TL;DR.

On September 30, Cloudflare put a model router into public beta inside AI Gateway. Set the model to cloudflare/auto and a classifier at the edge picks a model for each request. It's free while in beta. Cloudflare published its own benchmark with the launch, and the headline is good: on 291 tool-using knowledge-work trials, Auto Router cost $2.10, against $5.91 for Claude Opus 5.5 and $2.64 for GPT-6 Sol. That's 35% of the Opus bill.

The same table has a second number. Opus 5.5 completed 96.6% of trials. Auto Router completed 86.6%. Ten points of success rate is what the 65% discount buys, and Cloudflare reports it honestly. The question it leaves to you is what a failed task costs.

• Break-even is 13 cents. Auto Router saves $3.81 across the 291 trials and fails 29 more of them. If detecting and redoing one failed task costs you more than $0.13, Opus alone is the cheaper way to run this workload. A minute of an engineer's time is worth several times that.
• Against Sol, the router wins outright. It costs 20% less and its success rate is 2.4 points higher, though the intervals overlap. If your default is a mid-tier model, Auto Router is a free upgrade on this benchmark.
• A check after the answer recovers most of the gap. In our model, running Auto Router first and sending tasks a cheap verifier flags to Opus reaches 93.5% to 95.5% success at $11 to $12 per 1,000 tasks, against Opus at $20.31. A perfect check would match Opus quality at about half its cost. Real checks land in between.

Below: how Auto Router decides, what the benchmark measures, the failure-cost math with a script to run it on your own eval, and a one-afternoon test plan for deciding whether cloudflare/auto belongs in front of your traffic.

Benchmark figures are Cloudflare's, from its September 30, 2026 launch post. The failure-cost model, the cascade scenarios and every per-1,000-task figure derived from them are our own illustrative calculations, not Cloudflare or Nadir customer data.

What Cloudflare shipped.

Auto Router lives inside AI Gateway, Cloudflare's proxy for model APIs. You keep calling the gateway, change the model string to cloudflare/auto, and the router chooses from a pool of models Cloudflare maintains. The launch post, by Ming Lu and Harrison Harnisch, describes four stages, and they're worth reading closely because they're a clean public description of how a production pre-generation router is built.

1. Build the pool. Drop models that can't serve the request format or mode, that your gateway credentials, billing settings, access policies or spend limits exclude, and providers that are currently unhealthy.
2. Classify the conversation. A multi-head classifier running on Workers AI GPUs reads a compact view of the conversation, weighted toward the newest turns. It outputs probabilities over 14 task categories (coding, planning, research, data analysis and so on) and a 1 to 5 rating on four dimensions: complexity, ambiguity, stakes, and dependence on earlier context.
3. Score the fit. A matrix combines those signals with each model's benchmark results. Adding a model means adding its benchmark-derived weights, not retraining the classifier.
4. Pick by utility. utility = expected quality - adaptive cost penalty, priced from each model's input and output token rates. The cost penalty shrinks as difficulty rises, so hard tasks drift toward strong models and easy ones toward cheap ones.

Two details show the team has run this in production. Inside a turn, the router stays on one model. Across turns, a switching penalty grows with the tokens already in context, because the model holding a warm prompt cache reads it at the cache rate while any other model pays full price to re-read it, and reasoning tokens can't be handed from one model to another. We've written about routers that fight the prompt cache and cache-blind routing. Auto Router prices the switch, which is the right fix.

The output is a ranked list, and AI Gateway falls through it if a provider can't serve the request. What the post doesn't describe is a step that looks at the answer. Selection is one ranking pass before any tokens are generated. That's the design choice the rest of this post is about.

What the benchmark says.

Cloudflare ran an internal benchmark of 97 knowledge-work tasks with simulated workspace tools (email, calendars, Slack, files, travel, finance). Each task needs tool calls to reach a verifiable answer or action. Three samples per model per task gives 291 trials, with 95% intervals from 10,000 task-level bootstrap resamples.

Three readings.

Against Sol, it's a clean win on cost and a tie on quality. Auto Router is 20% cheaper and 2.4 points ahead, but the intervals overlap almost entirely, so "at least as good as Sol, for less" is the claim the data supports. Cloudflare's own framing is close to that.

Against Opus, the quality gap is real. Auto Router's upper bound (92.8%) touches Opus's lower bound (92.8%). On a 97-task benchmark that's about as separated as two numbers get. Cloudflare says Auto Router "does best" across a wide range of knowledge-work tasks and is the starting point, not the finished product. The table agrees: it's a cost play with a measurable quality price.

Cost per success flatters the cheap option. $0.0084 against $0.0210 reads like 2.5x better value. It divides spend by the tasks that worked and says nothing about the 39 that didn't. Those tasks didn't stop costing money when the model returned. Someone still has to notice the wrong calendar invite, the email that went to the wrong thread, or the expense report with the wrong total, and fix it.

The failure-cost break-even.

Put a price on a failed task. Call it F: the cost to detect it, redo it, and absorb whatever it broke. Then the full cost of a workload is model spend plus failures times F.

total = model_spend + failed_tasks × F

auto:  $2.10 + 39 × F
Opus:  $5.91 + 10 × F

break-even:  F = (5.91 − 2.10) / (39 − 10) = $0.131

Above 13 cents per failure, Opus alone is cheaper. Per 1,000 tasks:

The model bill is the small number in every row past the second. That's the uncomfortable part of routing math nobody puts on a launch slide: at realistic failure costs, the quality gap is worth more than the token savings. When F is near zero, cheap wins easily. Think background enrichment, draft suggestions a human edits anyway, or a classification a later stage double-checks. When a failure reaches a customer or eats an engineer's minute, it loses quickly.

This isn't a Cloudflare problem. It's the arithmetic of every router that commits before it sees an answer, ours included. Cloudflare deserves credit for publishing the number that makes it visible.

Buying back the ten points.

The usual fix is a cascade. Run the cheap path first, check the answer, and send only the flagged tasks to the strong model. Whether it pays depends on two properties of the check: recall (what share of real failures it flags) and false flags (what share of good answers it sends up anyway).

We modeled it on Cloudflare's numbers. Assumptions, all illustrative: a check costs $0.0005 per task, roughly a decision-model call on a short trace. Flagged tasks are rerun on Opus 5.5 at Opus's average per-trial cost. Opus solves 75% of the tasks Auto Router failed, below its 96.6% average because those are the harder tasks. It solves 98% of the good answers that get flagged by mistake.

Two things to take from it.

Recall is what you're paying for. Moving from 70% to 90% recall adds 54 cents per 1,000 tasks and pushes the break-even from 29 cents to 77. The check is the cheapest line on the bill and decides most of the outcome. That's why calibrating the check and sweeping its threshold matter more than adding another model to the pool.

A cascade still loses when failures are expensive. Even the 90%-recall row is beaten by Opus alone once a failure costs more than 77 cents. For agent tasks with side effects (a sent email, a booked flight, a merged PR), the real answer may be to route those task categories straight to the strongest model and keep the router for everything else. Auto Router's classifier already outputs a stakes score. Using it as a hard floor, not just one term in the utility, would be the cheapest quality fix available.

Run the numbers on your own eval.

Cloudflare's 97 tasks aren't your workload. The break-even is easy to compute from any head-to-head eval where you know, per task, whether each option succeeded and what it cost.

# breakeven.py: what does a failed task have to cost before the cheap option loses?
import csv, sys
from collections import defaultdict

# eval.csv columns: option,task_id,succeeded(0/1),cost_usd
spend, fails, n = defaultdict(float), defaultdict(int), defaultdict(int)
for row in csv.DictReader(open(sys.argv[1])):
    o = row["option"]
    spend[o] += float(row["cost_usd"])
    fails[o] += 0 if row["succeeded"] == "1" else 1
    n[o] += 1

cheap, strong = sys.argv[2], sys.argv[3]
per = lambda o: (spend[o] / n[o], fails[o] / n[o])
(c_cost, c_fail), (s_cost, s_fail) = per(cheap), per(strong)
print(f"{cheap}: ${c_cost*1000:.2f}/1k tasks, {c_fail:.1%} failed")
print(f"{strong}: ${s_cost*1000:.2f}/1k tasks, {s_fail:.1%} failed")

if c_fail <= s_fail:
    print(f"{cheap} fails no more often; it wins at any failure cost.")
else:
    be = (s_cost - c_cost) / (c_fail - s_fail)
    print(f"Break-even: ${be:.3f} per failed task.")
    for F in (0.01, 0.1, 0.5, 1, 5):
        tc, ts = 1000 * (c_cost + c_fail * F), 1000 * (s_cost + s_fail * F)
        print(f"  F=${F:<5} {cheap} ${tc:8.2f}  {strong} ${ts:8.2f}  -> {cheap if tc < ts else strong}")

python breakeven.py eval.csv cloudflare/auto claude-opus-5-5

Fed Cloudflare's totals, it prints the same $0.131. The hard part isn't the script. It's choosing F honestly. Three ways to anchor it:

• Human time. Minutes to notice and fix, times loaded cost per minute. At $90 an hour, one minute is $1.50.
• Retry cost. If failures are detected automatically and retried on the strong model, F is about one strong-model call plus latency. That's the floor.
• Blast radius. For actions with side effects, the cost of the wrong action, not the redo. This is usually the largest and the least measured. Our retry tax post covers the blind-retry version of the same trap.

An afternoon test plan for cloudflare/auto.

If you're already on AI Gateway, trying Auto Router is a config change, and it's free during the beta. Do it on your traffic before trusting either our numbers or Cloudflare's.

1. Pull 300 to 500 real requests with a checkable outcome: a test that passes, a field that matches, a tool call with the right arguments. Free-text quality needs a judge, and judges have their own bill.
2. Replay them three ways. Your current pinned model, cloudflare/auto, and the strongest model you'd accept. Through AI Gateway's OpenAI compatible endpoint that's one loop:

from openai import OpenAI

# AI Gateway's OpenAI compatible endpoint. Check Cloudflare's docs for your account's URL.
client = OpenAI(
    base_url="https://gateway.ai.cloudflare.com/v1/<ACCOUNT_ID>/<GATEWAY>/compat",
    api_key="<CF_AIG_TOKEN>",
)

for option in ["cloudflare/auto", "<your-current-model>", "<your-strongest-model>"]:
    for req in sample:
        r = client.chat.completions.create(model=option, messages=req["messages"])
        record(option, req["id"], r.model, r.usage, check(req, r))

1. Log which model the router actually chose (r.model) per request. If 90% of your traffic lands on one model, you're paying for a classifier that makes one decision.
2. Score, cost, and run `breakeven.py`. Then split the result by task type. A router that's fine on summaries and weak on multi-step tool calls is a router you use on summaries.
3. Re-run monthly. The pool, the weights and the models behind them change under a stable model name. Cloudflare lists more models, zero-data-retention filtering and reasoning-level selection as coming next.

When to use it, and when not.

Where Nadir fits.

Nadir is the same idea from a different starting point. Send model="auto" to an OpenAI compatible endpoint, and a trained classifier reads each prompt and picks among the models you've configured, against a quality floor you set per API key. That floor is the hard stakes rule from the table above. Each response's metadata reports the model that answered, the cost, and the routing decision, so the break-even inputs come from your own traffic instead of a launch post. You keep your own provider keys.

For the ten points, we built an opt-in verifier for cheap-first cascades, with the costs and limits written up in the Jev and Opus 5.5 cascade post. It's off unless you turn it on, and the recall numbers above are a model, not its measured performance. And if you'd rather not move traffic to find out, Observe mode records the decision Nadir would make on every request without changing a single response. That gives you the replay in step 2 without building the harness.

The useful comparison isn't Nadir against Cloudflare. It's any router against your current pinned model, at your F. Start with a free key, run a week in Observe mode, and put both numbers next to each other.

Sources: Cloudflare, ["Cut your AI spend with AI Gateway's Auto Router"](https://blog.cloudflare.com/auto-router/), September 30, 2026, for the router design, beta pricing, roadmap, and every benchmark figure in the measured table (97 tasks, three samples each, 95% bootstrap intervals). The cost-per-1,000-task conversions, failure-cost break-evens, cascade recall and false-flag scenarios, the $0.0005 check cost, the 75% and 98% escalation success rates, and both charts' modeled lines are our own illustrative calculations. The AI Gateway endpoint in the code sample follows Cloudflare's OpenAI compatible gateway format; confirm it against Cloudflare's current documentation.