After the Answer

TL;DR.

A paper published on October 9, 2026 measured, with a common yardstick, how much of a reasoning model's thinking happens after it already had its best answer. Across six open reasoning models and four benchmark groups, the answer is 64.3% of thinking tokens. Thinking all the way to the end also cost 9 points of accuracy on average against stopping at the peak.

Both halves matter for the bill. Reasoning tokens are billed as output, the most expensive token on every rate card, and on these runs the average question used 8,281 of them. At $10 per million output tokens, the 64% after the peak is about $0.053 a question, or $53,000 per million questions, for thinking that, on average, made the answer worse.

The paper also asks which fix could recover most of it, and the ranking isn't the one a router vendor would write:

These are hindsight optima, not methods you can deploy. But the order is the useful part: a yes/no decision before generation is the coarsest lever, a per-question budget is twice as strong, and stopping mid-thought is strongest of all.

This post covers:

• What the overthinking ratio measures, and why it isn't the same as harm
• Why the three strategies rank the way they do, and which ones a hosted-API team can actually use
• A Python tutorial: sweep reasoning effort on a labeled sample of your own traffic and get a per-category effort policy with its cost change
• A short recipe to measure the real overthinking ratio on a self-hosted thinking model

All model and benchmark numbers are from the paper. The dollar figures apply an illustrative $10 per million output tokens to the paper's average thinking length. The tutorial output is synthetic and labeled as such.

The measurement.

The method is called Step-wise Truncation Probing (STP). Take a reasoning trace. Cut it at 16 evenly spaced points. At each cut, close the thinking block and force the model to answer from what it has so far. Score each forced answer. The earliest cut that reaches the trace's best accuracy is the reference point.

Two numbers fall out of that:

• Overthinking ratio (OR): the share of thinking tokens generated after the reference point. Tokens you paid for that didn't improve the answer.
• Accuracy degradation (AD): best accuracy minus full-thinking accuracy. How much the extra thinking actively hurt.

The authors ran it on GPT-OSS-20B, Qwen3-4B-Thinking-2507, Qwen3-30B-A3B-Thinking-2507 and three DeepSeek-R1-Distill-Qwen models (7B, 14B, 32B), over AIME 2024 to 2026, MATH500, GPQA-Diamond and LiveCodeBench. That's 963 questions and 92,448 trajectories, released as a benchmark called OTBench.

Read the two columns separately. MATH500 has the most wasted thinking and the least damage: the model usually has the answer early and then keeps checking it, which costs money but rarely changes the answer. GPQA-Diamond is the opposite: less waste, but the extra thinking talks the model out of right answers 17 points' worth. Same symptom on the invoice, different problem in the output.

Waste is not harm.

Per model, the two numbers barely move together. The Spearman correlation across model-benchmark pairs is -0.33, weak and negative.

GPT-OSS-20B wastes the least, 41%, and still loses almost 8 points. Qwen3-30B-A3B wastes the most, 74%, and loses the least. The smallest distilled model, R1-Distill-7B, loses 13.4 points, the most of any.

Two practical consequences:

1. A token-count dashboard can't tell you which problem you have. High reasoning-token spend on a task can be harmless padding (cut it, save money) or active damage (cut it, save money and gain accuracy). You need a quality signal next to the token signal.
2. The answer moves per trajectory, not just per model. The paper reports an average pairwise gap of 20 percentage points in OR between trajectories of the same model on the same question. The same prompt, sampled twice, can stop paying off at very different points. That's why fixed rules leave so much on the table.

One caveat worth stating: the abstract cites 70.5 billion reasoning tokens generated, while the appendix lists 576.7 million stored thinking tokens. The paper doesn't reconcile the two. Neither changes the ratios.

Three ways to stop.

The paper puts the three families of fixes on the same chart, using the full probe data to compute the best each could possibly do.

Dynamic routing here means one decision per question, before generation: think fully, or don't think at all. Matching full-thinking accuracy takes 71.4% of the budget, and the best it ever reaches is about 73%.

Fixed budget means one token cap per question, also set before generation. It needs 35.0% of the budget to match, and tops out near 76%.

Dynamic early exit decides during generation, per trajectory, whether to stop. It needs 14.2% to match, and reaches about 81% at 58.6% of the budget. Stopping at the right moment doesn't just save tokens, it buys 9 points of accuracy.

The order follows from how much each strategy is allowed to know. A before-generation decision sees only the prompt. A budget is still blind to the trajectory but can pick a point between nothing and everything. Early exit sees the reasoning as it unfolds. More information, better frontier.

Three honest caveats:

• These are oracles. They're built from every probe of every trajectory, in hindsight. The authors call them "idealized theoretical references rather than deployable stopping algorithms." The published methods they tested, ThinkSwitcher, Predictive Scheduling and ThinkBrake, all land below their own frontier.
• "Routing" in the paper is think-or-don't inside one model. It doesn't cover choosing a different model, where the price difference between tiers is often 10x or more on its own. A model router and a thinking router are two different levers on the same bill.
• Early exit needs the trace. Hosted reasoning APIs bill for thinking tokens but mostly return a summary or nothing, and you can't inject a stop-and-answer mid-thought. Early exit is a self-hosting technique today. For hosted models, the budget lever is the one you can pull.

What this means for a hosted-API bill.

Most production traffic runs on hosted reasoning models, where the controls are an effort level (reasoning_effort, Anthropic's effort) or a token budget (thinking_budget, budget_tokens). We compared how each vendor exposes those in the overthinking tax.

The OTBench result says that dial is worth much more than people treat it as. In the paper, choosing the budget per question beats a yes/no thinking switch by a factor of two. Most deployments do neither: one effort level per deployment, usually the vendor default, for every prompt from field extraction to multi-step proofs. That's a fixed budget chosen once for the whole workload, which is the worst version of the second-best strategy.

You can't get per-question oracle budgets. You can get per-category budgets from data, and that captures a lot of it, because the biggest differences are between kinds of work, not between individual questions. Here's how.

Tutorial: an effort policy from your own traffic.

You need:

1. A labeled sample per task category. 50 to 200 prompts each, with a way to score the answer: exact match, a unit test, a rubric, or a judge model you trust for that category.
2. An effort ladder your model accepts, cheapest first.
3. Reasoning tokens per call. Most providers report them in usage (OpenAI's completion_tokens_details.reasoning_tokens). If yours doesn't, completion tokens minus visible answer tokens is close enough.

Step 1: sweep.

from collections import defaultdict

LADDER = ["low", "medium", "high", "xhigh"]   # cheapest first

def sweep(samples, run, score, n=4):
    """samples: dicts with category, prompt, expected.
    run(prompt, effort) -> (answer, reasoning_tokens)."""
    rows = []
    for s in samples:
        for effort in LADDER:
            for _ in range(n):
                answer, used = run(s["prompt"], effort)
                rows.append({"category": s["category"], "effort": effort,
                             "ok": score(answer, s["expected"]), "used": used})
    return rows

n=4 samples per setting matters. The paper's 20-point spread between trajectories of the same question is a warning that one sample per setting is noise. This is STP at the granularity a hosted API allows: instead of cutting one trace at 16 points, you run the whole question at four budgets.

Step 2: pick the cheapest level that's as good as the best.

def effort_policy(rows, default="high", tolerance=0.02):
    """Per category: the cheapest effort within `tolerance` of the best accuracy,
    and the share of the default's reasoning tokens it removes."""
    acc, used = defaultdict(list), defaultdict(list)
    for r in rows:
        acc[r["category"], r["effort"]].append(r["ok"])
        used[r["category"], r["effort"]].append(r["used"])
    mean = lambda xs: sum(xs) / len(xs)
    policy = {}
    for cat in sorted({c for c, _ in acc}):
        curve = [(e, mean(acc[cat, e]), mean(used[cat, e])) for e in LADDER]
        best = max(a for _, a, _ in curve)
        effort, a, u = next(p for p in curve if p[1] >= best - tolerance)
        d_acc, d_used = mean(acc[cat, default]), mean(used[cat, default])
        policy[cat] = {"effort": effort, "accuracy": a, "default_accuracy": d_acc,
                       "tokens": u, "default_tokens": d_used}
    return policy

This is the overthinking ratio's logic applied to a budget ladder: find the earliest point that reaches peak accuracy, and treat everything past it as waste. The tolerance is your quality floor. Set it per category if some categories matter more than others.

Note what the policy does not do: it never assumes lower is better. If a category's best accuracy needs more effort than the default, the policy raises it.

Step 3: price the change.

def money(x):
    return ("-" if x < 0 else "+") + f"${abs(x):,.0f}"

def report(policy, usd_per_m_output, calls_per_month):
    total = 0.0
    for cat, p in policy.items():
        change = p["tokens"] / p["default_tokens"] - 1
        usd = (p["tokens"] - p["default_tokens"]) * usd_per_m_output / 1e6 * calls_per_month[cat]
        total += usd
        print(f"{cat:<8} {p['effort']:<6} acc {p['accuracy']:.0%} vs {p['default_accuracy']:.0%} "
              f"at default, reasoning tokens {change:+.0%}, spend {money(usd)}/month")
    print(f"net spend change: {money(total)}/month")

On a synthetic workload of four categories and one million calls a month at $10 per million output tokens, it prints:

extract  low    acc 96% vs 94% at default, reasoning tokens -88%, spend -$9,200/month
proofs   xhigh  acc 80% vs 63% at default, reasoning tokens +103%, spend +$7,600/month
sql      high   acc 85% vs 85% at default, reasoning tokens +0%, spend +$0/month
support  medium acc 92% vs 91% at default, reasoning tokens -61%, spend -$6,900/month
net spend change: -$8,500/month

The synthetic numbers are made up, but the shape is the one to expect. Easy categories give back most of their reasoning tokens. Some category is under-thinking at the default and the honest fix costs money: here proofs gains 17 points for $7,600 a month. And the net is still a saving, because the savings come from high-volume, low-difficulty work, which is where most traffic is.

Step 4: re-run it when the model changes.

The paper's per-model spread is the reason this isn't a one-time job. GPT-OSS-20B and Qwen3-30B-A3B have opposite profiles. A new model version, a new effort default (like Haiku 5.5's medium default), or a new tokenizer moves every number in the table. Keep the labeled sample, and rerun the sweep on every model change.

Self-hosting? Measure the real ratio.

If you serve an open thinking model with vLLM, you can run STP itself on a sample of your traffic. The loop is short: take a finished trace, cut it, close the thinking block, force an answer.

from openai import OpenAI
from transformers import AutoTokenizer

MODEL = "Qwen/Qwen3-4B-Thinking-2507"
tok = AutoTokenizer.from_pretrained(MODEL)
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")

def probe(question, thinking, score, expected, cuts=16, n=4):
    """Accuracy of forced answers at `cuts` evenly spaced prefixes of one trace.
    Returns (overthinking_ratio, accuracy_lost)."""
    ids = tok.encode(thinking)
    # The Qwen3 thinking template opens <think> in the generation prompt.
    base = tok.apply_chat_template([{"role": "user", "content": question}],
                                   tokenize=False, add_generation_prompt=True)
    curve = []
    for k in range(cuts + 1):
        prefix = tok.decode(ids[: len(ids) * k // cuts])
        out = client.completions.create(model=MODEL, prompt=base + prefix + "\n</think>\n\n",
                                        max_tokens=512, n=n, temperature=0.6)
        curve.append(sum(score(c.text, expected) for c in out.choices) / n)
    best = max(curve)
    first = curve.index(best)
    return 1 - first / cuts, best - curve[-1]

Run it over a few hundred real prompts and average. If your ratio is high and accuracy lost is near zero, a lower budget is free money. If accuracy lost is high, you have a quality problem that more thinking is making worse, and an early-exit method (the paper's best frontier) is worth trying. Check the chat template of your model before trusting the prompt construction: the <think> handling differs between model families.

Where Nadir fits.

There are two separate decisions on every reasoning request: which model, and how hard it thinks. The OTBench paper is about the second. Most of the money in a typical workload is in the first, because the price gap between model tiers is often larger than the gap between effort levels on one model. A request that doesn't need a reasoning model at all shouldn't be paying for 8,000 thinking tokens, however well they're budgeted.

Nadir works on both. With model="auto", a trained classifier reads each prompt and sends it to the cheapest model that can handle it, among the models you've configured and against the quality floor you set on the API key. Pass one normalized reasoning field and the proxy maps it to reasoning_effort, effort or a thinking budget for whichever provider the request lands on, so an effort policy like the one from Step 2 is one field on the request instead of three provider branches. Every response reports the model served and its cost, and tagging requests with X-Nadir-Use-Case gives you the per-category split that Step 3 needs.

If you want the sweep's categories filled in from live traffic before changing anything, Observe mode records the routing decision Nadir would make on every request without altering a response. Start with a free key.

What Nadir doesn't do: it doesn't stop a hosted model mid-thought. Nobody outside the provider can, today. For agent loops that run too many steps, which is the same problem one level up, see the stopping problem.

Conclusion.

On six open reasoning models, 64% of thinking tokens came after the best answer, and thinking to the end cost 9 points of accuracy on average. The waste and the harm are different numbers, and a token dashboard sees only the first. The paper's oracle ranking is clear: deciding think-or-don't up front is the weakest lever, a per-question budget is twice as strong, and stopping mid-thought is strongest but only available when you host the model. For a hosted-API team, the move is a per-category effort policy measured on your own labeled traffic, priced honestly in both directions, and rerun on every model change. Then put model choice in front of it, because the cheapest thinking token is the one a smaller model never needed.

Sources: Weixin Guan, Liang Li, Shiyu Xie, Jiapeng Liu, Bing Li, Peng Fu, Chengyang Fang, Xiaoshuai Hao, Can Ma and Weiping Wang, ["Constructing Pareto Frontiers of Optimal Reasoning Termination Methods for Overthinking in Reasoning Models"](https://arxiv.org/abs/2610.13570), arXiv 2610.13570, submitted October 9, 2026; Table 1 for overthinking ratio and accuracy degradation, the frontier analysis for the strategy comparison. Found via [Token Cost Radar](https://jcodemunch.com/radar/2026-10-13), October 13, 2026. Dollar figures use an illustrative $10 per million output tokens and the paper's 8,281-token average thinking length on open models; hosted models think for different lengths. Tutorial output is synthetic and is not Nadir customer data.