Spend It Upstream

If a multi-step agent workflow can afford the frontier model on only one step, the September 2026 Planner-as-Router paper says to put it on the first one. In its table, frontier on the root step bought accuracy at $2.74 per point per 1,000 tasks, and frontier on the last step at $11.00, for less accuracy. The planner-routed version cut cost 44% against all-frontier for 2.9 points, while a FrugalGPT-style cascade cost 2.9x all-small and scored lower. The study is small, 54 tasks with a ±6-point interval, so read it as a direction. Today's 40x gap between Haiku 5.5 and Opus 5.5 makes placement matter more. Includes upstream.py, 67 lines that price every tier placement on your own chain.

Published 2026-10-31 by Dor Amir on the Nadir blog.

Filed under Routing & Cascades.

TL;DR.

A multi-step agent workflow has a question no per-request router answers: if you can afford the frontier model on one step, which step? A September 2026 paper, Planner-as-Router (arXiv:2609.32917), priced seven ways to place model tiers across the same enterprise workflows. The answer in its table is upstream.

Paper figures are from the authors' main results table. The four-step chain, its pass rates and token counts in the tutorial are ours, and are illustrations, not measurements. No figure here comes from Nadir customer data.

The question per-request routing skips.

Most routing advice is per request: look at a prompt, pick the cheapest model that can answer it. That works for chat. An agent workflow is a chain. A planner splits the job, one step writes a query, another runs it and extracts fields, a last step writes the answer. Every step reads what the step before it produced.

In a chain, a step's model choice isn't independent. A wrong query in step two poisons extraction in step three and the summary in step four, and steps three and four still bill you for their tokens. The paper's question is where in that chain the expensive model earns its price.

Planner-as-Router (PaR) folds the tier decision into planning. When the planner splits a query into subtasks, it also tags each subtask small, mid or frontier, with a short cost rationale. It sees the whole dependency graph before any specialist runs, and it needs no separate router model and no training data.

What the paper measured.

The authors built EntBench: 54 enterprise agent tasks across seven classes, including SQL and MongoDB generation graded by running the query against a live database. Tiers were Claude Haiku 4.5 ($1/$5 per million tokens), Sonnet 4.6 ($3/$15) and Opus 4.7 ($5/$25). The planner was Haiku 4.5 for every router, so planning cost is held constant. 1,157 evaluations across eight routers and three seeds.

Scatter chart of seven routers on EntBench, accuracy against cost per 1,000 tasks. All frontier: 29.0% at $18.20. Source frontier: 28.5% at $14.60. Planner-as-Router: 26.1% at $10.10. Sink frontier: 25.4% at $12.70. All small: 24.6% at $3.90. PaR-Lite: 24.6% at $10.50. FrugalGPT cascade: 23.2% at $11.30. All small, Planner-as-Router, Source frontier and All frontier form the observed cost-accuracy frontier.
Scatter chart of seven routers on EntBench, accuracy against cost per 1,000 tasks. All frontier: 29.0% at $18.20. Source frontier: 28.5% at $14.60. Planner-as-Router: 26.1% at $10.10. Sink frontier: 25.4% at $12.70. All small: 24.6% at $3.90. PaR-Lite: 24.6% at $10.50. FrugalGPT cascade: 23.2% at $11.30. All small, Planner-as-Router, Source frontier and All frontier form the observed cost-accuracy frontier.
RouterWhat it doesAccuracyCost per 1K tasks
All frontierOpus 4.7 on every step29.0%$18.20
Source frontierOpus on root steps, Haiku elsewhere28.5%$14.60
Planner-as-RouterPlanner tags each step's tier26.1%$10.10
Sink frontierOpus on terminal steps, Haiku elsewhere25.4%$12.70
All smallHaiku 4.5 on every step24.6%$3.90
PaR-LiteHaiku classifier skips the planner on easy tasks24.6%$10.50
FrugalGPTHaiku first, escalate low-confidence outputs23.2%$11.30

Excludes SQL-Compose, where every router scored 0%. 138 runs per router. The paper reports cost per task; we multiplied by 1,000.

The authors are careful about what this shows, and so should we be. With 54 tasks the confidence interval is about ±6 points, so they claim "frontier position," not an accuracy win. Absolute accuracies are low across the board. Learned routers like RouteLLM weren't in the comparison, and every model came from one provider.

Three things the table says anyway.

1. Spend at the root, not at the sink.

Source frontier and sink frontier spend roughly the same budget on the same idea: put the big model at one end of the chain. Root-first scored 3.1 points higher. Divide each router's extra cost over all-small by its extra accuracy and the gap is clearer.

Bar chart of extra cost per 1,000 tasks for each accuracy point gained over all-small. Source frontier: $2.74 per point. All frontier: $3.25. Planner-as-Router: $4.13. Sink frontier: $11.00. PaR-Lite spent $6.60 more for no accuracy gain. FrugalGPT spent $7.40 more and lost 1.4 points.
Bar chart of extra cost per 1,000 tasks for each accuracy point gained over all-small. Source frontier: $2.74 per point. All frontier: $3.25. Planner-as-Router: $4.13. Sink frontier: $11.00. PaR-Lite spent $6.60 more for no accuracy gain. FrugalGPT spent $7.40 more and lost 1.4 points.

The intuition is error propagation. The last step's input is already fixed: if the query upstream was wrong, the best summarizer in the world writes a fluent summary of wrong numbers. The first step's output is read by everything after it. On the hardest class, SQL-Compose, the paper attributes 79% of failures to the first SQL generation step.

That's the same shape we saw in Same Weights, Different Model: per-step accuracy compounds, so a small gap on an early step becomes a large gap end to end.

2. A cascade needs a check it can trust.

FrugalGPT was the worst router in the table: more expensive than PaR, and less accurate than sending everything to Haiku. A cascade only saves money when the signal that triggers escalation catches real failures. A confidence score on step four can't see that step two picked the wrong table.

The fix isn't "no cascades." It's putting the cascade where a cheap, hard check exists. A SQL query either runs and returns rows or it doesn't. JSON parses or it doesn't. That check is worth more than any self-reported confidence, and it's the step where escalation pays. We went through the general version in The Verification Discount and routing without verification.

3. Skipping the planner wasn't free either.

PaR-Lite adds a Haiku classifier that bypasses the planner on simple tasks. It cost slightly more than PaR and lost 1.5 points. Routing overhead is real, and a router that routes the router is one hop too many. The Routing Tax has the per-request math.

The gap got wider since the paper's models.

The paper's tiers were Haiku 4.5 at $1/$5 and Opus 4.7 at $5/$25: 5x apart. At October 2026 list prices, Haiku 5.5 is $0.10/$0.50 and Opus 5.5 is $4/$20: 40x apart.

Two things follow. All-small gets much cheaper relative to everything else, so the bar a frontier upgrade has to clear goes up. And the cost of a misplaced frontier call grows 8x, so where you put it matters more than it did when the paper ran.

Tutorial: upstream.py.

upstream.py prices every tier placement on a chain of steps. You give it each step's tokens and its pass rate on each tier, measured on your own eval set. It treats the chain as a product: a step's output is useful only if every step before it passed. Steps with a hard check (the SQL runs, the JSON parses) get a third option, cascade, which runs small first and re-runs on frontier only when the check fails. A failed check also stops a doomed run, so the later steps don't bill.

"""upstream.py: choose which agent steps get the frontier model.

Each step lists its tokens and its pass rate per tier, measured on your eval
set. A step's output is only useful if every step before it passed, so the
end-to-end pass rate is the product along the chain. A step with a cheap check
(SQL that runs, JSON that parses) can also "cascade": run small, and re-run on
frontier only when the check fails. A failed check also stops a doomed run.
"""
import itertools

PRICE = {"small": (0.10, 0.50), "frontier": (4.00, 20.0)}  # Haiku 5.5, Opus 5.5, $/1M
RECALL = 0.8  # share of real failures the check catches

# name: (input tokens, output tokens, small pass rate, frontier pass rate, has check)
STEPS = {
    "plan":      (2_000, 400, 0.80, 0.95, False),
    "write_sql": (3_000, 300, 0.70, 0.88, True),
    "extract":   (6_000, 600, 0.93, 0.96, False),
    "summarize": (4_000, 500, 0.95, 0.97, False),
}


def price(name, tier):
    i, o = STEPS[name][:2]
    return (i * PRICE[tier][0] + o * PRICE[tier][1]) / 1e6


def step(name, tier):
    """(cost, pass rate, share of runs that continue) for one step."""
    _, _, ps, pf, check = STEPS[name]
    if tier == "cascade":
        escalate = (1 - ps) * RECALL
        ok = ps + escalate * pf
        return price(name, "small") + escalate * price(name, "frontier"), ok, ok + (1 - ps) * (1 - RECALL)
    ok = ps if tier == "small" else pf
    return price(name, tier), ok, (ok + (1 - ok) * (1 - RECALL)) if check else 1.0


def run(plan):
    cost, ok, alive = 0.0, 1.0, 1.0
    for name, tier in zip(STEPS, plan):
        c, p, cont = step(name, tier)
        cost += alive * c
        ok *= p
        alive *= cont
    return cost, ok


def plans():
    tiers = [["small", "frontier"] + (["cascade"] if s[4] else []) for s in STEPS.values()]
    return itertools.product(*tiers)


def show(label, plan):
    cost, ok = run(plan)
    tag = " ".join(t[0].upper() for t in plan)
    print(f"{label:<17} {tag:<9} ${cost * 1000:>6.2f}  {ok:6.1%}  ${cost / ok * 1000:>7.2f}")


print(f"{'plan':<17} {'tiers':<9} {'per 1K':>7}  {'pass':>6}  {'per 1K ok':>9}")
show("all small", ("small",) * 4)
show("all frontier", ("frontier",) * 4)
show("frontier at sink", ("small",) * 3 + ("frontier",))
show("frontier at root", ("frontier",) + ("small",) * 3)
for target in (0.60, 0.75):
    ok_plans = [p for p in plans() if run(p)[1] >= target]
    show(f"cheapest >= {target:.0%}", min(ok_plans, key=lambda p: run(p)[0]))

python3 upstream.py prints:

plan              tiers      per 1K    pass  per 1K ok
all small         S S S S   $  2.03   49.5%  $   4.10
all frontier      F F F F   $ 90.05   77.8%  $ 115.67
frontier at sink  S S S F   $ 21.29   50.5%  $  42.15
frontier at root  F S S S   $ 17.63   58.8%  $  30.00
cheapest >= 60%   S C S S   $  6.68   64.4%  $  10.36
cheapest >= 75%   F C S S   $ 22.28   76.5%  $  29.13
Bar chart of six tier plans on an illustrative four-step chain, cost per 1,000 tasks and end-to-end pass rate. All small: $2.03, 49.5%. Frontier at sink: $21.29, 50.5%. Frontier at root: $17.63, 58.8%. Cascade the SQL step: $6.68, 64.4%. Root plus cascade: $22.28, 76.5%. All frontier: $90.05, 77.8%.
Bar chart of six tier plans on an illustrative four-step chain, cost per 1,000 tasks and end-to-end pass rate. All small: $2.03, 49.5%. Frontier at sink: $21.29, 50.5%. Frontier at root: $17.63, 58.8%. Cascade the SQL step: $6.68, 64.4%. Root plus cascade: $22.28, 76.5%. All frontier: $90.05, 77.8%.

The illustrative chain reproduces the paper's pattern. Opus on the summary step costs 10x all-small and buys one point, because the summary step was already passing 95% of the time. Opus on the plan buys 9 points for less money. Root plus a cascade on the checked step gets within 1.3 points of all-frontier for 25% of the cost.

How to adapt it:

  1. Measure pass rates per step, not per task. Run each step on both tiers against a fixed input from a known-good upstream, and grade the step's output alone. That's the number the product needs. Fifty examples per step is enough to see a 10-point gap, not a 2-point one.
  2. Use real token counts. Pull input and output tokens per step from your traces. Retrieval-heavy steps like extract are usually the biggest, which is why putting Opus on them is the expensive mistake.
  3. Be honest about `RECALL`. It's the share of real failures your check catches. A query that runs can still return the wrong rows. Set it from a labeled sample, and set it low if the check is "the model says it's confident."
  4. Model a DAG as its critical path. The script handles a chain. For a fan-out, price each branch as a chain that shares the root, which is one more reason the root is where frontier pays: every branch reads it.
  5. Add the mid tier when you have numbers for it. PRICE takes any number of tiers. Sonnet 5.5 at $2/$10 sits between, and its newer tokenizer produces about 30% more tokens for the same text, so scale its token counts before you compare.

A checklist before you pick tiers by hand.

  1. Draw the chain. List each step, what it reads and what reads it.
  2. Find the steps with hard checks. Executable output, schema validation, a test that runs. Those are your cascade candidates.
  3. Measure each step on both tiers. Where the gap is under 3 points, the step stays small.
  4. Put frontier on the earliest step with a large gap and no check. That's usually the plan or the first generation step.
  5. Track cost per correct task, not cost per call. All-small looks cheapest per call and often is per correct answer too. If 50% end-to-end isn't acceptable, the comparison that matters is the cheapest plan that clears your floor.

Where Nadir fits.

Nadir doesn't plan your workflow. Your agent knows which step it's on; a gateway outside the loop doesn't. What Nadir provides is the request-path part of root-first placement, which is tedious to build yourself:

With your own provider keys, Nadir charges no fee on any plan, and the provider bills you directly. Savings depend on your workload, and nothing in this post forecasts yours.

Start with a free key, or run upstream.py on one workflow first and see which step deserves the expensive model.

Conclusion.

If a workflow can afford the frontier model on one step, the Planner-as-Router table says to spend it upstream: the root step bought accuracy at a quarter of the price per point that the last step did, and the textbook cascade cost more than all-small while scoring less. The study is small and the gaps sit inside its error bars, so treat it as a direction and measure your own chain. With a 40x gap between Haiku 5.5 and Opus 5.5, a misplaced frontier call costs more than it ever has. Measure each step, cascade where a hard check exists, and put the big model where everything else reads its output.


Paper figures are from Vivek Kumar Singh, Preeti Priyam and Gautam Bhowmick, ["Planner-as-Router: Joint Plan-Time Model Routing for Cost-Efficient Multi-Agent Workflows"](https://arxiv.org/abs/2609.32917), arXiv:2609.32917, submitted September 26, 2026 and listed as accepted at AIxSET 2026, main results table excluding SQL-Compose. The per-point costs and per-1,000-task figures are our arithmetic on that table. Code and EntBench are at [github.com/vsingh45/par-entbench](https://github.com/vsingh45/par-entbench). Claude 5.5 prices are from [Anthropic's pricing page](https://platform.claude.com/docs/en/about-claude/pricing), retrieved October 31, 2026, with Haiku 5.5 at its rate for prompts up to 100K tokens. The four-step chain, its pass rates, token counts and check recall are ours, and are illustrations, not measurements.

More on routing & cascades

Pages on this site

Machine-readable summaries of this site: llms.txt and llms-full.txt.