Abstract. Every pitch for model routing describes the savings and skips the bill for the router itself. A classifier call to tag a step's complexity, a verifier call to check a cheap model's draft answer, a cache warm-up, an occasional escalation when the cheap route turns out wrong: these are real API calls, and every one of them costs its own tokens and adds its own latency. This post builds a step-level cost model of a 5-step ReAct agent and asks a narrow question: once you add up everything a routing and verification layer costs to run, does it still pay for itself, and where exactly does the payoff come from? Four configurations are compared, a naive premium-only agent, a router with no safety net, a router with compression added, and a cache-aware router with a verifier and an escalation path. The routing decision itself turns out to be nearly free, about $0.07 per 1,000 steps. The verifier and its escalation path cost roughly forty times more than that, about $2.70 per 1,000 steps, and still return the best quality-per-dollar of any configuration tested, because what it buys isn't cost reduction, it's insurance against a specific, expensive failure mode: a misrouted step that ships silently wrong. All data, pricing assumptions, and code in this post are illustrative, generated from a synthetic step-level cost model built for this analysis. Not derived from proprietary production traces. Premium-tier pricing reflects Claude Opus 4.8 ($5/$25 per million input/output tokens, Anthropic, as cited elsewhere on this blog); cheap-tier pricing ($0.25/$1.25 per million input/output tokens) is a representative small-model rate chosen to illustrate the shape of the tradeoff, not quoted from a specific vendor list. Methodology and full assumptions below. Why the router needs its own line item. This blog has measured where an agent's tokens go and which optimization levers return the most savings. Both of those analyses, and most of the routing literature this blog has covered, treat the routing decision as free: a function call that magically knows which model to use, with no cost of its own. The threshold a router uses to decide when to trust a cheap answer gets tuned. What happens when that confidence is wrong gets studied. What almost never gets measured is the far more basic question: how much does it cost to run the router itself, and does that cost show up anywhere that matters? It matters for two reasons. First, a classifier hop and a verifier hop are additional LLM calls, and additional LLM calls mean additional tokens, additional round trips, and additional latency, all of which a routing vendor's savings math tends to net out before publishing a number. Second, and less obviously, the overhead isn't the real tax. A router with no verification step is cheap to run but carries an unpriced tail risk: when it routes a step to a cheap model and gets it wrong, nothing catches that before it ships. The real cost of a routing layer is the sum of what it spends checking its own work, plus what it costs on the occasions it turns out to be wrong anyway. Research question. Given a fixed-baseline ReAct agent, what does a routing and verification layer actually cost to run, in tokens and latency, and does that overhead pay for itself once the risk of an unverified misroute is priced in rather than assumed away? Methodology. The baseline is the same 5-step ReAct research agent shape used in this blog's earlier token-anatomy work (reason, generate a search query, ingest retrieved context, read and extract from it, synthesize an answer), measured per step so the numbers are additive and reproducible. The bucket values here are a fresh model built for this analysis, not a restatement of that post's figures. Two model tiers: premium at Claude Opus 4.8's published rate ($5/$25 per million input/output tokens) and a cheap tier modeled at $0.25/$1.25 per million input/output tokens, a representative small-model rate rather than a specific vendor's number, since the point of this analysis is the shape of the tradeoff, not any one price card. Naive baseline (config A), every call on the premium tier: | Stage | Input / output tokens | Cost/step | % of baseline | |---|---:|---:|---:| | Reasoning | 850 / 220 | $0.00975 | 15.3% | | Search query generation | 300 / 40 | $0.00250 | 3.9% | | Retrieved context (ingested alongside the reading call) | 3,400 / 0 | $0.01700 | 26.7% | | Result reading / extraction | 1,200 / 260 | $0.01250 | 19.6% | | Final synthesis | 1,800 / 520 | $0.02200 | 34.5% | | Total | | $0.06375 | 100% | Baseline: $63.75 per 1,000 steps. Four configurations were built on top of this baseline, each one a superset of the last, so the marginal cost of each new piece is isolated: Config B, router-based. A classifier call (220 input / 15 output tokens, cheap tier, $0.000074/step) tags each step and routes search-query-generation and result-reading, the two lower-stakes extraction-shaped calls, to the cheap tier. Reasoning and synthesis stay on the premium tier. No verification, no compression, no caching. Config C, compression + router. Same routing as B, plus two-stage retrieval and reranking cuts the retrieved-context bucket by 60% before it reaches any model, cheap or premium. Config D, cache-aware optimized. Same as C, plus prompt caching on the stable system-prompt and tool-schema prefix inside the two premium calls (a 400-token cached prefix in reasoning, a 300-token cached prefix in synthesis, both at a 90% cache discount, consistent with Anthropic's documented cache pricing), plus a verifier hop (450 input / 25 output tokens, cheap tier) that reads each cheap-tier draft against the original question and accepts or rejects it. Rejected drafts escalate to a fresh premium-tier call for that step. The escalation rate is set at 12%, an assumed, labeled-illustrative miscalibration rate for the cheap tier on this task shape, not a measured figure. Latency and the cost-vs-quality comparison use a separate, matching illustrative model: four retrieval/reasoning pipeline shapes (sequential, parallelized retrieval, selective reading, and the fully optimized pipeline) and five whole-agent strategies (premium-only, small-model-first, hybrid routing with no verifier, compression-only, and the cache-aware verified stack), scored for quality as a percentage of always-premium output quality under the same assumptions used in this blog's elasticity analysis. All figures are illustrative and computed from the assumptions stated here, not measured from production traffic. Results. Where the naive baseline's cost actually sits. Chart 1: Token consumption by workflow stage in a naive premium-only agent. Final synthesis is the largest line at 34.5% of cost, followed by retrieved context at 26.7%. Reasoning, the step most associated with "needing a smart model," is only 15.3% of cost. Retrieved context and result reading together are 46.3%, nearly half the bill, for the mechanical work of getting information in front of the model rather than reasoning over it. That's the volume a router is actually being asked to move. The router itself is almost free. The verifier and its escalation path are not, and that's the point. Chart 2: Cost per 1,000 steps across four configurations. Naive costs $63.75, router-based $33.42, compression+router $32.91, cache-aware optimized $32.39. A zoomed panel shows the routing/verification tax: $0.07 for the router alone versus $2.70 once the verifier and expected escalation are included. | Config | Total cost/1,000 steps | Routing/verification tax | vs. baseline | |---|---:|---:|---:| | A: Naive (premium only) | $63.75 | — | — | | B: Router-based | $33.42 | $0.07 | -47.6% | | C: Compression + router | $32.91 | $0.07 | -48.4% | | D: Cache-aware optimized | $32.39 | $2.70 | -49.2% | The classifier hop that decides where to route each step costs seven cents per 1,000 steps, roughly 0.2% of the routed total. It is, for practical purposes, free. Almost all of the savings in this analysis come from what gets routed, not from the routing decision's own overhead. The verifier is a different story. Its own token cost is trivial too, but the escalation path it triggers, the 12% of cheap-tier drafts that get rejected and re-run on the premium tier, costs $2.49 per 1,000 steps by itself. Combined, D's tax is $2.70, about 40x B's. And D is still the cheapest configuration in the set. The tax is worth paying, but it is not free, and a team that assumes "adding a verifier costs almost nothing because it's a small model call" is measuring the wrong line item. The verifier's real cost is the escalations it causes, not the tokens it consumes. What an unverified router is actually betting on. Config B saves $30.33 per 1,000 steps over the naive baseline, provided every routed step lands correctly. Nothing in B catches a misroute before it ships. If an undetected wrong answer forces a full step retry, redone on the premium tier to be safe, at a rate p across steps, B's expected cost becomes $33.42 + p × $63.75. That crosses back above the $63.75 baseline once p exceeds 47.6%, an unrealistically high failure rate for a reasonably calibrated classifier. Router B is fairly robust to occasional misroutes on cost grounds alone. What it is not robust to is a misroute nobody notices. A retry-on-detection assumption is optimistic: the more common failure mode for an unverified router is a wrong answer that ships to a user or downstream system with no retry at all, because nothing flagged it as wrong. That failure doesn't show up as a cost line, it shows up as a quality regression with no dollar figure attached to it, which is exactly why it survives so many vendor cost comparisons unmeasured. Config D's verifier converts that unbounded, invisible tail risk into a visible, bounded cost: $2.70 per 1,000 steps, always paid, in exchange for catching the 12% the router alone would have shipped wrong. That's the actual trade the tax buys. Latency follows scheduling and reading discipline, not the routing decision. Chart 3: Latency across four pipeline shapes. Sequential search-then-read averages 6.8s P50 and 12.1s P95. A fully optimized pipeline reaches 2.1s P50 and 3.8s P95. None of the four pipeline shapes change which model answers the request; the gains come entirely from how retrieval and reading are scheduled. Parallelizing search and retrieval calls cuts P50 by 32% on its own. Reading only the top-ranked chunks in full, instead of every retrieved chunk, cuts it a further 26%. The routing and verification layer from section 2 adds at most one extra round trip (the classifier or verifier hop) on top of this pipeline; it is not what determines whether an agent feels fast. Verification is what lets the cheap route hold quality, not the model choice. Chart 4: Cost vs. quality across five strategies. Small-model-first drops to 61% quality at $3.19. The router+verifier+cache stack holds 97% quality at $32.39, beating both hybrid routing without a verifier and compression-only on the quality axis at a lower cost than either. | Strategy | Cost/1,000 steps | Quality vs. always-premium | |---|---:|---:| | Premium-only (always Opus) | $63.75 | 98% | | Small-model-first (always cheap tier) | $3.19 | 61% | | Hybrid routing (no verifier) | $33.42 | 90% | | Aggressive compression only | $53.55 | 95% | | Router + verifier + cache (this post) | $32.39 | 97% | Hybrid routing without a verifier is 47.6% cheaper than premium-only but gives up 8 points of quality, the cost of shipping every misroute unexamined. Adding compression, caching, the verifier, and its escalation path on top (config D) doesn't cost more than plain router-based routing, it costs $1.03 less per 1,000 steps ($32.39 vs B's $33.42), because compression's savings outweigh the verifier's tax, and it recovers nearly all of the quality gap: 97% versus 90% for the unverified router. Compression alone, with no routing at all, holds quality well (95%) because it never changes which model reasons, but it leaves most of the savings on the table because it never routes anything, and it still costs more than the fully verified stack ($53.55 vs $32.39). The verified router is the only configuration in this set that wins on cost and quality at once, and section 3 is why: it is the only one that prices in what happens when the cheap route is wrong instead of assuming it never is. Where the token waste actually is, and why routing alone doesn't touch it. Chart 5: Token waste breakdown. Irrelevant retrieved chunks are the largest category at 30%, followed by duplicate context across turns at 22%. Irrelevant retrieved chunks (30%): fetched, billed, and rarely cited in the final answer. Duplicate context across turns (22%): the same tool result appearing raw and, later, summarized. Over-reading documents (20%): full-document reads where only a fraction of the text is used. Verbose tool outputs (16%): unformatted JSON or logs passed to the model unmodified. Repeated reasoning steps (12%): re-deriving a conclusion already reached earlier in the same run. Routing a step to a cheap model doesn't shrink any of these five categories; it just changes who pays cheap-tier rates for the same wasted token volume. That's why section 4's latency gains come from a completely different lever, parallelizing and narrowing what gets read, not from which model reads it: the token volume behind sections 1 and 6 is what drives round trips and time-to-first-token, and routing alone never touches it. Cutting the waste itself is a separate lever from routing, and this blog has ranked it above routing on cost-per-effort grounds before. Routing and waste reduction solve different problems and are worth measuring, and budgeting for, separately. Discussion. The router is cheap. The thing the router is insurance against is not. The single most counter-intuitive number in this analysis is that the classifier hop costs $0.07 per 1,000 steps and the verifier's escalation path costs $2.49. A team budgeting for "the cost of adding routing" using only the classifier's own token count will underestimate the real number by roughly 40x, because the real cost isn't the decision, it's the fraction of decisions that turn out wrong and need correcting. An unverified router is not obviously unsafe on cost grounds, but it is unsafe on quality grounds. Section 3 showed B stays cheaper than the naive baseline even at a fairly high misroute-retry rate. The actual risk isn't that B gets expensive, it's that a misroute with no verifier attached doesn't trigger a retry at all; it ships. That risk doesn't appear anywhere in a cost comparison unless someone explicitly measures quality alongside dollars, which is why chart 4, not chart 2, is where the real argument for a verifier lives. Caching's return depends on how much of the overhead is genuinely stable. The cached prefixes modeled in config D (400 and 300 tokens in the two premium calls) assumed the system prompt and tool schemas don't change between calls. As this blog has found before, that assumption breaks down fast under overcaching pressure or frequent prompt edits; a team regenerating its system prompt per session will see caching's contribution in this model shrink toward zero without the rest of the analysis changing. Escalation rate is the variable that matters most and the one teams measure least. This model assumed a 12% escalation rate as a labeled illustrative input, not a measured one. The real number for any given workload determines almost the entire size of the verifier's tax: halve the escalation rate and D's tax drops from $2.70 to roughly $1.35; double it and the tax roughly doubles too. A miscalibrated router with an unmeasured confidence threshold is exactly the failure mode that inflates this number silently, which is why threshold calibration and escalation-rate monitoring aren't optional extras on top of a verified router, they're the mechanism that keeps its tax predictable. None of this argues against routing. Every configuration with routing beats the naive baseline on cost, several beat it on cost and quality together, and the tax, even at its highest measured point here (D's $2.70 against $31.36 in savings vs. the naive baseline), is under 9% of the savings routing delivers. The finding isn't "don't route." It's "measure what routing costs to run, including the corrections it triggers, before quoting its savings as a clean number." Implementation framework. Step 1: Instrument the routing and verification calls as their own cost line, separate from the model calls they gate. Most cost dashboards fold a classifier or verifier's tokens into general overhead. Tag them separately from day one; the number is small in isolation but it's the one this analysis shows gets systematically mis-estimated. Step 2: Measure the escalation rate, don't assume it. The 12% figure in this model is illustrative. A team's real escalation rate is a direct, measurable output of its verifier's accept/reject decisions and is the single biggest driver of the routing tax's size. def routing_tax(escalation_rate: float, cheap_cost: float, premium_cost: float, verifier_cost: float, classifier_cost: float) -> float: """Expected routing/verification overhead per step.""" expected_escalation = escalation_rate (premium_cost - cheap_cost) return classifier_cost + verifier_cost + expected_escalation Step 3: Separate "routing saved money" from "routing shipped a wrong answer nobody caught." These are different failure surfaces and need different monitoring. A cost dashboard alone will not show the second one; it needs a quality or accuracy signal sampled independently of the routing decision. Step 4: Compress before routing, not instead of it. Section 6 showed compression alone leaves most of the savings on the table because it never changes which model answers. Section 2 showed routing without compression still captures nearly all the savings routing has to offer. Do both, but if forced to choose one first, this model's numbers favor routing. Step 5: Set an escalation-rate ceiling, and alert on it, not just on cost. A verifier whose escalation rate creeps up is a router whose classifier is drifting out of calibration, and the routing tax will silently grow with it. Treat a rising escalation rate as an accuracy incident, not a cost incident. Step 6: Re-check the cache-hit assumption whenever prompts or tool schemas change. Caching's contribution to config D depends entirely on prefix stability. A prompt or schema edit that looks unrelated to cost can quietly zero out this lever. Step 7: Re-run this comparison whenever the escalation rate or traffic mix shifts meaningfully. The tax is a function of how often the cheap tier is wrong on a specific workload, not a fixed property of routing itself. The product angle. The number this analysis keeps coming back to, the escalation rate, is exactly the thing that's invisible without per-request routing and verification telemetry. A team running its own classifier-plus-verifier stack has to build that instrumentation itself, or discover the tax only when a spend report looks larger than the savings math predicted. This is the kind of accounting problem a routing and observability layer exists to make visible rather than something a team reconstructs after the fact. Nadir logs the classifier decision, the verifier's accept/reject outcome, and every escalation on a per-request basis, so the routing tax modeled here (classifier cost, verifier cost, expected escalation cost) is a number a team can read off its own traffic instead of estimating from a blog post's synthetic baseline. Routing to a cheaper model when a step is simple is the easy half of the problem; knowing, with real numbers, what the safety net around that decision actually costs is the half this post was built to make explicit. Conclusion. The future of LLM cost optimization is not just cheaper models routed more aggressively. It's knowing, in dollars, what the routing decision itself costs to run, and knowing that the number that matters most isn't the classifier's own tokens, it's the rate at which its decisions turn out wrong. In this model, the router was nearly free and the safety net around it was not, and the safety net was still the better trade. That's a specific, falsifiable claim about one synthetic workload, not a universal one; the escalation rate that made it true here is exactly the number every team should measure on its own traffic before trusting any routing vendor's savings claim, this post's included. Measure the tax before you trust the savings. See what your own router's escalation rate actually costs → All figures in this post are illustrative, computed from a synthetic step-level cost model built for this analysis, not measured from proprietary production traffic. Pricing: Claude Opus 4.8 at $5/M input, $25/M output (Anthropic, as cited elsewhere on this blog); cheap tier modeled at $0.25/M input, $1.25/M output as a representative small-model rate. Sources: Anthropic Prompt Caching Documentation. Anthropic Claude Pricing.*