Prompt-side routing predicts before generation while outcome evidence arrives later, so Nadir can start in shadow mode and accept measured outcomes. Complete non-streaming proxy responses can optionally enter a reference-free verifier cascade, while streaming bypasses it. The old 98% RouterBench result was reference-assisted research, not deployed performance.
Published 2026-05-27 by Dor Amir on the Nadir blog.
Evidence correction, 27 August 2026: the 0.961 AUROC and 60% / 98% cascade figures in the original article came from a reference-assisted experiment. The verifier had the expensive answer unavailable to the deployed reference-free path. Production verification is optional for complete non-streaming responses; streaming bypasses it and failures fail open.
The blind spot in prompt-only routing
A model router normally looks at the prompt and predicts which model will handle it. That is useful, but it is still a pre-generation prediction. When the route is wrong, the decision layer needs an outcome signal to learn that it was wrong.
Nadir therefore starts with an auditable contract: run the decision beside the existing gateway in shadow mode, return model, effort, cache, context, policy, rationale, and economics, then attach the measured outcome. Enforce only after the policy clears the customer's gate.
What a verifier-gated cascade can add
On the managed non-streaming path, a complete candidate response can optionally enter a reference-free verifier:
The pre-classifier selects a starting tier.
The selected model returns a complete candidate answer.
The reference-free verifier scores that answer.
A reject can escalate one rung and retry.
Streaming responses bypass this stage because the verifier needs the complete response. Verifier or escalation errors fail open and retain the available answer. Those boundaries matter as much as the mechanism.
The corrected RouterBench evidence
The older experiment compared a prompt-only classifier and a cascade on 11,420 held-out RouterBench triples:
Strategy
Cost (x)
Catastrophic
Quality preserved
Always-Opus
12.0x
0%
100%
Prompt-only classifier
4.8x
3.4%
96.6%
Reference-assisted cascade
4.7x
1.7%
98.3%
Always-Haiku
1.0x
26.0%
74.0%
The verifier's AUROC 0.961 and ECE 0.016 came from supplying the expensive-model reference answer. A cost-saving production cascade does not have that reference unless it pays for the expensive call first. The table is therefore a research ceiling, not deployed accuracy, a customer forecast, or a billing baseline.
The experiment still teaches something: output evidence can improve a routing curve when the evidence available during evaluation matches what the live system actually receives. It does not prove that a reference-free verifier realizes the same improvement.
The production wedge
Lead with auditable adoption, not a universal cost or quality percentage:
Observe: shadow the decision without changing execution.
Decide: return one typed receipt for model, effort, cache, context, and policy.
Verify: optionally score complete non-streaming responses or run deterministic checks where available.
Prove: attach measured outcomes and promote only after evaluation.
Nadir exposes feedback and outcome ingestion, but a customer-specific data flywheel begins only after labelled traffic exists and survives offline evaluation and explicit promotion gates.
How to try it
For decision-only adoption, call the decision API beside your gateway and start in shadow mode. For managed execution, swap the base URL to https://api.getnadir.com and set model="auto".
The raw reports behind the old reference-assisted figures are local artifacts, not a complete checked-in evidence bundle. Read the corrected applicability notes in the RouterBench cascade benchmark.
Nadir is an LLM router. Nadir sizes every prompt and routes it to the
cheapest model that still clears your quality bar. A small decision model
reads each prompt against your model menu and picks the lowest-cost model
it expects to handle it. When the decision model is uncertain or
unavailable, the request goes to the strongest model on the menu. The
routing step makes no call to your model provider.
Nadir runs two ways. The decision API returns a model, reasoning-effort,
cache, context, and policy recommendation without calling a model
provider, beside the gateway you already run. That is how a shadow-mode
evaluation works, and its projected savings stay advisory. The OpenAI
compatible managed proxy executes the route, and migration is a two-line
change: point the base URL at api.getnadir.com and set model to auto. On
that path an optional verifier can score a complete non-streaming answer
and escalate to a stronger model when it misses the configured bar.
Streaming bypasses post-generation verification. BYOK is supported on
every tier.
For coding agents, Nadir Auto is a terminal launcher for Claude Code or
Codex that asks Nadir for the main-session model before each turn, with
your configured model as the fallback and cost ceiling. Delegation
integrations for Claude Code, Codex, or Cursor instead recommend a model
tier for subagent work, and the agent decides; the Claude plugin can also
move a main-thread turn to Haiku when Nadir's cache check prices it
cheaper, unless its switch_main_model option is off. Either way inference runs
on your own provider account, so Nadir holds no provider keys on that path.
What the numbers are, and what they are not
Nadir publishes each evaluation with its scope. On checkable code,
run-check-escalate solved 392 of 395 common HumanEval and MBPP problems
(99.2%), graded by running the canonical tests; that applies only to tasks
with runnable deterministic tests. Nadir-Tumbler posts an arena_score of
72.3 on RouterArena's public scorer, 5th of 23 routers, which measures the
routing decision on RouterArena's own model pool. A reference-assisted
RouterBench evaluation over 11,420 held-out triples produced a 60% lower
projected cost than always-Opus with about 98% retained quality and a 1.7%
catastrophic-route rate. That experiment gave the verifier the
expensive-model reference answer, which production does not have, so it
is a research ceiling and not the deployed path.
None of these is a production guarantee, a universal savings rate, or a
forecast for any particular workload. Customer savings are reported from
measured execution against a declared baseline, and customer quality only
from outcome-labelled traffic. Projected savings and realized savings are
separate artifacts and are never blended.
Design-partner program
Three rungs, picked by risk appetite. Rung 0 Shadow runs advisory decision
calls alongside live traffic and returns a projected receipt, with nothing
in the request path changed. Rung 1 Hosted is the two-line swap on a
production slice and returns a realized receipt. Rung 2 On-prem is a
supervised six-week proof of concept inside the partner's VPC, where no
prompt, response, or usage reaches Nadir. The commitments are the same at
every rung. Apply for a rung directly:
Rung 0 Shadow,
Rung 1 Hosted, or
Rung 2 On-prem.
Not sure which fits? Start at
getnadir.com/contact/?reason=design-partner.
Licensing
NadirClaw is the self-hosted core, source-available under the PolyForm
Noncommercial License. Source-available is the correct label; NadirClaw is
not open source. Nadir Route's hosted plan has no base fee and charges a
variable fee only on measured savings from requests Nadir executed.