The short answer.
We replayed 504 tool-using agent tasks against recorded Claude 5.5 outcomes. Nadir's agent router reached 92.5% task success, against 92.7% for always-Opus. It cost 79.8% as much as always-Opus and made 1 harmful downgrade.
Levanto Sage 2 sent 498 of the 504 tasks to Sonnet. In effect it ran as always-Sonnet: 89.9% success and 14 harmful downgrades. Sage 2 cost less, at 52.0% of always-Opus. But always-Sonnet with no router gets the same success for 50.9%.
On agent traffic, a router is worth having only if it finds the few tasks that need the top model. Nadir found them. Sage 2 mostly did not. These are offline replays on public benchmarks, run on 3 November 2026, and they are not a forecast for your traffic.
What we tested.
- ToolBottleneckBench. 504 tool-agent tasks in 36 task groups. Published runs give each task's success and dollar cost on Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5. Every routing decision is scored against a real recorded outcome, with no new generations.
- Same input. Each router saw the task's user message, which is what an agent sends to a router.
- Same questions, same tuning. Both routers answered the same two tier questions: a 0 to 2 difficulty score and a three-way model choice. We fitted each router's threshold on the same 249 labelled training prompts with the same rule. The rule allows at most 6% of prompts below their labelled tier and requires catching at least 65% of complex prompts, then takes the lowest cost. That gave 0.25 for Sage 2 and 0.40 for Nadir. Neither threshold was tuned on the outcomes below.
- Sage 2 was called as
levanto-sage-2through Levanto's Jev-compatible endpoint, with default settings. - A harmful downgrade is a task where the router picked a model below Opus, that model failed, and Opus would have passed. These are the failures a router causes.
Results.
| Route | Task success | Cost vs always-Opus | Harmful downgrades | Haiku / Sonnet / Opus |
|---|---|---|---|---|
| Always Opus 5.5 | 92.7% | 100% | 0 | 0 / 0 / 504 |
| Nadir | 92.5% | 79.8% | 1 | 0 / 207 / 297 |
| Levanto Sage 2 | 89.9% | 52.0% | 14 | 0 / 498 / 6 |
| Always Sonnet 5.5 | 89.9% | 50.9% | 14 | 0 / 504 / 0 |
| Always Haiku 5.5 | 85.9% | 2.6% | 36 | 504 / 0 / 0 |
| Hindsight best | 93.1% | 30.5% | 0 | 433 / 24 / 47 |
"Hindsight best" sends each task to the cheapest model that passed it, which nobody can know in advance. It shows how much room is left, and there is plenty.
Nadir leads Sage 2 on success by 2.6 points. A bootstrap over the 36 task groups gives a 95% interval of 0 to 7.5 points in Nadir's favour. That puts the gap at the edge of statistical significance.
Did the router know which tasks were hard?
The tasks that matter most to a router are the ones where Sonnet fails and Opus passes. This set has 14 of them. We ranked every task by each router's probability that it was complex. Then we measured how well that ranking separates those 14 from the rest, using AUROC: 0.5 is a coin flip and 1.0 is perfect.
| Router | AUROC, "Sonnet fails, Opus passes" |
|---|---|
| Nadir | 0.849 |
| Levanto Sage 2 | 0.584 |
With only 14 positive cases, the interval is wide. The harmful-downgrade count points the same way: 1 for Nadir, 14 for Sage 2.
Same budget, compared with a random mix.
A fair bar for any router is a random split of traffic between always-Sonnet and always-Opus at the same total cost. We swept each router's threshold across the panel and took its best success under each budget. Picking the best threshold this way uses hindsight, but both routers got the same treatment.
| Budget, share of always-Opus cost | Random mix | Sage 2 | **Nadir** |
|---|---|---|---|
| 50% | 89.8% | 89.9% | 89.9% |
| 60% | 90.4% | 89.9% | 90.5% |
| 70% | 91.0% | 90.5% | 92.5% |
| 80% | 91.5% | 90.5% | 92.5% |
At a 70 to 80% budget, Nadir beats the random mix by 1.0 to 1.5 points, while Sage 2 sits at or below it. Below 60% of always-Opus cost, neither router does much better than a random split on this panel.
SWE-bench Verified.
We also replayed 500 SWE-bench Verified issues from published mini-swe-agent runs. No Claude 5.x ladder has been published there, so this panel uses Haiku, Sonnet and Opus 4.5.
| Route | Resolved | Cost vs always-Opus | Harmful downgrades |
|---|---|---|---|
| Always Opus 4.5 | 76.8% | 100% | 0 |
| Nadir | 76.2% | 99.9% | 3 |
| Levanto Sage 2 | 72.4% | 90.0% | 33 |
| Always Sonnet 4.5 | 71.4% | 87.3% | 38 |
On this panel the gap is clear. Nadir resolved 1.4 to 6.2 points more issues than Sage 2 (95% interval). But Nadir sent 477 of the 500 issues to Opus, so it saved almost nothing. At matched cost, neither router beat a random mix. Nadir held quality on hard coding issues, and neither router found real savings in them.
Where Sage 2 did better.
Sage 2 did better on short, chat-style classification prompts. SenseBench has 4,861 word-sense classification prompts, and Sage 2 sent 2,720 of them to Haiku. It reached 94.2% success at 30.8% of always-Opus cost. Nadir's agent router reached 95.0% at 64.4%. At matched cost, Sage 2 beat a random mix by up to 0.4 points, and Nadir's agent router came out about level. Sage 2 also matched LLM-written tier labels on general chat prompts more closely.
Nadir does not use this router for that traffic. In production it serves agent requests only, and general chat goes through a separate classifier. One router for every workload is the wrong design, and this panel is one reason why.
Where the router runs.
Nadir's agent router is a model that runs on CPU next to the gateway, so a prompt is never sent to a separate vendor just to be classified. Sage 2 is a hosted API, so every prompt routed through it also goes to Levanto. For teams whose prompts must stay inside approved infrastructure, that decides it before accuracy comes into it.
Limits.
- These are offline replays of recorded outcomes on public benchmarks, not live traffic. Nadir publishes no universal production savings or quality percentage, and none of these numbers is a forecast for your workload.
- ToolBottleneckBench has only 14 tasks where routing below Opus changes the outcome. That is why the intervals above are wide.
- We tested Sage 2 through its Jev-compatible endpoint with our questions. Its native decision format, a prompt written for Sage, or its reasoning mode might do better.
- We did not compare latency under production load. In our run, Sage 2 answered in 0.3 to 0.45 seconds per round trip. Our laptop CPU run of Nadir's router was slower, and that run says nothing about the hosted service.
Check it on your own agents.
If you run coding agents, we can replay your own tasks the same way, against your recorded outcomes, before anything touches production. Talk to us.