Two products keep hiding under one word
If you are shopping for a way to stop sending every LLM request to the most expensive model, two names come up fast: vLLM Semantic Router and Nadir. They make the same promise, spend less without losing quality, and they both call themselves routers. That shared word hides the actual decision. One is an open source gateway you install and operate. The other is a managed decision layer you point your existing code at. Choosing between them is not a benchmark question. It is a build-versus-buy question, and the honest answer depends on who is going to own the thing at 2am.
This is a straight comparison: what each one is, what each one costs to run, where each one wins, and how to tell which side of the line your team is on. We are not going to pretend one of these is bad. They are both good. They are good for different teams.
What vLLM Semantic Router is
vLLM Semantic Router (VSR) is an open source project from Red Hat's CTO office and the vLLM community, Apache-2.0 licensed, that launched in September 2025 and crossed 2,000 GitHub stars within two months. Source: Red Hat, "Bringing intelligent, efficient routing to open source AI with vLLM Semantic Router". It runs as an Envoy External Processor: a gRPC filter, written in Go for the network path and Rust for the ML path, that inspects every prompt before Envoy forwards it and decides which backend model should answer. Source: Red Hat Emerging Technologies, "Intelligent inference request routing for large language models," November 11, 2025.
By its Athena release it had grown into eight neural classifiers on a 307-million-parameter mmBERT backbone (intent, jailbreak, PII, hallucination, domain, complexity, safety, and fact-check), a Boolean signal-decision engine, an HNSW semantic cache, and eleven model-selection algorithms. Source: Red Hat Developer, "Getting started with the vLLM Semantic Router project's Athena release," March 25, 2026. Its own published benchmark is strong and worth respecting: routing reasoning mode on or off per query on MMLU-Pro with Qwen3-30B cut tokens 48.5%, cut latency 47.1%, and raised accuracy 10.2%. Source: Red Hat Developer, "vLLM Semantic Router: Improving efficiency in AI reasoning," September 11, 2025. Open sourcing this category is a genuinely good outcome for the industry.
What it is not is free to operate. The documented production footprint is Kubernetes, Envoy, vLLM, Hugging Face model infrastructure, and Prometheus with Grafana, shipped through Helm charts and two custom resources (IntelligentPool and IntelligentRoute), with roughly 25GB of classifier models kept warm before it routes a single request. Source: Red Hat Developer, "Getting started with the vLLM Semantic Router project's Athena release," March 25, 2026. Open source describes the license, not the running cost. Someone on your team owns a Kubernetes-native gateway, an eight-classifier ML pipeline, and a monitoring stack, across major releases that landed roughly every two months this year. We wrote a full breakdown of that operational cost here.
What Nadir is
Nadir is a hosted decision layer that speaks the OpenAI API. You change one base URL, set the model to auto, and Nadir reads each request, picks the cheapest model that can answer it well, and returns the completion in the same call. There is no cluster to stand up, no classifier to host, and no client rewrite, because the request and response shapes are the ones your code already uses.
from openai import OpenAI
# The only change: the base URL, and model="auto"
client = OpenAI(base_url="https://api.getnadir.com/v1", api_key="YOUR_NADIR_KEY")
resp = client.chat.completions.create(
model="auto", # Nadir picks the model per request
messages=[{"role": "user", "content": "Summarize this contract clause..."}],
)Under that endpoint is a learned routing brain plus optional post-generation verification for complete non-streaming responses. Streaming bypasses the verifier. Nadir's older 60% / 98% result on 11,420 RouterBench triples was reference-assisted—the verifier had the expensive answer unavailable in production—so it is a research ceiling, not deployed performance. The corrected evidence boundary is documented here.
Around the routing decision, Nadir ships the parts VSR leaves to you: a cost and savings dashboard, per-request logs, fallback chains, spend controls, and billing. For teams with a compliance boundary, Nadir also self-hosts and runs on-prem, so keeping classification inside your own network does not force you onto a build-it-yourself gateway.
The real question is build versus buy
Both projects route. The difference is what you have to own to get there. VSR hands you a powerful, controllable substrate and asks you to operate it. Nadir hands you the outcome and operates the substrate for you.
| vLLM Semantic Router | Nadir | |
|---|---|---|
| What it is | Open source Envoy filter you self-host | Managed decision layer behind an OpenAI-compatible endpoint |
| License | Apache-2.0, free | Free with BYOK; usage-based on the hosted plan |
| Infrastructure you own | Kubernetes, Envoy, classifier hosting, Prometheus/Grafana | None; change one base URL |
| Time to first routed request | Cluster, Helm, CRDs, then warm ~25GB of models | A base URL change and model=auto |
| Routing brain | You train, host, and upgrade the classifiers | Maintained and versioned for you |
| Published quality result | Strong reasoning-mode benchmark (48.5% fewer tokens); a routing-quality target on your own traffic is yours to establish | 99.2% on checkable code; 60% / 98% RouterBench is a reference-assisted research ceiling |
| Post-generation verification | Plugins and signals you configure | Optional reference-free cascade for complete non-streaming responses; streaming bypasses it |
| Observability and billing | You assemble it | Cost dashboard, per-request logs, spend controls, billing included |
| Compliance and on-prem | In-network by design | Hosted, or self-host and on-prem when classification must stay inside your boundary |
| Best fit | Teams already operating vLLM plus Kubernetes at scale | Teams whose calls already hit hosted APIs and want routing without a platform team |
Both columns are legitimate answers to the same question. They are answers for different teams. If you are already running a vLLM fleet behind Kubernetes and Envoy, the marginal cost of adding VSR is low and full control over the classifier is a feature. If your traffic already goes to hosted frontier APIs and nobody wants to own a new Kubernetes gateway, the routing is a base URL change away.
You may not have to choose the layer
The table reads as an either/or, but the line is softer than that. Nadir's decision API can sit beside an existing gateway such as LiteLLM or an Envoy stack. Keep the carrier, begin with shadow recommendations, attach outcomes, and let the decision layer be the component you evaluate before enforcement.
The part that makes adoption auditable
Every router makes a pre-generation bet. The production question is how a team observes that bet, links it to an outcome, and decides when to enforce it. Nadir starts with a typed receipt and customer-declared baseline rather than asking the customer to trust a universal benchmark.
For complete non-streaming responses, Nadir can optionally add a reference-free verifier and escalation. For checkable code it can use deterministic tests, which is stronger evidence. VSR gives teams a broad signal and plugin framework to build their own policy. The practical difference is managed, auditable adoption versus operating and composing the routing control plane yourself—not an unconditional quality guarantee.
Which one fits your team
- Choose vLLM Semantic Router if you already operate vLLM, Kubernetes, and Envoy at scale, you need classification to stay fully in-house for compliance, and you have platform engineers who can own a gateway that shipped three major versions in five months. Full control is the payoff, and for you it is worth the operating cost.
- Choose Nadir if your traffic already hits hosted APIs, you want to shadow model-and-effort decisions without replacing your gateway, and you value decision receipts, outcome ingestion, optional execution, and self-hosted delivery.
- Consider both if you want the vLLM/Envoy control plane and a separate auditable decision contract. Keep the gateway and evaluate Nadir beside it before enforcement.
Bottom line
vLLM Semantic Router is a strong open-source control plane, and teams already living in vLLM, Kubernetes, and Envoy may reasonably choose its control and community. Nadir is the managed decision contract: shadow first, one receipt per decision, measured outcomes, optional execution, and self-hosted delivery. The trade is control-plane ownership versus an auditable service you can adopt beside what you already run.
Point your base URL at Nadir, set model=auto, and watch the savings land on the dashboard from the first request. Start free, or read the operational-cost breakdown of self-hosting vLLM Semantic Router if you are leaning toward running it yourself. If you are still deciding whether routing belongs in your stack at all, the LLM routing guide starts one level up.
The vLLM Semantic Router figures in this piece come from the project's published documentation. Nadir's 60% / 98% RouterBench result is a reference-assisted research ceiling on 11,420 triples, not deployed performance. Sources: [vLLM Semantic Router](https://github.com/vllm-project/semantic-router), [Red Hat Developer](https://developers.redhat.com/articles/2026/03/25/getting-started-vllm-semantic-router-athena-release), and [the corrected Nadir evidence note](/blog/routing-without-verification-dead-reckoning).