A gateway owns the pipe, a router owns the choice. What each layer does, why most teams need both, and how to add routing beside the gateway you already run.
Questions answered on this page
What is the difference between an AI gateway and a model router?
A gateway owns the pipe. A router owns the choice. An AI gateway gives you one endpoint in front of many providers and handles keys, retries, rate limits, budgets, and spend logging; it moves the request you already decided on. A model router decides which model the request should go to in the first place, based on the prompt. They are different layers, and running a gateway does not give you routing any more than running a load balancer gives you a scheduler.
Is LiteLLM a gateway or a router?
LiteLLM is a gateway that has grown routing features. Its core job is the unified endpoint, key and budget management, spend tracking, and failover, and it is the most widely deployed open-source gateway in the category. Since v1.94.x it also ships auto_router and complexity_router, a four-tier map driven by a sub-millisecond heuristic scorer over seven hand-weighted dimensions, with an optional LLM classifier. That is real routing, and it is heuristic rather than trained, with no published accuracy on any public benchmark. Nadir runs LiteLLM inside its own backend, so this is not a claim about whether LiteLLM is good.
Do I need both an AI gateway and a model router?
Usually yes, and you almost certainly already have the gateway. Keep it. The gateway is where key management, budgets, failover, and spend attribution belong, and replacing a working one to get routing is a large migration for a small feature. Add a router only if your traffic is mixed enough in difficulty that model choice moves the bill, and add it in the position that lets you measure that before you change anything the request path depends on.
Can a model router run without taking my traffic?
Yes, if it exposes the decision separately from the execution. A decision-only call sends the prompt metadata, gets back a recommended model and effort, and makes no provider call and carries no response traffic, so your gateway still executes exactly as it does today. This is how a shadow evaluation works: run the advisory decision beside live traffic, compare what the router would have picked against what actually ran and what the outcome was, and enforce only once the evidence clears your own bar. Routers that can only run in the request path have to be trusted before they can be measured.
Does an AI gateway save money on its own?
A gateway saves money in ways that do not depend on which model you chose: negotiated or arbitraged provider prices, prompt caching passthrough, retry and failover behaviour that stops you paying twice, and the spend visibility that lets you find waste yourself. Those are real and they are cumulative with routing. What a gateway does not do is change the model that a given request lands on, so it cannot capture the difference between running an easy request on a small model and running it on a frontier one.
Where does a decision layer sit in the request path?
In front of the gateway, or beside it. In front means the router returns a model name and your gateway executes the call as usual, which keeps your keys, budgets, and logs where they are. Beside means the router is called only for the recommendation and the result is recorded rather than acted on, which is shadow mode. Running the router as the gateway, so that it both decides and executes, is a third option and it is the one that requires the most trust up front, because the first thing it changes is the path every request takes.