Budget Bankruptcy

Abstract.

"You've hit your org's monthly spend limit." An engineer quoted that Claude error in July 2026, and noted that several teammates had hit the same wall at about the same point in the month. The Pragmatic Engineer's April survey of token budgets found a 2,000-person finance company whose $100 per-user cap ran out in "3-5 working days." Per-engineer caps are now normal. What isn't normal yet is a router that knows it's working under one. A May 2026 paper, SeqRoute (Xu, Zheng, and Wang, arXiv:2605.25424), names the failure budget bankruptcy: a router that decides every request on its own merits spends the budget early and has nothing left for the hard request that shows up later. In the paper's setup, a router trained to imitate good per-query choices went broke in 31.8% of sessions. This post walks through the paper, carries the idea from a 4-turn chat session to a 21-day engineering month, and simulates 50 engineers under a $300 cap. A budget-blind router got a third of the team locked out and produced fewer good answers than using Haiku for everything. The same router with a pacing controller got 28% more good answers than the budget-blind version, for 2.6% more spend. The tutorial is about 60 lines of Python.

All costs and charts in this post are illustrative, modeled from public list prices and the cited paper, not measured production traces. Not derived from proprietary customer data. Sources cited throughout.

The research question.

Every router we've written about answers one question per request: what's the cheapest model that clears the bar for this prompt? That's the right question when the budget is unlimited, or when someone pays whatever the bill turns out to be.

Most companies don't work that way anymore. After Uber spent its 2026 AI budget in four months, per-engineer caps became common. The Pragmatic Engineer survey found a games studio that considered $200 a month per developer "too high," and a seed-stage infra company whose per-developer spend went from $200 to $3,000 a month in six months. The cap is set in dollars per month. The router decides per request. Nothing connects the two.

We've argued before that a spending cap is not a cost cut. This post asks a narrower question: once a cap exists, how should the router behave under it, and how much does getting that wrong cost in work that never got done?

What SeqRoute found.

"SeqRoute: Global Budget-Aware Sequential LLM Routing via Offline Reinforcement Learning" (Zhongling Xu, Shunan Zheng, and Wei Wang, arXiv:2605.25424, May 2026) starts from an observation that anyone with a capped Claude Code seat will recognize. The paper puts it this way: a router that is "optimal for independent queries" may meet "an early simple question and confidently route it to the expensive 70B model. By the third turn, when the user asks a genuinely hard follow-up question, the budget is already depleted."

The setup is small and clean. Two models: Llama-3.1-8B at 1x cost and Llama-3.1-70B at 10x. Sessions of up to four turns, each with a fixed budget. Queries come from ShareGPT, WildChat, and Chatbot Arena, and answers are scored with a reward model. The authors frame routing as a finite-horizon decision problem where the remaining budget is part of the state, train it offline with Conservative Q-Learning, and use a trick they call Hindsight Budget Relabeling to turn 10,000 sessions into 2.38 million training transitions by replaying each one under different virtual budgets.

What the results table says, at an evaluation budget of 5,000:

Three things are worth taking from it.

1. Per-query intelligence doesn't prevent bankruptcy. Behavior cloning copies good individual choices and still goes broke in almost a third of sessions. That's what most routers in production are: a per-request classifier.
2. The single most important input is the remaining budget. The crude heuristic, "use 70B only if the remaining budget still covers it," takes bankruptcy to 0.1%. It isn't clever. It just looks at the balance.
3. The useful part of the learned policy is the dial. SeqRoute decides with argmax Q(s, a) − λ·cost(a). One trained policy, one λ, and you can slide along the whole cost-quality frontier at inference time without retraining. The heuristic avoids bankruptcy but spends like the Random policy. The λ-policy lets you choose where to sit.

The honest reading: you don't need offline RL to stop budget bankruptcy. You need a router that can see the balance, and a price on spending it that goes up when you're spending too fast. SeqRoute's contribution is showing that the price can be one scalar, tuned at runtime.

From a 4-turn session to a 21-day month.

A monthly engineering cap has the same structure as SeqRoute's session, stretched out. The budget is fixed at the start. Requests arrive one at a time. Some are easy, some are hard, and you don't know tomorrow's mix today. The difference is that bankruptcy on day 14 isn't a lower reward score. It's an engineer who can't use their coding agent for the rest of the month, or files a ticket to get the cap raised, which is the cap failing in a different way.

So we simulated it.

Methodology.

An analytical simulation, not a benchmark. The prices are September 2026 list ratios. The quality table is an assumption; measure yours before trusting any of it.

Five policies, all on the same requests:

• Flagship for everything. Opus 5 on every task.
• Router, budget-blind. Easy to Haiku, medium to Sonnet, hard to Opus. A good per-request router. It never looks at the balance.
• Haiku for everything. The cheapest possible policy.
• Router + downgrade at 80%. The budget-blind router until 80% of the cap is spent, then Haiku only. This is the "soft limit" many teams bolt on.
• Paced router. The same three models, choosing with quality − λ·cost, where λ is set on every request so that expected cost per task matches what's left of the budget divided by the tasks expected before month end. The tutorial below is this policy.

Finding 1: a budget-blind router goes broke mid-month.

The budget-blind router costs $0.81 per task on this mix, about $20 a working day. That's about $420 a month for an average engineer against a $300 cap. The first engineer hits the wall on day 6. By day 15, 19 of 50 are locked out. By month end, 34 of 50, and 31.9% of all requested tasks were refused.

This is the SeqRoute result at team scale. Every individual routing decision was reasonable. Hard tasks went to Opus, easy tasks went to Haiku. The router spent the budget well on the first two weeks of work and had none for the last one.

The 80% downgrade helps, but not enough: 20 engineers still hit the cap, and 19.1% of tasks are refused. It switches too late and too hard. After the switch, hard tasks go to Haiku at 0.30 quality, and the remaining 20% of budget still runs out for the heaviest users.

The paced router locks out two engineers, on day 17. They're the two heaviest users in the simulation, and Haiku for everything caps two engineers on the same day as well. Their workload doesn't fit in $300 on any model mix. That's a cap-sizing problem, not a routing one, and no router fixes it.

Finding 2: the smart router loses to the dumb one.

Score each policy by what the team actually got done:

The line to look at is the second one. A per-request router, the thing most teams build first, produced fewer good answers than sending everything to Haiku, while spending almost twice as much. Under a cap, a refused task is worth zero, and zero is worse than a mediocre Haiku answer.

The paced router gets 416, 28% more than the budget-blind router, for $7.27 more per engineer. It sends 64% of tasks to Haiku, 25% to Sonnet, and 10% to Opus. The budget-blind router's split was 36%, 45%, and 20%. The paced router didn't find better models. It found a better order: it bought fewer Opus calls early so there was money left for hard tasks at the end of the month.

Haiku for everything has the lowest cost per good answer. It also gets 0.29 quality on hard tasks, and if hard tasks are what your senior engineers do all day, that's the number that matters. The paced router gets 0.64 on hard tasks, the budget-blind router 0.61, because about a third of its hard tasks were refused.

Finding 3: pacing is a price, not a limit.

The mechanism is simple enough to explain in one paragraph. At every request, compute an allowance: money left, divided by the number of tasks you expect before the period ends. Then find the smallest λ at which your routing policy's expected cost per task fits the allowance. Route this request with that λ.

When an engineer is spending fast, the allowance drops, λ rises, easy tasks move to Haiku first, where they lose almost nothing. Then medium tasks leave Opus. Hard tasks move from Opus to Sonnet only when the allowance gets tight, and to Haiku last. When spending is slow, the allowance rises, λ falls, and the router spends more freely. No cliff at 80%. No lockout on day 6. A continuous price on budget that the router sees on every request.

That's SeqRoute's λ-sweep, done by hand. The paper learns Q(s, a); we assume a quality table per tier. The structure is the same: one scalar that turns "how hard is this prompt?" into "how hard is this prompt, given what's left?"

Tutorial: a paced router in Python.

You need three things: a difficulty tier per request, a quality estimate per tier and model, and the spend so far. The first can come from any classifier. The example uses Nadir's /v1/bucket, which returns a tier without spending provider tokens.

Step 1: price and quality tables.

MODELS = ["claude-haiku-4-5", "claude-sonnet-5", "claude-opus-5"]

# Average $ per task for YOUR workload. Take these from your own logs, not list prices.
COST = {"claude-haiku-4-5": 0.30, "claude-sonnet-5": 0.90, "claude-opus-5": 1.50}

# P(good answer) per tier. Measure on a labelled sample of your own traffic.
QUALITY = {
    "simple":  {"claude-haiku-4-5": 0.95, "claude-sonnet-5": 0.97, "claude-opus-5": 0.98},
    "medium":  {"claude-haiku-4-5": 0.70, "claude-sonnet-5": 0.92, "claude-opus-5": 0.95},
    "complex": {"claude-haiku-4-5": 0.30, "claude-sonnet-5": 0.70, "claude-opus-5": 0.90},
}

def pick(tier: str, lam: float) -> str:
    """Best quality minus lambda times cost. lam=0 buys the best model."""
    return max(MODELS, key=lambda m: QUALITY[tier][m] - lam * COST[m])

Step 2: the λ curve.

For each λ, the expected cost per task under your tier mix. This is monotone: a higher λ never picks a more expensive model.

LAMBDAS = [i * 0.005 for i in range(400)]   # 0.0 to 2.0

def expected_cost(lam: float, mix: dict[str, float]) -> float:
    return sum(p * COST[pick(tier, lam)] for tier, p in mix.items())

def lam_for(allowance: float, mix: dict[str, float]) -> float:
    """Smallest lambda whose expected cost per task fits the allowance."""
    for lam in LAMBDAS:
        if expected_cost(lam, mix) <= allowance:
            return lam
    return LAMBDAS[-1]

Step 3: the pacer.

One per budget holder: an engineer, a team, an API key. It tracks spend, request count, and the tier mix it has seen.

import time
from collections import Counter

class BudgetPacer:
    def __init__(self, budget: float, period_start: float, period_end: float,
                 expected_tasks: int, reserve: float = 0.03):
        self.budget, self.reserve = budget, reserve
        self.start, self.end = period_start, period_end
        self.prior_tasks = expected_tasks          # used until we have our own rate
        self.spent, self.n = 0.0, 0
        self.tiers = Counter({"simple": 35, "medium": 45, "complex": 20})  # prior mix

    def elapsed(self, now: float) -> float:
        return min(max((now - self.start) / (self.end - self.start), 1e-3), 1.0)

    def choose(self, tier: str, now: float | None = None) -> str:
        now = now or time.time()
        t = self.elapsed(now)
        self.n += 1
        self.tiers[tier] += 1
        rate = max(self.n / t, self.prior_tasks) if t < 0.05 else self.n / t
        tasks_left = max(rate * (1 - t), 1.0)
        allowance = ((1 - self.reserve) * self.budget - self.spent) / tasks_left
        total = sum(self.tiers.values())
        mix = {k: v / total for k, v in self.tiers.items()}
        model = pick(tier, lam_for(allowance, mix))
        # Never let one call cross the hard cap: step down instead of failing.
        for alt in reversed(MODELS):
            if COST[alt] <= COST[model] and self.spent + COST[alt] <= self.budget:
                return alt
        return model

    def record(self, actual_cost: float) -> None:
        self.spent += actual_cost

The 3% reserve keeps the last day from being a lockout by a few cents. The step-down at the end turns a would-be refusal into a cheaper answer, which in the simulation was worth more than the refusal every time.

Step 4: wire it in.

import requests
from openai import OpenAI

client = OpenAI(base_url="https://api.getnadir.com/v1", api_key=NADIR_KEY)
pacers: dict[str, BudgetPacer] = {}   # one per engineer or key

def complete(user: str, messages: list[dict]) -> str:
    tier = requests.post(
        "https://api.getnadir.com/v1/bucket",
        headers={"X-API-Key": NADIR_KEY},
        json={"messages": messages},
        timeout=2,
    ).json()["bucket"]                          # simple | medium | complex

    pacer = pacers[user]
    model = pacer.choose(tier)
    resp = client.chat.completions.create(model=model, messages=messages)
    pacer.record(cost_of(resp))                 # from your price map or the gateway's cost header
    return resp.choices[0].message.content

Use real cost, not COST[model], when you record. Coding-agent tasks vary by 10x in tokens, and the pacer only works if the balance it sees is the balance your provider will bill.

Step 5: watch λ, not just spend.

Log λ with every request. It's the most useful number on the dashboard:

• λ near zero all month: the cap is loose. You can lower it, or stop pacing.
• λ climbing steadily: the engineer is on track to hit the cap and the router is already trading quality for runway. This is the early warning a spend alert gives you on day 14.
• λ pinned at the top: everything is on Haiku and it still won't fit. Raise this person's cap or look at what they're running. No routing policy fixes a workload that is bigger than the budget.

When not to bother.

• No hard cap. If overage is billed rather than refused, the right policy is a plain per-request router with a tuned threshold. Pacing only earns its keep when running out has a cost.
• Caps that nobody hits. If your 95th-percentile engineer spends half the cap, λ stays near zero and the pacer does nothing. Keep the λ log so you'd see it change.
• Latency-critical traffic. Pacing may move a request to a cheaper model mid-month. For a customer-facing path with a quality SLA, use an SLA-constrained router and budget the path separately.
• One model. With nothing to trade down to, a budget can only be enforced by refusing. That's a cap, not a router.

Where Nadir fits.

Nadir doesn't run a budget pacer for you today. The pacer in this post is about 60 lines of your code, and the budget and its owner are your decisions. What the pacer needs from outside is the part that's hard to build well: a difficulty tier per request that doesn't itself cost tokens, and a spend figure you can trust per request.

That's what Nadir provides. POST /v1/bucket classifies each prompt as simple, medium, or complex with a confidence score and no provider call. Pass a ladder and it returns the model you'd run for that tier. Route through the OpenAI compatible gateway and every response reports the model that served it and what it cost, so pacer.record() sees real dollars. If you already run per-team chargeback, those are the same numbers.

If your team got the "monthly spend limit" message this month, start with a free key, bucket a week of your heaviest engineer's traffic, and check what share of their Opus calls were tier-simple. That share is how much runway a pacer can give them back.

Conclusion.

Per-engineer caps are the most common AI cost control of 2026, and they were designed for people, not routers. A router that decides each request alone treats the budget as someone else's problem, spends it on the first two weeks, and leaves the last week with nothing. SeqRoute put a name and a number on it: 31.8% of sessions bankrupt for a policy that makes good individual choices. At team scale, in our simulation, the budget-blind router locked out 34 of 50 engineers and did worse than Haiku alone. The fix isn't a smarter classifier. It's a price on the balance: one λ, recomputed per request from money left and time left, applied to the same routing decision you already make. The paced router refused 2.1% of tasks instead of 31.9% and got 28% more work done on the same cap.

Costs and quality figures in this post are illustrative, modeled for this post from public list prices and assumed quality rates, and are not derived from customer data. Sources: [Xu, Zheng, and Wang, "SeqRoute: Global Budget-Aware Sequential LLM Routing via Offline Reinforcement Learning," arXiv:2605.25424, May 2026](https://arxiv.org/abs/2605.25424). [Gergely Orosz, "The Pulse: token spend breaks budgets, what next?", The Pragmatic Engineer, April 2026](https://newsletter.pragmaticengineer.com/p/the-pulse-token-spend-breaks-budgets). [Daniela Baron, "How I Stopped Running out of Tokens," July 2026](https://danielabaron.me/blog/how-i-stopped-running-out-of-tokens/). [Anthropic, Pricing](https://platform.claude.com/docs/en/about-claude/pricing).