Watch the Wire

TL;DR.

Last weekend a three-minute blog post hit the Hacker News front page with 302 points and 219 comments. Its title, cleaned up for HN: "If you don't run your own LLM proxy, how do you know whats going on?" Jared Siegel's argument is that vendor dashboards show you charts, not the per-request data an engineer needs to make a decision. His one concrete example is a cache miss in a coding session: "I can pinpoint why that happened in 20 seconds by hand."

The thread argued about whether that's worth a hop in the request path. This post is the practical half of the argument.

• A cache miss is the most expensive ordinary event in an agent session. On Claude Fable 5.1, rewriting a 65,000-token prefix costs 50x what reading it from cache costs. On Sonnet 5.5 it's 25x. On Haiku 5.5, 12.5x.
• Most misses have one of five causes, and each one is visible in the request body: a model switch, a tool list change, a system prompt change, an idle gap past the TTL, or rewritten history.
• Below is wire.py, about 200 lines of standard-library Python. It sits between Claude Code and the Anthropic API, logs every request's cost, names the cause of every cache miss, and refuses requests once a session passes a dollar budget.
• In a simulated 54-request session, four avoidable misses cost $2.66 of $8.08, or 33% of the session. Each miss cost about as much as ten normal turns.

The session is simulated against a mock upstream that reproduces Anthropic's prompt-cache accounting, priced at October 2026 list rates. It's an illustration of the mechanics, not a measurement of anyone's usage, and no figure here comes from Nadir customer data.

What the thread was actually arguing about.

The post itself is short. Siegel's list of reasons to own a proxy: one endpoint for every provider and model, logs of which key called which model from which client and with what status, virtual keys so your real key isn't copied into every harness, and fast model switching. He calls the vendor analytics panels "a distraction" and OpenTelemetry export a partial answer.

The comments split three ways, and each camp had a point.

Two smaller comments matter more for cost than either. One user flagged the 5-minute cache TTL as a trap that "would destroy my 5h usage" during long test runs. Another pointed out that Claude Code already writes JSONL session files under ~/.claude/projects/, which are easier to collect than proxy traffic if you're on a subscription.

Here's how the options compare on the questions that move a bill.

The last row is the real cost of a proxy, and the commenter who raised it is right. The fix is to make the proxy small enough to read in one sitting, local, and easy to bypass: unset one environment variable and you're back on the direct path.

Why cache misses are the thing to watch.

A coding agent re-sends its whole context on every turn: tool definitions, system prompt, and the full conversation so far. Prompt caching is what makes that affordable. Anthropic caches the prefix in a fixed order, tools first, then system, then messages, and bills a cache hit at a fraction of the input rate.

When the cache misses, the whole prefix is billed again as a cache write. The gap between the two is the multiplier on every miss.

List prices per million tokens, October 2026. A 1-hour cache write bills 2x input.

Notice that the multiplier is largest on the most expensive model. Fable 5.1's cheap cache reads are why it beats some same-priced models on agent work, as we measured in Fable 5.1 vs GPT-6 Astra. That advantage only holds while the cache keeps hitting.

The five causes, in the order Anthropic's prefix is built:

1. Model changed. Caches are per model. A /model switch mid-session starts cold, which is the cache switch penalty every per-request router has to price.
2. Tool definitions changed. Tools sit at the very front of the prefix. Connecting an MCP server mid-session changes the tool list and invalidates everything after it. We covered how large those tool lists get in the MCP catalog tax.
3. System prompt changed. A timestamp, a git status line or a changed instruction file near the top of the system prompt breaks every turn after it.
4. Idle past the TTL. The default cache lives five minutes from its last use. A long test run, a code review or a coffee break ends it.
5. History rewritten. Compaction, or editing an earlier turn, changes the message prefix. That's the compaction tax.

Every one of these is visible in the request body if you keep the previous request to compare against. That's the whole trick. Vendor dashboards don't keep it, and traces don't carry it.

Tutorial: wire.py.

wire.py is a forwarding proxy for the Anthropic Messages API. It does four things:

• Forwards each request to the upstream API unchanged, streaming the response back line by line so the agent sees no difference.
• Logs each request's model, token counts and list-price cost to wire.jsonl, read from the message_start and message_delta usage events in the stream.
• Explains every full cache miss by comparing a fingerprint of the request to the session's previous one. Fingerprints hash the model, tools, system prompt and each message, with cache_control markers stripped first, because Claude Code moves those markers forward every turn while the content stays put.
• Refuses requests once a session's spend passes WIRE_BUDGET, with an HTTP 402 and an Anthropic-format billing_error body. The Anthropic SDKs retry 408, 409, 429 and 5xx responses but not a 402, so the agent surfaces the error instead of hammering the proxy.

Sessions are keyed on metadata.user_id, which current Claude Code versions fill with a per-session identifier. Clients that don't send it all land in one default session.

"""wire.py: a local Anthropic API proxy that logs cost and explains cache misses.

  python wire.py                      # listen on 127.0.0.1:8787
  ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude
  python wire.py report               # summarize wire.jsonl

Standard library only. One JSON line per request goes to wire.jsonl.
WIRE_BUDGET (dollars, default 25) caps each session: past it, requests get a 402.
"""
import hashlib
import http.client
import json
import os
import sys
import threading
import time
from collections import defaultdict
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import urlsplit

UPSTREAM = urlsplit(os.environ.get("WIRE_UPSTREAM", "https://api.anthropic.com"))
BUDGET = float(os.environ.get("WIRE_BUDGET", "25"))
LOG = os.environ.get("WIRE_LOG", "wire.jsonl")

# $ per 1M tokens: input, 5-minute cache write, cache read, output.
# A 1-hour cache write bills 2x input.
PRICES = {
    "claude-fable-5-1": (10.00, 12.50, 0.25, 50.00),
    "claude-sonnet-5-5": (2.00, 2.50, 0.10, 10.00),
    "claude-haiku-5-5": (0.10, 0.125, 0.01, 0.50),
}
HOP = {"host", "content-length", "connection", "transfer-encoding",
       "accept-encoding", "content-encoding", "keep-alive"}

sessions = {}  # session id -> last request's fingerprint, spend
lock = threading.Lock()


def strip_cc(x):
    """Drop cache_control markers. They move every turn; content doesn't."""
    if isinstance(x, dict):
        return {k: strip_cc(v) for k, v in x.items() if k != "cache_control"}
    if isinstance(x, list):
        return [strip_cc(v) for v in x]
    return x


def h(x):
    return hashlib.sha256(json.dumps(strip_cc(x), sort_keys=True).encode()).hexdigest()[:12]


def fingerprint(body):
    return {
        "model": body.get("model"),
        "system": h(body.get("system")),
        "tools": h(body.get("tools")),
        "msgs": [h(m) for m in body.get("messages", [])],
        "ttl": 3600 if '"ttl": "1h"' in json.dumps(body) else 300,
        "t": time.time(),
    }


def why_miss(prev, cur):
    if prev is None:
        return "first request in session"
    if prev["model"] != cur["model"]:
        return f"model changed {prev['model']} -> {cur['model']}"
    if prev["tools"] != cur["tools"]:
        return "tool definitions changed"
    if prev["system"] != cur["system"]:
        return "system prompt changed"
    gap = cur["t"] - prev["t"]
    if gap > prev["ttl"]:
        return f"idle {gap / 60:.0f} min, past the {prev['ttl'] // 60}-min TTL"
    if cur["msgs"][:len(prev["msgs"])] != prev["msgs"]:
        return "earlier messages rewritten (compaction or edit)"
    return "unexplained (prefix below cache minimum, or no cache_control)"


def price(model):
    return next((p for k, p in PRICES.items() if model and model.startswith(k)), (0, 0, 0, 0))


def cost(model, u):
    inp, write, read, out = price(model)
    cc = u.get("cache_creation") or {}
    w1h = cc.get("ephemeral_1h_input_tokens", 0)
    w5m = u.get("cache_creation_input_tokens", 0) - w1h
    return (u.get("input_tokens", 0) * inp + w5m * write + w1h * inp * 2
            + u.get("cache_read_input_tokens", 0) * read
            + u.get("output_tokens", 0) * out) / 1e6


class Wire(BaseHTTPRequestHandler):
    def do_POST(self):
        raw = self.rfile.read(int(self.headers.get("content-length", 0)))
        body = json.loads(raw or b"{}") if self.path.startswith("/v1/messages") else {}
        sid = str((body.get("metadata") or {}).get("user_id", "default"))
        counting = self.path.split("?")[0] == "/v1/messages"
        with lock:
            spent = sessions.get(sid, {}).get("spent", 0.0)
        if counting and spent >= BUDGET:
            return self.refuse(f"wire.py: session budget ${BUDGET:.2f} spent (${spent:.2f})")

        conn_cls = http.client.HTTPSConnection if UPSTREAM.scheme == "https" else http.client.HTTPConnection
        up = conn_cls(UPSTREAM.netloc, timeout=600)
        hdrs = {k: v for k, v in self.headers.items() if k.lower() not in HOP}
        hdrs["accept-encoding"] = "identity"
        up.request("POST", self.path, body=raw, headers=hdrs)
        resp = up.getresponse()

        self.send_response(resp.status)
        for k, v in resp.getheaders():
            if k.lower() not in HOP:
                self.send_header(k, v)
        self.send_header("connection", "close")
        self.end_headers()

        usage, model = {}, body.get("model")
        if "text/event-stream" in resp.getheader("content-type", ""):
            for line in iter(resp.readline, b""):
                self.wfile.write(line)
                self.wfile.flush()
                if not line.startswith(b"data: "):
                    continue
                ev = json.loads(line[6:])
                if ev.get("type") == "message_start":
                    model = ev["message"].get("model", model)
                    usage.update(ev["message"].get("usage", {}))
                elif ev.get("type") == "message_delta":
                    usage.update({k: v for k, v in ev.get("usage", {}).items() if v is not None})
        else:
            data = resp.read()
            self.wfile.write(data)
            try:
                msg = json.loads(data)
                usage, model = msg.get("usage", {}), msg.get("model", model)
            except ValueError:
                pass
        up.close()
        if counting and resp.status == 200:
            self.record(sid, body, model, usage)

    def record(self, sid, body, model, usage):
        cur = fingerprint(body)
        dollars = cost(model, usage)
        with lock:
            prev = sessions.get(sid, {}).get("fp")
            spent = sessions.get(sid, {}).get("spent", 0.0) + dollars
            sessions[sid] = {"fp": cur, "spent": spent}
        read = usage.get("cache_read_input_tokens", 0)
        write = usage.get("cache_creation_input_tokens", 0)
        miss = None
        if read == 0 and write >= 1024:
            miss = why_miss(prev, cur)
        row = {"ts": round(cur["t"], 3), "session": sid[-12:], "model": model,
               "input": usage.get("input_tokens", 0), "cache_write": write,
               "cache_read": read, "output": usage.get("output_tokens", 0),
               "cost": round(dollars, 6), "session_spend": round(spent, 4), "miss": miss}
        with lock, open(LOG, "a") as f:
            f.write(json.dumps(row) + "\n")
        flag = f"  MISS: {miss}" if miss else ""
        print(f"{model:<20} in {row['input']:>6} w {write:>7} r {read:>7} "
              f"out {row['output']:>5}  ${dollars:.4f}  session ${spent:.2f}{flag}", flush=True)

    def refuse(self, msg):
        out = json.dumps({"type": "error", "error": {"type": "billing_error", "message": msg}}).encode()
        self.send_response(402)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(out)))
        self.end_headers()
        self.wfile.write(out)

    def log_message(self, *args):
        pass


def report(path=LOG):
    rows = [json.loads(line) for line in open(path) if line.strip()]
    total = sum(r["cost"] for r in rows)
    print(f"{len(rows)} requests, ${total:.2f}")
    misses = defaultdict(lambda: [0, 0.0])
    for r in rows:
        if r["miss"] and not r["miss"].startswith("first"):  # cold starts are unavoidable
            reason = r["miss"].split(" -> ")[0].split(" past")[0]
            reason = {"idle": "idle past TTL", "model": "model changed"}.get(reason.split()[0], reason)
            _, write, read, _ = price(r["model"])
            # What the rewrite cost beyond reading the same tokens from cache.
            misses[reason][0] += 1
            misses[reason][1] += r["cache_write"] * (write - read) / 1e6
    tax = sum(v[1] for v in misses.values())
    print(f"cache-miss tax ${tax:.2f} ({tax / total:.1%} of spend)" if total else "")
    for reason, (n, dollars) in sorted(misses.items(), key=lambda kv: -kv[1][1]):
        print(f"  {n:>4}x  ${dollars:>7.2f}  {reason}")


if __name__ == "__main__":
    if sys.argv[1:2] == ["report"]:
        report(*sys.argv[2:3])
    else:
        print(f"wire.py on http://127.0.0.1:8787 -> {UPSTREAM.geturl()}, budget ${BUDGET}/session")
        ThreadingHTTPServer(("127.0.0.1", 8787), Wire).serve_forever()

Run it in one terminal and point Claude Code at it in another:

python wire.py
# in another terminal
ANTHROPIC_BASE_URL=http://127.0.0.1:8787 WIRE_BUDGET=8 claude

(WIRE_BUDGET is read by the proxy, so set it where wire.py runs.) The proxy forwards whatever auth headers the client sends, so it works with an API key. How a subscription login behaves behind a proxy depends on the client, and costs logged for subscription traffic are API-equivalent list prices, not what you pay.

What it caught.

We drove a simulated session through it: a 60,000-token starting context (tools plus system prompt) on Fable 5.1, 20 seconds between turns, about 2,000 tokens of new context and 1,200 output tokens per turn. Along the way the script triggered each miss on purpose: a seven-minute idle gap, an MCP tool added after turn 10, a /model switch to Sonnet 5.5 and back, and a compaction that replaced the first ten messages with a summary. Then it kept going until the budget ran out. The proxy's terminal:

claude-fable-5-1     in     30 w   61233 r       0 out  1200  $0.8257  session $0.83  MISS: first request in session
claude-fable-5-1     in     30 w     682 r   61233 out  1200  $0.0841  session $0.91
claude-fable-5-1     in     30 w     681 r   61915 out  1200  $0.0843  session $0.99
claude-fable-5-1     in     30 w     682 r   62596 out  1200  $0.0845  session $1.08
claude-fable-5-1     in     30 w     682 r   63278 out  1200  $0.0846  session $1.16
claude-fable-5-1     in     30 w     682 r   63960 out  1200  $0.0848  session $1.25
claude-fable-5-1     in     30 w   65323 r       0 out  1200  $0.8768  session $2.12  MISS: idle 7 min, past the 5-min TTL
claude-fable-5-1     in     30 w     682 r   65323 out  1200  $0.0852  session $2.21
claude-fable-5-1     in     30 w     682 r   66005 out  1200  $0.0853  session $2.30
claude-fable-5-1     in     30 w     682 r   66687 out  1200  $0.0855  session $2.38
claude-fable-5-1     in     30 w   68468 r       0 out  1200  $0.9162  session $3.30  MISS: tool definitions changed
claude-fable-5-1     in     30 w     682 r   68468 out  1200  $0.0859  session $3.38
claude-fable-5-1     in     30 w     682 r   69150 out  1200  $0.0861  session $3.47
claude-fable-5-1     in     30 w     682 r   69832 out  1200  $0.0863  session $3.56
claude-sonnet-5-5    in     30 w   71195 r       0 out  1200  $0.1900  session $3.75  MISS: model changed claude-fable-5-1 -> claude-sonnet-5-5
claude-fable-5-1     in     30 w    1363 r   70514 out  1200  $0.0950  session $3.84
claude-fable-5-1     in     30 w   69765 r       0 out  1200  $0.9324  session $4.77  MISS: earlier messages rewritten (compaction or edit)
claude-fable-5-1     in     30 w     682 r   69765 out  1200  $0.0863  session $4.86
...
claude-fable-5-1     in     30 w     681 r   93627 out  1200  $0.0922  session $7.99
claude-fable-5-1     in     30 w     682 r   94308 out  1200  $0.0924  session $8.08
HTTP 402 {"type": "error", "error": {"type": "billing_error", "message": "wire.py: session budget $8.00 spent ($8.08)"}}

A steady turn cost about $0.09. Each Fable miss cost about $0.88, roughly ten turns' worth.

Two details in that log are worth a second look. The turn after switching back from Sonnet to Fable hit the cache, because Fable's prefix was still inside its five-minute window. Switching away cost a miss; switching back was free. And the Sonnet miss itself was cheap, $0.19, because Sonnet rewrites at $2.50 per million instead of $12.50. The penalty for switching models depends on which model you switch to.

Then python wire.py report totals what the avoidable misses cost over and above reading the same tokens from cache. Cold starts are left out, since every session pays one.

54 requests, $8.08
cache-miss tax $2.66 (33.0% of spend)
     1x  $   0.85  earlier messages rewritten (compaction or edit)
     1x  $   0.84  tool definitions changed
     1x  $   0.80  idle past TTL
     1x  $   0.17  model changed

A third of the session went to four events, each of which a person caused and none of which shows up in a daily usage chart.

What to do with the log.

Each cause has a fix, and most of them are habits rather than code.

• Idle past TTL. If your sessions often sit idle for 5 to 60 minutes, the 1-hour cache TTL is worth pricing. It writes at 2x input instead of 1.25x, so it pays off when the misses it prevents cost more than the extra write premium. The log tells you how often that happens.
• Tools changed. Connect MCP servers before the session starts, not halfway through. A tool list that changes mid-session invalidates the entire prefix.
• System prompt changed. Search your system prompt and instruction files for anything that changes between turns: dates, git status, file listings. Move it to the end of the context or out of it.
• Model changed. Decide the model when the task starts. Switching mid-session pays a rewrite on the new model, and a cheap model's rewrite is cheap, so moving down costs far less than moving up.
• History rewritten. Compaction is sometimes worth a rewrite, because a shorter prefix is cheaper on every turn after it. Compact once, at a natural break, rather than repeatedly. Our cache write tax post has the break-even arithmetic.

Two limits of the script, so you know what it doesn't see. It flags full misses only (zero cache reads). A partial miss, where the cache breaks halfway through the history, shows up as a large cache_write with nonzero reads, and you can add a check for that in a few lines. And it budgets per session, in memory. A restart forgets the spend, and a fleet of agents needs a shared budget, not one per laptop.

Where Nadir fits.

Nadir is the hosted version of the same idea, with routing in the path as well as logging. Point Claude Code at it the same way:

export ANTHROPIC_BASE_URL="https://api.getnadir.com"
export ANTHROPIC_CUSTOM_HEADERS="X-Nadir-Use-Case: coding-agent
X-Nadir-Tags: team:platform"

Every response reports which model served it and what it cost, tagged with the use case and team you set, so the per-request log above exists for the whole team without anyone running a script. Governance policies set a daily dollar budget per org, team or user. Once it's spent, requests are refused with a 402 until 00:00 UTC, and the budget attaches to the person, not the key, so deleting a key doesn't reset it. That's the shared, persistent version of WIRE_BUDGET.

The routing is where the savings come from. With model="auto", Nadir sends each request to the cheapest model in your configured set that clears your quality floor. The cache arithmetic in this post applies to routing too: a router that switches models every turn pays a rewrite every turn, which is why the decision should stick for a task instead of flipping per request. With your own provider keys, Nadir charges no fee on any plan, and the provider bills you directly.

If one more hop in your agent's path is the objection, start in Observe mode: it serves exactly the model you asked for and records what routing would have chosen, so you get your own miss rate and task mix before anything changes. Savings depend on your workload, and the simulated session here is not a forecast of yours.

Start with a free key, or keep wire.py running locally and read your own log first.

Conclusion.

The HN thread framed the proxy as a question of control, and it is. But the most useful thing a proxy shows a coding-agent user is unglamorous: why the cache missed. A miss is the single most expensive ordinary event in a session, up to 50x the cost of a hit on a frontier model, and every cause is sitting in the request body. Keep the previous request, compare it to the current one, and you can name the cause in milliseconds instead of the 20 seconds Siegel spends by hand. In our simulated session, four misses were a third of the bill. Run the script for a day and find out what they are in yours.

The session in this post is simulated: a mock upstream that reproduces Anthropic's prompt-cache accounting (prefix order, per-model caches, a 5-minute TTL), driven by a script that triggers each miss on purpose. Prices are list prices as of October 2026, from our earlier posts on [Haiku 5.5](/blog/claude-haiku-5-5-pricing-100k-cliff-tokenizer-effort) and [Fable 5.1 vs Astra](/blog/same-sticker-different-bill-cache-read-pricing-fable-astra). No figure is derived from customer data. Sources: [Jared Siegel, "If you don't run your own LLM proxy, how the fuck do you even know whats going on?", October 18, 2026](https://jrdsgl.com/if-you-dont-run-your-own-llm-proxy-how-the-fuck-do-you-even-know-whats-going-on/). [Hacker News discussion, 302 points, 219 comments](https://news.ycombinator.com/item?id=50145507) (all quoted comments). [Anthropic, Prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) (prefix order, TTLs, cache write and read multipliers). [Anthropic, Streaming messages](https://docs.anthropic.com/en/docs/build-with-claude/streaming) (usage in message_start and message_delta). [Anthropic, Errors](https://docs.anthropic.com/en/api/errors) (402 billing_error).