Delete What It Already Read

Abstract.

A ReAct agent does not pay once for each piece of information it reads. It pays again on every subsequent turn, because the standard agent loop re-sends the full conversation history, unmodified, as input tokens on every API call. This post models that mechanism directly: in an uncompacted 30-turn agent session, the share of input tokens spent re-billing prior turns grows from 0% at turn 1 to 82% by turn 20, even though the task itself has not gotten more complex. We call this the trajectory tax. A 2026 systems paper (AgentDiet, accepted at FSE 2026) shows this is fixable at inference time, without retraining or a smaller model, by pruning stale and superseded context from the trajectory before it is re-sent: 39.9-59.7% fewer input tokens, 21.1-35.9% lower total cost, no measured drop in task performance. This analysis builds an illustrative cost, latency, and quality model around that finding, stacks it against output compaction and prompt caching, and compares it to the default cost-cutting move: swapping to a cheaper model. The practical takeaway is that trajectory-level filtering and model routing solve different problems and are not substitutes for each other.

All charts and the 30-turn session model in this post use illustrative, synthetic data, built from public research and Anthropic's published pricing as of August 2026. They are not derived from proprietary production traces. Reduction rates are anchored to the ranges reported in the cited papers; the specific dollar and latency figures are a modeled scenario, not a benchmark result. Sources cited throughout.

The mechanism nobody names.

Most cost-optimization advice for agentic AI starts with the model: use a cheaper one, route between several, cascade from small to large. That advice is correct as far as it goes, but it treats the input to each call as fixed and only asks which model should read it. It skips a prior question: why is the input that large in the first place.

A ReAct agent (reason, act, observe, repeat) does not process a task once. It reconstructs the entire trajectory on every turn: system prompt, tool schemas, every prior tool call and its raw output, every intermediate plan the agent wrote and later abandoned, and whatever it retrieved along the way. None of that is optional context the agent could choose to skip. It is billed as input tokens on every single call, whether or not the model still needs it.

Xiao, Gao, Peng, and Xiong, "Reducing Cost of LLM Agents with Trajectory Reduction," arXiv:2509.23586, accepted at FSE 2026 name this directly: agent trajectories accumulate "useless, redundant, and expired information," and their method, AgentDiet, removes it at inference time. Across two LLMs and two benchmarks, AgentDiet cuts input tokens 39.9-59.7% and total computational cost 21.1-35.9%, while holding agent task performance at the same level as the uncompacted baseline. That last part is the finding that matters: the tokens being removed were not contributing to the answer.

Research question.

In a multi-turn ReAct agent session, how does the share of input tokens spent re-billing already-read context grow as the session gets longer, and what happens to cost, latency, and quality when that context is pruned rather than left to accumulate?

Modeling a 30-turn session.

To make the mechanism concrete, we modeled a single agent session against a fixed task profile, held constant across every turn, so the only variable is how much prior context each call carries:

• Model: Opus-class pricing, $5/M input tokens, $25/M output tokens (Anthropic, August 2026)
• Static overhead per call: 2,400 tokens (900-token system prompt + 1,500 tokens of tool schemas, resent identically every turn)
• New content per turn: 700 tokens (this turn's actual task-relevant tokens: the new tool result and the agent's reasoning about it)
• Carried history, uncompacted: grows by 650 tokens per completed turn, re-sent in full on every subsequent call
• Raw output per turn: 350 tokens of unformatted reasoning and tool-call output

At turn 1, the entire input is static overhead and the first turn's content: no history to carry yet. By turn 10, carried history is already 65% of the 8,950-token input. By turn 20, it is 82%. By turn 30, 86% of the 21,950-token input is context the model already read on a prior turn. The task did not get harder. The bill got heavier because the loop never stops re-billing its own past.

This is a different failure mode than the one usually described in "agents are expensive" posts. It is not about over-retrieval or verbose tool schemas in isolation (both real, both covered below); it is about the trajectory itself compounding, turn over turn, independent of task complexity.

Cost: trajectory reduction and output compaction stack.

Trajectory reduction targets the carried-history component specifically. It is not the only lever available. Two others are worth stacking on top of it: rewriting verbose raw tool output before it enters the trajectory at all, and caching the static prefix that gets resent unchanged every turn.

Output compaction alone is the smallest lever here because it only touches output tokens, which are a minority of the bill in a read-heavy agent loop. Some open-source tools built to rewrite verbose tool-call output into terse summaries report output-token reductions in the 60-70% range on coding-agent traces; we used 65% as the illustrative rate here. Trajectory reduction, applied to the carried-history component at roughly the midpoint of AgentDiet's reported 39.9-59.7% range, does more on its own because it targets the component that is actually growing. Layering prompt caching on the now-smaller static prefix adds a further discount at effectively zero engineering cost once the other two are in place.

Latency does not wait for the invoice.

The same mechanism that inflates cost also inflates time-to-first-token, because prefill cost scales with the number of input tokens a call carries, independent of what the model does with them.

By turn 30 in this model, the uncompacted session's per-turn latency has roughly doubled from where it started, purely from carried-history growth. Applying trajectory reduction each turn keeps that curve substantially flatter. This is a directional relationship (real prefill latency depends on batching, hardware, and provider-specific serving behavior we do not model here), but the direction is not in question: a call that carries 22,000 tokens of mostly-already-read context is slower to start generating than one that carries 4,000. Teams that treat cost and latency optimization as separate projects are often solving the same root cause twice.

Where the wasted tokens actually come from.

"Trajectory tax" is not one thing. Breaking down the overhead in the modeled session shows five distinct categories, each with a different fix:

• Verbose raw tool output (26%): unformatted JSON, stack traces, and logs passed straight from a tool call into the trajectory without summarization.
• Stale tool schemas (22%): the same tool definitions, resent byte-for-byte on every turn of the session, whether or not that tool gets called again.
• Duplicate retrieved content (20%): a document or search result that was already summarized in an earlier turn, retrieved and included again in full.
• Superseded reasoning steps (18%): intermediate plans the agent wrote, then abandoned in favor of a different approach, still sitting in the history it re-reads every turn. This is close to what Entropic Context Shaping, arXiv:2601.11585 is arguing when it says relevance ranking is the wrong filter for agent context: a plan the agent discarded can be semantically related to the task and still contribute nothing to solving it.
• Irrelevant retrieved chunks (14%): the long-documented pattern where models meaningfully use only a fraction of retrieved passages regardless of how many are provided.

Two of these five categories, stale schemas and duplicate retrieval, are close cousins of problems covered elsewhere: see MCP tool-schema overhead and over-retrieval in RAG pipelines. The other three, verbose output, superseded reasoning, and general context accumulation, are specific to the trajectory itself and do not show up in a chunk-count or schema-size audit.

Cost vs. quality: pruning is not the same move as downgrading.

The instinctive response to a high agent bill is to swap in a cheaper model. The data here argues that trajectory reduction and model downgrading are not interchangeable, because they remove different things.

A blind model downgrade cuts cost 84% and quality drops 23 points, because a weaker model is now doing the actual reasoning on the actual task. Trajectory reduction alone cuts cost 56% for a 1-point quality change, because it removes tokens the strong model was never using to produce the answer in the first place, which AgentDiet's own held-performance result across two benchmarks supports. Adding routing on top, sending only the mechanical filter-and-summarize steps (not the task reasoning) to a small model, adds more savings back at a real but much smaller quality cost than blanket downgrading. The order of operations matters: prune first, downgrade selectively second, not the reverse.

Implementation framework.

Step 1: Instrument token usage by stage. Log input tokens before and after each component is appended: system prompt, tool schemas, retrieved context, history. Most teams have never measured their own turn-10-versus-turn-1 growth curve.

import anthropic
client = anthropic.Anthropic()

def stage_tokens(system: str, history: list, tools: list, model: str) -> dict:
    base = client.messages.count_tokens(model=model, system=system, messages=[]).input_tokens
    with_tools = client.messages.count_tokens(
        model=model, system=system, messages=[], tools=tools
    ).input_tokens
    full = client.messages.count_tokens(
        model=model, system=system, messages=history, tools=tools
    ).input_tokens
    return {
        "static_overhead": with_tools,
        "carried_history": full - with_tools,
        "total_input": full,
    }

Step 2: Separate reasoning tokens from context tokens. The split that matters is what the model needs to decide the next action versus what is along for the ride. History and retrieved content are the second category, and it is where most of the overhead in this analysis lives.

Step 3: Detect over-reading and duplicate retrieval. If a chunk or tool result already appeared, summarized, in an earlier turn, re-including it in full is a duplicate-context bug, not a retrieval quality issue.

Step 4: Route simple steps to cheaper models. Chunk relevance scoring, tool-output summarization, and query generation are classification-grade tasks. They do not need the model doing the task's actual reasoning.

Step 5: Compress only after retrieval, not blindly before. Summarizing a document before you know which parts the task needs risks deleting the answer. Compress what was retrieved, after it is scoped to the current turn.

Step 6: Add confidence thresholds and fallback logic. A pruning or routing decision that is wrong on 5% of turns and silently degrades output is worse than a slower path that is consistently correct. Gate aggressive pruning behind a cheap confidence check, and fall back to the fuller context when it fails.

Step 7: Monitor cost, latency, and quality together. A cheaper model or a smaller context that causes retries can cost more than the unoptimized version. Track all three as a triad, not cost in isolation.

What this means for engineering teams.

The 82%-by-turn-20 finding is a property of how the ReAct loop is built, not a property of the task. An agent that re-sends its own trajectory unmodified is going to compound overhead regardless of which model reads it, and no model swap fixes that on its own: it just makes each unit of overhead cheaper, not smaller.

This is the layer model routing needs to sit on top of, not the layer it replaces. Blind routing without context discipline still pays to re-send stale history on every call, just at a lower per-token rate. Trajectory reduction without routing still uses a frontier model for filter-and-summarize work that a cheaper model handles correctly. Neither alone gets you the full picture in the cost-vs-quality chart above; the combination does. Nadir is built for the second half of that combination: classifying the complexity of each step, routing filter and summarization work to the minimum-cost model that preserves quality, and surfacing the savings against an always-premium baseline on a per-request basis, without requiring the trajectory-level engineering work to happen first.

Conclusion.

The future of agentic AI cost optimization is not a single lever. It is instrumenting where tokens actually go by stage, pruning the ones the model has already read and does not need again, compressing what is left after retrieval rather than before it, and routing the mechanical steps to a cheaper model while keeping the reasoning steps on the model that can do them. Trajectory reduction and model routing solve adjacent, not overlapping, problems: one shrinks what gets sent, the other decides who reads it. Teams that only do the second one are still paying the trajectory tax; they are just paying a discounted rate on it.

Data in charts is illustrative, modeled from public research and Anthropic pricing as of August 2026. Not derived from proprietary production traces. Sources: [Xiao, Gao, Peng, and Xiong, "Reducing Cost of LLM Agents with Trajectory Reduction" (AgentDiet), arXiv:2509.23586, FSE 2026](https://arxiv.org/abs/2509.23586). [Kim, "Entropic Context Shaping: Information-Theoretic Filtering for Context-Aware LLM Agents," arXiv:2601.11585](https://arxiv.org/pdf/2601.11585). [Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," arXiv:2307.03172](https://arxiv.org/abs/2307.03172). [Pinggy, "8 Open Source Tools to Slash AI Coding Agent Token Usage," 2026](https://pinggy.io/blog/tools_to_reduce_ai_coding_agent_token_usage/). [Anthropic Claude Pricing, August 2026](https://www.anthropic.com/pricing). [Anthropic Token Counting API](https://docs.anthropic.com/en/docs/build-with-claude/token-counting).