What is observation masking in an AI agent?

Observation masking replaces old tool outputs with a short placeholder while keeping the agent's own reasoning and actions. A 2025 JetBrains Research study (arXiv:2508.21433) found it matched LLM summarization on solve rate at about half the cost of the raw agent in SWE-agent on SWE-bench Verified.

Does context compaction always reduce LLM cost?

No. In our illustrative simulation with cache reads at a tenth of the input price, compaction barely changed cost at 40 turns, and masking on every turn cost 2.6x more than doing nothing because it invalidated the prompt cache each turn. Savings appeared on longer tasks: 25% to 37% at 100 turns.

Is LLM summarization or masking cheaper for agents?

In our model, block masking was cheaper than summarization on the agent model, because the summarizer call reads the whole history. Sending the summary step to a small model, in a hybrid, was cheapest at long task lengths. These are simulated figures, and the simulation does not measure answer quality.

When does masking stop paying off?

When the agent has to re-fetch masked output too often. In our simulated 60-turn task, block masking lost its advantage over raw at roughly a 13% re-fetch rate, and the hybrid at about 24%. Measure the re-fetch rate on your own traces.

Mask Late, Not Often

A 2025 JetBrains study found that masking old tool output matches LLM summarization at about half the raw agent's cost. It ignored the prompt cache. In our simulated 40-turn agent, masking on every turn cost 2.6x raw, block masking and raw roughly tied, and compaction only paid off on long tasks, with a break-even near a 13% re-fetch rate. Five seaborn charts, labeled illustrative, plus a seven-step framework.

Abstract.

Agent frameworks keep context from growing by either masking old tool observations or summarizing them with an LLM. A 2025 JetBrains Research study found the simple option, masking, matched summarization on solve rate at about half the cost of the raw agent. That comparison did not price the prompt cache. We simulate a 40-turn tool-using agent with cache reads at one tenth of the input price and find that, once the cache is counted, compaction barely changes cost at 40 turns, masking on every turn costs 2.6x more than doing nothing, and the benefit only appears on long tasks and only if masks are applied in blocks. The practical takeaway: decide when to compact before deciding how.

Every number and chart in this post comes from a simulation with the assumptions listed below. It is illustrative, not measured on production traffic, and says nothing about answer quality. Prices are placeholders.

Introduction.

An agent that calls a tool on every turn re-sends everything it has seen so far. Tool observations dominate that history: file reads, search results, command output. Two fixes are now standard. Observation masking replaces old tool output with a short placeholder and leaves the agent's own reasoning and actions intact. LLM summarization asks a model to rewrite the history into a shorter note.

The JetBrains paper "The Complexity Trap" ran both inside SWE-agent on SWE-bench Verified across five model configurations. It reports that masking halves cost relative to the raw agent while matching, and sometimes slightly exceeding, the solve rate of LLM summarization, and that a hybrid cuts cost a further 7% against masking and 11% against summarization (Lindenbauer et al., arXiv:2508.21433). We take that result as given. Our question is narrower and about money, not accuracy.

Research question.

When prompt caching is priced in, which context-compaction strategy actually lowers the bill for a tool-using agent, and when does it do so?

This extends earlier posts on the compaction tax and the cache-write cost of clearing tool results by putting masking, summarization and a hybrid on one cost model.

Methodology.

We simulate one agent loop and vary only the compaction policy. All parameters are assumptions.

Prompt prefix of 6,000 tokens, 250 output tokens and one 1,800-token observation per turn.
Placeholder prices per million tokens: $3.00 input, $0.30 cache read, $3.75 cache write, $15 output. A small summarizer model costs $0.25 in and $1.25 out.
A turn's cache hit is the longest prefix identical to the previous prompt. Anything after the first changed token is billed as a cache write.
Latency uses assumed prefill speeds of 8,000 tokens/s cold and 60,000 tokens/s on a hit.
Five policies: raw append-only; mask every turn (keep the last 10 observations); mask in blocks (every 10 turns); LLM summarization every 10 turns using the agent model; and a hybrid of block masking plus a small-model summary of the stub trail every 20 turns.

Not modeled: answer quality. Chart 4 prices the quality risk indirectly, as re-fetch cost.

Results.

Masking caps context, but cost follows the cache

Chart 1: Prompt size per turn over 40 turns. Raw reaches about 86,000 tokens. Mask every turn plateaus near 34,000; block masking, summarization and the hybrid sawtooth between roughly 35,000 and 50,000. Illustrative simulation.

Every policy flattens the growth that raw context shows: about 86,000 tokens at turn 40 for raw, against 34,000 to 50,000 for the others. If you pay uncached input rates, that is a large cut, and it is the regime the paper's "half the cost" lives in.

Chart 2: Simulated cost per 40-turn task. Raw $1.00, mask every turn $2.64, block mask $0.99, LLM summarization $1.16, block mask plus small-model summary $0.97. Illustrative.

With a cache, the dollars tell a different story. Raw costs $1.00 because most of its input tokens are billed at the read rate. Masking every turn edits the middle of the prompt each time, so nearly the whole prompt becomes a cache write: $2.64, with cache writes at 89% of cost. Block masking ($0.99) and the hybrid ($0.97) roughly tie raw at 40 turns. Summarization on the agent model is the worst of the sensible options at $1.16 because the summarizer call itself reads the whole history.

The savings arrive with task length

Chart 3: Simulated cost per task as task length grows from 10 to 100 turns. Raw $4.32 at 100 turns, block mask $3.24, summarization $3.48, hybrid $2.71, mask every turn $7.93. Illustrative.

Below about 20 turns, every policy that compacts either does nothing yet or loses. At 60 turns block masking is 10% below raw and the hybrid 17% below. At 100 turns they are 25% and 37% below. In this model, the cache makes append-only cheap early, and compaction wins only after context is large enough that read charges on the stale tail exceed the cost of one rewrite.

Quality risk has a price

Chart 4: Simulated cost of a 60-turn task as the share of masked observations that must be re-fetched rises from 0 to 30 percent. Block masking crosses the raw line near 13 percent; the hybrid near 24 percent; summarization stays above raw. Illustrative.

Masking is lossy. If the agent needs a masked observation again, it re-runs the tool. We model each re-fetch as one extra average turn. In this run block masking stops paying once about 13% of masked observations must be re-fetched; the hybrid tolerates about 24%. That threshold, not the headline discount, is the number to measure on your own traces.

Cache misses are latency

Chart 5: Modeled prompt-processing time per 40-turn task. Mask every turn 85.9 seconds, raw 40.0, block mask 35.9, summarization 34.4, hybrid 34.7. Illustrative.

Mask-every-turn also more than doubles modeled prefill time, 85.9 seconds against 40.0 for raw, because each turn recomputes the prompt tail. Block-style policies come in 10 to 14% under raw.

Discussion.

Compaction is not free below the knee. In our model short tasks should not compact at all.
Batch your edits. Anything that mutates old context should do so rarely and all at once. Provider features that clear tool results work this way for a reason.
Summarization is a model call and should be routed like one. It is a compression task, not reasoning. Sending it to a small model is what moves the hybrid ahead.
Masking needs a recovery path. Keep placeholders that say what was removed and how to re-run it, so a re-fetch is one cheap call.
Caches expire. If your agent idles past the cache TTL, the read discount disappears and the gap between raw and compacted widens. Our model assumes continuous work.

Implementation guide.

Instrument token usage by stage. Log input, cached input and output per call, tagged by turn.
Separate reasoning tokens from context tokens. Observations and tool schemas are context; actions are not.
Detect over-reading and duplicate retrieval. Flag observations never referenced again, and tool calls repeated with identical arguments.
Route simple steps to cheaper models. Summaries and relevance filters go to a small model.
Compress after retrieval, not blindly before. Mask only observations the agent has already acted on.
Add confidence thresholds and fallback. If the agent re-fetches a masked item twice, stop masking that tool's output.
Monitor cost, latency and answer quality together. Track re-fetch rate next to cost per resolved task.

def should_compact(turn, cached_tokens, stale_tokens, p_read=0.30, p_write=3.75):
    """Compact when per-turn read charges on the stale tail, summed over the
    turns left, exceed one rewrite of the prompt (prices per M tokens)."""
    turns_left = max(0, 100 - turn)            # your expected horizon
    keep_cost = stale_tokens * p_read / 1e6 * turns_left
    rewrite_cost = (cached_tokens - stale_tokens) * p_write / 1e6
    return keep_cost > rewrite_cost

Where Nadir fits.

This is the kind of problem model routing is built for. Instead of treating every step as a flagship-model problem, teams can classify the complexity of each step and route accordingly. Summarizing a stale observation is a classification-grade step. Nadir returns per-request cost, cached-token and routing metadata, so the re-fetch rate and the cache-miss share above can be computed from response headers rather than a new tracing stack.

Conclusion.

The future of LLM cost optimization is not only cheaper models. It is better orchestration, better routing, better retrieval discipline and better measurement. In our simulation, the choice between masking and summarizing mattered less than when and how often context was rewritten.

To see how much of your own agent spend is cache misses, estimate your savings, route a slice of agent traffic, and compare cost, latency and re-fetch rate against your baseline.

Sources: [Lindenbauer et al., "The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management," arXiv:2508.21433](https://arxiv.org/abs/2508.21433). Related: [token ledger by stage](/blog/token-ledger-stage-attribution-agent-cost-observability), [trajectory reduction](/blog/trajectory-reduction-agent-token-cost), [cache write tax](/blog/cache-write-tax-prompt-caching-cost). Simulation parameters are in the Methodology section; all figures are illustrative.