Read It Once

TL;DR.

Document agents have an expensive habit: every question reopens the same files. An agent asked about a company's revenue reads the 10-K. Asked about its headcount an hour later, it reads the 10-K again, from the first token, at the full input rate. On FanOutQA, a benchmark of multi-hop questions over Wikipedia, a single question can pull up to a million tokens of documents into context.

A paper from Stratos Idreos's group at Harvard, the people behind database cracking, applies the database fix to agents. Agentic context cracking turns documents into structured facts as a side effect of reading them, only for the parts that questions actually touch, and answers later questions from those facts without reopening anything. Same model, same tools:

• 53% lower mean cost per question on FanOutQA, from $0.26 to $0.12, with LLM-judged accuracy at 42% against 43% (p = 0.39, not significant).
• Median cost down 3.4x, from $0.246 to $0.072, and mean prefill down from 189K to 87K tokens.
• 3x cheaper in a case study where 20 related questions came first.
• An ideal, fully pre-structured store would be 28x cheaper. Cracking is a way to get part of that without building the store up front.

It's not free. On the questions with no reuse the extraction is pure overhead: at the 90th percentile, cracking cost 1.24x the baseline. This post covers how it works, when it pays, a cost model for deciding whether your workload is a fit, and a Python tutorial that builds a minimal cracking layer on the Claude API with SQLite.

Paper figures are from arXiv 2608.31082 v2 and used Claude Haiku 4.5 with prompt caching. The break-even model and the tutorial's prices are illustrative, at October 9, 2026 list prices. None of this is Nadir customer data.

The rereading tax.

Most of what companies want agents to reason over isn't in a database. It's in filings, contracts, reports, call transcripts, PDFs and wiki pages. The standard ways to put it in front of a model each have a known failure:

• Long context loads everything and pays for everything. We priced that at $1.00 a query against $0.007 for RAG.
• Top-k RAG is cheap but brittle on questions whose evidence is spread across many documents, and it bills for chunks the model never uses.
• Agentic search lets the model search, open documents and follow leads. It's the most accurate of the three on wide questions and the most expensive, because the agent opens full documents and carries them through every later turn.

The agentic approach has a waste the others hide: it learns things and then throws them away. Once an agent has read a filing and found the revenue figure, that figure is gone when the session ends. The next question about the same filing starts from zero. On a workload where questions cluster around the same entities, which is most real ones, the agent rereads the same pages day after day.

If the facts were already in a table, those later questions would be a lookup. The paper measured this ceiling on ten sampled FanOutQA questions: with every needed fact hand-extracted into a store, answering was 28x cheaper. The catch is the "hand-extracted". Nobody can structure an enterprise document corpus in advance. Each document holds far more possible facts than any workload will ask about, and you don't know which ones matter until the questions arrive.

Borrowing from databases.

Database cracking, introduced by Idreos, Kersten and Manegold in 2007, faced the same problem with indexes. Building a full index up front is expensive and wasted on columns nobody queries. Cracking instead reorganizes data a little on every query, along the boundaries that query used. Columns that get queried a lot end up well organized. Columns nobody touches never pay anything.

Agentic context cracking, by Milad Rezaei Hajidehi, Qitong Wang and Stratos Idreos, applies that to text. There's one difference the authors point out: database cracking rearranges data that's already structured, while this creates structure where there was none.

The design has four parts.

Facts with receipts. Each extracted fact is an edge: subject, relation, object, a cardinality (single value or list), an optional unit, and the evidence, meaning the source document and the passage that supports it. Objects are entities or typed values: strings, integers, dates. Post-processing validates and normalizes each one, and only grounded facts are stored.

A catalogue in the search results. When the agent searches, each result shows which subject-relation pairs have already been extracted from that document. The agent sees "this filing has Acme Corp | revenue 2025" before deciding whether to open the file. If it does, it calls a constrained read by subject, relation or both. On a miss, it opens the raw document as before, so accuracy never depends on the store being complete.

Extraction that forks, not rereads. This is the part that makes the economics work. Extraction only happens when the agent has already opened a document for its own reasons. At that moment the document is in context and in the provider's prompt cache. A cracking sub-agent forks from that exact prefix, so it pays cache-read rates for the document rather than full input rates, and runs in parallel, off the answer path. In the authors' words, the system never opens a document only to extract structure.

Speculation, on a budget. The sub-agent extracts more than the current question needs. If the question asks for a player's career points per game, it also pulls rebounds per game, because the next question is likely to ask. It sees prior queries to guide this. Each fork gets a fixed 4,000-token decode budget and yields about 150 facts on FanOutQA. Each run must cover the whole logical document, so a list relation is never stored half-finished.

What it saved.

The authors ran FanOutQA with Claude Haiku 4.5 and prompt caching on, comparing the plain agentic baseline against the same agent with the cracking interface added. FanOutQA questions rarely revisit documents on their own, so the team generated one related question per test question (with an LLM, each verified by a human) and scored only the original test questions.

A few details matter for reading these numbers.

The savings are almost all prefill. Decode stayed under 3,000 tokens per question in every setting. That's the usual shape of agent cost: the bill is in the context the agent drags through each turn, not in what it writes. We broke down the same pattern in what agents spend before the first word.

The extraction cost is included. The fork's 4K-token decode plus its cache reads added about 12% per question on FanOutQA. The $0.12 already contains it.

The case study shows what reuse is worth. On the Hitchcock filmography set, 20 related questions ran before the 10 test questions, so the store was warm by the time the scored questions arrived. Those questions also had two to four times the fan-out. With more reuse, the gap widened from 2x to 3x.

It loses on some questions. Comparing each question against itself, cracking was 9x cheaper at the 10th percentile and 1.24x more expensive at the 90th. The paper's summary is that it pays off on three quarters of the questions. The losers are the ones where nothing extracted is ever reused: the fork ran, cost money, and saved nothing.

When cracking pays.

Strip it down and cracking is a bet: spend a little extra now, while the document is cached, so that later questions skip the document. Whether the bet wins depends on one number, the share of later questions the store can answer without opening anything.

Here's a simple model at Claude Sonnet 5.5 prices ($2 input, $10 output, $2.50 cache write, $0.10 cache read per million tokens), for questions about one 40,000-token document:

A miss in this model reopens the document and cracks it again. The first question always pays $0.162. Each later question costs $0.012 on a hit and $0.162 on a miss, against $0.118 for the baseline every time.

The break-even hit rate for a single follow-up is 29%. To also earn back the first question's extra $0.044 within ten questions, you need about a third. At 80%, ten questions cost 54% less. At 10%, they cost 26% more, which is the paper's 90th-percentile loser playing out over a whole workload.

Three things push the numbers in your favor or against you:

• Reuse is the whole game. Support agents answering about the same product docs, analysts working one company's filings, legal teams asking a contract the same twenty questions: high reuse. One-off research questions over a fresh corpus: low reuse, skip it.
• Expensive models make it pay sooner. The fork's cost scales with the model's output price and the reopen cost with its input price. The ratio between them is what matters, and it barely moves across the Sonnet and Haiku tiers. What changes is the absolute dollars: at Haiku 5.5's $0.10 rate, the same ten questions cost cents either way.
• Bigger documents make the store look better. The reopen cost grows with document length. A structured answer doesn't.

What it can't do yet.

The authors list the limits plainly, and they matter if you're thinking of building this:

• Lookups only. The agent can read facts by subject, relation or both. There's no SQL, so no counting, aggregation, joins or multi-hop queries inside the store. Those still go through documents.
• Static documents. The store assumes files don't change. In production you'd need to drop or refresh the facts from an edited document, keyed on a content hash.
• A fixed budget. Every fork gets 4,000 tokens. Larger budgets gave more facts, but narrower ones that were reused less.
• Wikipedia. The benchmarks use Wikipedia, which the model has likely seen in training, and a synthetic related-question workload. The real test is private documents with real query history, which the authors call out as the next step.

Tutorial: a minimal cracking layer.

This builds the pattern on the Claude API: a document agent with four tools, a SQLite fact store, and a fork that extracts facts from the cached prefix. It's a sketch for measuring the effect on your own documents, not the paper's system. You need pip install anthropic, an API key, a dictionary of documents and a retriever.

Step 1: check whether your workload reopens documents.

Before writing any extraction code, measure the thing cracking depends on. If you log tool calls, count how often agents open a document that some earlier session already opened:

from collections import Counter

def reopen_rate(open_events):
    """open_events: (session_id, doc_id) pairs, in time order, from your traces."""
    seen, reopens, total = set(), 0, 0
    per_doc = Counter()
    for _session, doc in open_events:
        total += 1
        per_doc[doc] += 1
        if doc in seen:
            reopens += 1
        seen.add(doc)
    print(f"{total} opens, {len(seen)} distinct docs, reopen rate {reopens / total:.0%}")
    print("most reopened:", per_doc.most_common(5))
    return reopens / total

The reopen rate is an upper bound on the hit rate. A reopen of a document only becomes a hit when the later question asks for facts the fork extracted. If this prints under 30%, the model above says cracking won't pay; stop here.

Step 2: the store.

import json, sqlite3, threading

db = sqlite3.connect("cracked.db", check_same_thread=False)
DB_LOCK = threading.Lock()
db.execute("""CREATE TABLE IF NOT EXISTS facts (
    doc TEXT, subject TEXT, relation TEXT, object TEXT, unit TEXT,
    is_list INTEGER, evidence TEXT,
    PRIMARY KEY (doc, subject, relation, object))""")

def catalogue(doc, limit=40):
    with DB_LOCK:
        rows = db.execute("SELECT DISTINCT subject, relation FROM facts WHERE doc = ? LIMIT ?",
                          (doc, limit)).fetchall()
    return [f"{s} | {r}" for s, r in rows]

def read_facts(subject=None, relation=None):
    sql, args = "SELECT subject, relation, object, unit, doc, evidence FROM facts WHERE 1=1", []
    if subject:
        sql += " AND lower(subject) = lower(?)"; args.append(subject)
    if relation:
        sql += " AND lower(relation) = lower(?)"; args.append(relation)
    cols = ("subject", "relation", "object", "unit", "doc", "evidence")
    with DB_LOCK:
        rows = db.execute(sql, args).fetchall()
    return [dict(zip(cols, row)) for row in rows]

Step 3: one tool list for the agent and the fork.

The fork only gets a cache hit if its request matches the agent's request byte for byte up to the document: same model, same system prompt, same tools, same thinking and effort settings. Changing tool_choice also invalidates the cached messages, so instead of forcing a tool in the fork we give both requests the same tools and steer the fork with its prompt.

import anthropic

client = anthropic.Anthropic()
MODEL = "claude-haiku-5-5"

SYSTEM = (
    "You answer questions over a document collection. Search first. If a search "
    "result's catalogue lists the subject and relation you need, call read_facts and "
    "answer from it. Open a document only when the facts you need are not stored. "
    "Name the source document for every fact you use."
)

TOOLS = [
    {"name": "search_docs", "description": "Find documents. Returns ids and each document's catalogue of stored subject | relation pairs.",
     "input_schema": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}},
    {"name": "read_facts", "description": "Read stored facts by subject, relation, or both.",
     "input_schema": {"type": "object", "properties": {"subject": {"type": "string"}, "relation": {"type": "string"}}}},
    {"name": "open_doc", "description": "Open a document and read its full text.",
     "input_schema": {"type": "object", "properties": {"doc": {"type": "string"}}, "required": ["doc"]}},
    {"name": "record_facts", "description": "Store facts extracted from an opened document. Only use when asked to extract.",
     "input_schema": {"type": "object", "properties": {"doc": {"type": "string"}, "facts": {"type": "array", "items": {
         "type": "object", "properties": {
             "subject": {"type": "string"}, "relation": {"type": "string"}, "object": {"type": "string"},
             "unit": {"type": "string"}, "is_list": {"type": "boolean"},
             "evidence": {"type": "string", "description": "Exact quote from the document."}},
         "required": ["subject", "relation", "object", "evidence"]}}},
         "required": ["doc", "facts"]}},
]

def call(messages, max_tokens=4000):
    # Top-level cache_control caches the request's last cacheable block, so the
    # fork, which extends the agent's prefix, reads what the agent's call wrote.
    return client.messages.create(model=MODEL, max_tokens=max_tokens, system=SYSTEM,
                                  tools=TOOLS, messages=messages,
                                  cache_control={"type": "ephemeral"})

Step 4: the fork.

The fork takes a snapshot of the conversation at the moment the document arrived, adds one instruction, and asks for facts. It keeps only facts whose evidence quote really appears in the document: a cheap grounding check that stops the model from storing things it inferred or invented. It returns how many facts were kept out of how many were proposed, which is worth logging: a low ratio means the fork is paraphrasing instead of quoting.

import copy

CRACK = (
    "Side task, not part of the answer. Extract facts from document {doc}, which you "
    "just opened, by calling record_facts once. Cover what this question needs and what "
    "similar questions would need: the same attribute for other entities in the document, "
    "and related attributes of the same entities. Recent questions: {recent}. Every fact "
    "needs an exact quote from the document as evidence. For a list relation, include "
    "every member or none."
)

def crack(snapshot, doc, text, recent, meter):
    fork = copy.deepcopy(snapshot)
    fork[-1]["content"].append({"type": "text",
                                "text": CRACK.format(doc=doc, recent="; ".join(recent[-5:]))})
    resp = call(fork, max_tokens=4000)          # the paper's fixed decode budget
    meter.add(resp, "crack")
    facts = [f for b in resp.content if b.type == "tool_use" and b.name == "record_facts"
             for f in b.input.get("facts", [])]
    grounded = [f for f in facts if f.get("evidence") and f["evidence"] in text]
    with DB_LOCK:   # forks run in threads
        db.executemany("INSERT OR IGNORE INTO facts VALUES (?,?,?,?,?,?,?)",
                       [(doc, f["subject"], f["relation"], str(f["object"]), f.get("unit"),
                         int(f.get("is_list", False)), f["evidence"]) for f in grounded])
        db.commit()
    return len(grounded), len(facts)

Step 5: the agent loop, with a meter.

# Claude Haiku 5.5, USD per million tokens, prompts up to 100K tokens.
PRICE = {"input": 0.10, "cache_write": 0.125, "cache_read": 0.01, "output": 0.50}

class Meter:
    def __init__(self):
        self.cost, self.by_kind, self.lock = 0.0, {}, threading.Lock()
    def add(self, resp, kind):
        u = resp.usage
        c = (u.input_tokens * PRICE["input"]
             + (u.cache_creation_input_tokens or 0) * PRICE["cache_write"]
             + (u.cache_read_input_tokens or 0) * PRICE["cache_read"]
             + u.output_tokens * PRICE["output"]) / 1e6
        with self.lock:
            self.cost += c
            self.by_kind[kind] = self.by_kind.get(kind, 0.0) + c

def ask(question, docs, search, meter, recent, cracking=True):
    messages = [{"role": "user", "content": question}]
    pending, forks, opened = [], [], False
    while True:
        resp = call(messages)
        meter.add(resp, "answer")
        # This call just wrote the conversation, documents included, to the cache.
        # Forks queued on the previous turn now read it instead of paying full price.
        for snap, d in pending:
            t = threading.Thread(target=crack, args=(snap, d, docs[d], recent, meter))
            t.start()
            forks.append(t)
        pending = []
        messages.append({"role": "assistant", "content": resp.content})
        if resp.stop_reason != "tool_use":
            break
        results, new_docs = [], []
        for b in resp.content:
            if b.type != "tool_use":
                continue
            if b.name == "search_docs":
                out = [{"doc": d, "catalogue": catalogue(d) if cracking else []}
                       for d in search(b.input["query"])]
                content = json.dumps(out)
            elif b.name == "read_facts":
                content = json.dumps(read_facts(b.input.get("subject"), b.input.get("relation")))
            elif b.name == "open_doc":
                opened = True
                content = docs[b.input["doc"]]
                new_docs.append(b.input["doc"])
            else:
                content = "Not available during answering."
            results.append({"type": "tool_result", "tool_use_id": b.id, "content": content})
        messages.append({"role": "user", "content": results})
        if cracking and new_docs:
            snap = copy.deepcopy(messages)   # the prefix the fork will extend
            pending += [(snap, d) for d in new_docs]
    for t in forks:   # only so the measurement sees every fact; don't wait in production
        t.join()
    recent.append(question)
    return "".join(b.text for b in resp.content if b.type == "text"), opened

The fork starts one call late on purpose. The agent's next request is what writes the document into the cache, and the fork, which extends that same prefix, reads it from there. Check usage.cache_read_input_tokens on the fork's responses. If it's near zero, something in the prefix differs, a tool list, a system prompt, a thinking setting, and you're paying full price for the document twice.

Step 6: run both arms.

def compare(questions, docs, search, check):
    """questions: in arrival order, related ones included. check(q, answer) -> bool."""
    out = {}
    for arm in ("baseline", "cracking"):
        with DB_LOCK:
            db.execute("DELETE FROM facts"); db.commit()
        meter, recent, right, opened = Meter(), [], 0, 0
        for q in questions:
            ans, did_open = ask(q, docs, search, meter, recent, cracking=(arm == "cracking"))
            right += check(q, ans)
            opened += did_open
        n = len(questions)
        out[arm] = meter.cost / n
        print(f"{arm:9s} ${meter.cost / n:.4f}/question  accuracy {right / n:.0%}  "
              f"opened a doc on {opened / n:.0%}  split {meter.by_kind}")
    print(f"cracking saves {1 - out['cracking'] / out['baseline']:.0%} per question")

The line to watch is "opened a doc on". In the cracking arm, that falls as the store warms up, and the share of questions answered without opening anything is the hit rate from the break-even model. Run the questions in the order they arrived in production. Shuffled or deduplicated sets understate reuse, and the paper's case study shows how much order matters: the same method saved 2x on one workload and 3x on another with more related questions first.

Where Nadir fits.

Cracking changes the mix of requests your agent sends. Turns that open a document are long, expensive and benefit from a capable model. Turns answered from the store are a few hundred tokens of grounded facts and a short answer, the kind of request a cheaper tier often handles correctly. That split is what a router is for.

Send model="auto" and Nadir's trained classifier reads each prompt and picks the tier from the models you've configured, against the quality floor you set on the API key. Each response reports the routing decision and its cost in the metadata, so you can see the per-question split the tutorial's meter prints, across all your traffic. If you'd rather call providers yourself, POST /v1/bucket returns simple, medium or complex with a probability for each.

One rule from this post applies to any router, Nadir included: keep the fork on the model that read the document. Prompt caches are per model. A fork routed to a different model, even a cheaper one, pays full input price for the whole document and loses the advantage that makes cracking cheap. We wrote about that trade-off in your router is fighting your prompt cache. Route the questions, pin the forks.

To see what routing would do on your agent's traffic before you change anything, Observe mode records the decision Nadir would make on every request without altering a single response. Start with a free key.

Conclusion.

Document agents pay to read the same pages again and again because what they learn dies with the session. Agentic context cracking keeps it: when an agent opens a document, a sub-agent forks from the cached prefix, extracts grounded facts on a fixed budget, and stores them where the next question can find them. On FanOutQA it cut mean cost per question by 53% at the same accuracy, and by 3x on a workload with more reuse. It's a bet on reuse, and it loses when questions don't come back to the same documents. Measure your reopen rate first. If it's high, the cheapest token in your document pipeline is the one you never send twice.

Sources: Milad Rezaei Hajidehi, Qitong Wang and Stratos Idreos, ["Agentic Context Cracking: Token-Efficient Data Reasoning Agents via Adaptive Structuring of Unstructured Data"](https://arxiv.org/abs/2608.31082), arXiv 2608.31082, v2, September 4, 2026 (Harvard University). Results from Sections 4 and 5; limitations from Section 6. Database cracking: Stratos Idreos, Martin Kersten and Stefan Manegold, "Database Cracking", CIDR 2007. Found via [Token Cost Radar](https://jcodemunch.com/radar/2026-10-07), October 7, 2026. Claude Sonnet 5.5 and Haiku 5.5 list prices as of October 9, 2026. The break-even model and tutorial are illustrative and were not run against the paper's benchmark.