The Verification Discount

Abstract.

Stanford RegLab tested the commercial legal research tools built specifically to solve this problem, Westlaw AI-Assisted Research, Lexis+ AI, and Practical Law AI, tools that market themselves as grounded and citation-checked, and found they still hallucinated on 17% to 33% of benchmark queries. Separately, a public tracker of court filings that cite fabricated AI output has logged 1,598 documented cases worldwide as of June 9, 2026, growing at roughly eight new cases a day, with penalties that have climbed from a typical $5,000 fine in 2023 to a $110,204.38 judgment handed down against two attorneys between December 2025 and March 2026. Outside the courtroom, a Stanford and BetterUp workplace study found that 40% of knowledge workers had received "workslop", AI output polished enough to look finished but too thin to actually help, and that resolving each incident costs an average of two hours and $186 in salary time. None of this is a model-quality story on its own. It is a checking story: in every one of these cases, an answer went out the door with nothing more critical than a person's trust standing between the model and the consequence. This post prices what the check would have cost instead.

All dollar figures beyond the cited studies are an illustrative cost model, not measured production data from any specific vendor or company.

The tools built to prevent this still don't.

The instinct, reasonably, is that purpose-built tools fix this. Retrieval-augmented legal research products exist specifically to ground answers in real case law instead of letting a general-purpose model recall citations from memory. Stanford RegLab's preregistered evaluation, published in the Journal of Empirical Legal Studies, tested three of the category leaders, Westlaw AI-Assisted Research, Lexis+ AI, and Practical Law AI, against 202 legal queries hand-scored by legal experts. Every one of them still hallucinated: Westlaw's tool on roughly a third of queries, the others on a smaller but still material share, a range the researchers report as 17% to 33% across the tools tested. That is a real improvement over general-purpose models tested the same way, a separate Stanford study of over 800,000 legal questions against 2023-era general models found hallucination rates from 58% (GPT-4) to 88% (Llama 2), but "cut the failure rate by half to two-thirds" and "solved" are not the same claim, and only one of them is what these products' marketing says.

The pattern generalizes past law: retrieval grounding, fine-tuning, and RAG all lower the base rate a model hallucinates at. None of them, so far, have driven it to zero for any deployed system tested independently. A base rate you cannot verify at the point of use is a base rate you are still exposed to, no matter how far it has already been pushed down upstream.

What an unverified answer actually costs when it ships.

Two separate literatures are now pricing this, one in courtrooms and one in ordinary knowledge work.

Damien Charlotin, a research fellow at HEC Paris, maintains a public, searchable database of court decisions addressing fabricated AI-generated legal content, filings, briefs, and citations that turned out not to exist. The database has grown from roughly 200 documented cases in mid-2025 to 1,598 by June 9, 2026, a pace of about eight new cases a day that, per the tracker, is still accelerating rather than flattening. The penalties attached to those cases have grown with them: from a typical $5,000 sanction in 2023 to $15,000-per-attorney fines paired with indefinite suspension by 2026, with the largest documented US penalty, $110,204.38, imposed across orders dated December 2025 and March 2026 in Couvrette v. Wisnovsky against attorneys Stephen Brigandi and Tim Murphy, whose briefs contained 15 nonexistent cases and 8 fabricated quotations that nobody on either side of the filing caught before it was submitted.

Most unverified AI output never reaches a courtroom, but it isn't free either. BetterUp Labs and Stanford's Social Media Lab surveyed 1,150 knowledge workers and found that 40% had received "workslop": AI-generated work, a slide deck, a report, a code change, that looks complete on first read but lacks the substance to actually move the task forward. Resolving each incident took an average of two hours, working out to roughly $186 in the receiving employee's time, before counting the trust cost: over half of respondents said receiving workslop made them see the colleague (or system) that sent it as less reliable. At 10,000 employees, the study's authors put the annualized productivity cost at over $9 million, a figure that, like the court penalties above, only exists because nobody checked the output before it moved downstream.

Methodology.

This is an illustrative cost model built on top of the cited studies' real figures, not a benchmark against any specific verifier vendor's production traffic.

The verification discount.

Put those numbers next to each other and the shape of the argument is hard to miss: a month of checking every single response a 10,000-response-a-day deployment produces costs about the same as a bag of coffee. One workslop incident that slips through unverified costs eight times that. The cheapest named failure mode in this entire post, an unresolved slide deck, still costs more than a month of catching every response before it ships.

This works because verification and generation are different problems with different price tags. Generating a good answer requires the model to search a large space and get one specific thing right. Checking a candidate answer against the prompt that produced it is a narrower task, closer to classification than generation, which is exactly why purpose-built judge models like Luna-2 can run 41x cheaper and 21x faster than the frontier model whose output they're grading, while matching it on agreement with human labels. The same asymmetry is what makes a verification pass a rounding error relative to the answer it's checking.

The harder technical question is what a verifier checks against when there's no ground-truth reference sitting nearby, which is most of production traffic. A KDD 2024 GenAI Evaluation Workshop paper, "Cost-Effective Hallucination Detection for LLMs" (Valentin, Fu, Detommaso, Xu, Zappella, and Wang), built exactly this: a pipeline that scores confidence, calibrates that score against the specific input and response it's judging, then thresholds on the calibrated value, finding that calibration, not the raw score, is what makes the decision usable in production. A separate 2025 paper, "HalluCounter" (Urlana, Kanumolu, Kumar, Garlapati, and Mishra), goes further and drops the need for any reference at all: it scores a candidate answer by its consistency with other responses to the same query and with the query itself, reporting over 90% average detection confidence across its benchmark, and because it needs no internal model states, no logits, no gradients, it works against closed-source models exactly as well as open ones. Reference-free, calibrated verification isn't a research promise anymore; it's a published, working method.

Not every answer needs the same scrutiny.

A verifier on every single token of every single response is its own kind of waste. The stakes of an answer, not the fact that a model produced it, should decide whether it gets checked.

This is the same principle routing by severity already applies inside a single code review pass: most output doesn't need the expensive treatment, but the fraction that does needs it every time, not on a sample.

Implementation framework.

Step 1: Verify the answer, not the prompt. A verifier reads the candidate response the model actually produced, alongside the prompt that produced it, and scores that specific pair. Scoring the prompt in isolation tells you nothing about whether the model's answer to it was any good.

Step 2: Default to reference-free scoring. Most production traffic has no ground-truth answer sitting nearby to check against. Consistency-based methods like HalluCounter's response-response and query-response alignment checks work without one, and work identically against closed-source and open-source models.

Step 3: Calibrate the threshold, don't just add a raw score. A confidence number with no calibration against your own traffic's distribution is not a decision. The KDD 2024 paper's central finding was that calibration, tuning what score counts as "pass" against your actual input and response patterns, is what turns a score into something you can act on.

Step 4: Escalate on failure, don't just flag it. A verifier that logs a low-confidence answer but still ships it has paid the checking cost and kept none of the benefit. Escalation, retrying with a stronger model or routing to human review, is what converts the check into a fixed answer instead of a documented miss.

Step 5: Route scrutiny by stakes, per the table above. Verify citation-bearing, side-effecting, and high-stakes answers every time. Sample or skip casual, low-consequence ones. The framework only pays for itself if the cheap majority of traffic isn't taxed at the same rate as the expensive minority.

def answer_with_verification(prompt, generator, verifier, escalation_model, threshold=0.8):
    """Generate normally, then check the specific answer against the prompt
    that produced it, reference-free. Escalate on a low-confidence result
    instead of shipping it and hoping."""
    draft = generator.generate(prompt)

    score = verifier.score(prompt=prompt, response=draft)  # reference-free,
    # consistency-based confidence, calibrated against this traffic's distribution

    if score.confidence >= threshold:
        return draft, score

    # Low-confidence answer: escalate rather than ship it unchecked.
    revised = escalation_model.generate(prompt, prior_attempt=draft, flagged_issue=score.reason)
    revised_score = verifier.score(prompt=prompt, response=revised)
    return revised, revised_score

What this doesn't solve.

A verifier is a model too, and it can be wrong in both directions: it can pass a hallucination that happens to look internally consistent, and it can flag a correct-but-unusual answer as low-confidence, sending easy traffic through an unnecessary escalation. Reference-free, consistency-based methods are also structurally weaker against a model that hallucinates the same wrong fact confidently and repeatedly across samples, since consistency alone can't distinguish "confidently right" from "confidently, consistently wrong." And verification adds a real step: a check that reads the full answer before it's considered final doesn't fit cleanly into a token-by-token streaming response, which is why a verifier gated on a complete answer, not a partial stream, is the version that's actually deployable today.

Conclusion.

The gap in every example in this post, the legal research tools, the workslop incidents, the fabricated citations, wasn't a gap in what the model knew. It was a gap in whether anything read the answer critically before a person or a court did. That gap is priced, now, in real settlements and real productivity studies, and it's priced far higher than the cost of closing it.

If you're running your own agent or application on top of an LLM API, the fix isn't a separate verification service to stand up and maintain. Point your OpenAI-compatible calls at Nadir with model=auto, and a reference-free verifier scores each complete, non-streaming response before it's returned, escalating to a stronger model when the answer doesn't clear the bar (streaming responses bypass this step today, and verifier or escalation errors fail open rather than blocking the response). The checking problem behind 1,598 documented court cases and a nine-figure productivity study isn't going away on its own. Whether your traffic pays $23 a month to catch it, or pays whatever the next unresolved incident costs, is still a design choice.

Sources: [Stanford RegLab, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools"](https://reglab.stanford.edu/publications/hallucination-free-assessing-the-reliability-of-leading-ai-legal-research-tools/). [Stanford Law School, "Hallucinating Law: Legal Mistakes with Large Language Models Are Pervasive"](https://law.stanford.edu/2024/01/11/hallucinating-law-legal-mistakes-with-large-language-models-are-pervasive/). [Damien Charlotin, AI Hallucination Cases database](https://www.damiencharlotin.com/hallucinations/). [Harvard Business Review, "AI-Generated 'Workslop' Is Destroying Productivity"](https://hbr.org/2025/09/ai-generated-workslop-is-destroying-productivity). [BetterUp, "What Is AI Workslop? Research on Costs and Solutions"](https://www.betterup.com/blog/hidden-costs-workslop). ["Cost-Effective Hallucination Detection for LLMs," GenAI Evaluation Workshop @ KDD 2024, arXiv:2407.21424](https://arxiv.org/abs/2407.21424). ["HalluCounter: Reference-free LLM Hallucination Detection in the Wild," arXiv:2503.04615](https://arxiv.org/abs/2503.04615). [Galileo, Luna-2 published benchmarks](https://galileo.ai/luna-2).