Hallucinating Law: Legal Mistakes with Large Language Models

The Stanford RegLab and Human-Centered AI Institute (HAI) produced the foundational empirical study of legal hallucination rates in large language models. Its core finding: general-purpose LLMs hallucinate legal authority in a substantial percentage of legal queries, and hallucination rates remain non-trivial even for retrieval- augmented legal AI products.

What the researchers measured

• The rate at which LLMs produced fabricated case citations, fabricated holdings, and material misstatements of statutory text.
• Variation in hallucination rates across query types (lower-court versus appellate, federal versus state, common versus niche topics).
• The performance of legal-specific retrieval-augmented tools relative to general chatbots.

Headline findings

1. General-purpose LLMs hallucinate legal authority on a majority of complex legal queries — the precise rate varies by model and prompt design, but the order of magnitude is unambiguous.
2. Retrieval-augmented legal AI tools materially reduce hallucination rates but do not eliminate them. Reported residual rates in the study remained high enough to require human verification of every output.
3. Hallucinations are worst in the contexts lawyers most need help — lower-court opinions, niche jurisdictions, and recent decisions where training data is thinnest.

Why the methodology matters

The study is widely cited because it tested against verifiable ground truth — actual case law and statutory text — rather than against human evaluator preferences. That methodological choice makes the results legally relevant: a hallucinated citation is either present in the reporter or it is not. There is no subjective interpretation.

Implications for the COUNSEL Framework

The Stanford study is the empirical foundation for COUNSEL's default posture of verification over trust. The pillar of Competence requires that every attorney understand these failure rates before deploying generative AI on a client matter. The pillar of Oversight requires that verification be built into workflow, not left to individual judgment.

Practitioner takeaways

• Treat AI output as a draft. The hallucination rate, even for legal-specific tools, is too high to permit any other posture.
• Pay extra attention in the highest-risk contexts: lower-court opinions, recent decisions, niche jurisdictions, and unsettled areas of law.
• Cite-check against primary sources, not against another AI.
• If you persuade partners or clients to adopt AI, the Stanford data is the evidence base for the verification protocol you build alongside it.

The full study and its updates are available at the source link above.