Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness

Summary

Researchers tested whether LLMs that cite a legal authority to justify a verdict actually depend on it. Across seven open-weight models (8B-70B) and four benchmarks, models named the correct authority in 66.7%-100% of generations, but the verdict changed when the authority was swapped in only 0.0%-21.7% on CaseHOLD, 30.0%-76.7% on ECHR and SCOTUS, and 43.3%-50.0% on ContractNLI. A red-teaming evaluation found that compliance with an adversarial instruction hidden in the case facts (73.3%-96.4%) far exceeded this verdict-swap sensitivity.