The Challenges of AI in Legal Research: A Study on Hallucinations in Legal AI Tools
Large language models (LLMs) are increasingly utilized for tasks requiring extensive information processing, with several companies developing specialized tools that leverage LLMs and information retrieval systems for legal research.
However, a recent study by Stanford University researchers reveals that, despite vendor claims, these tools still exhibit a considerable rate of "hallucinations"—outputs that are factually incorrect.
The Study Overview
This groundbreaking research is the first "preregistered empirical evaluation of AI-driven legal research tools," comparing offerings from major legal research providers to OpenAI’s GPT-4 across over 200 carefully crafted legal queries. While it was found that hallucinations were reduced compared to general-purpose chatbots, the legal AI tools still hallucinated in 17-33% of cases, indicating a troubling prevalence.
Understanding Retrieval-Augmented Generation in Legal Contexts
Many legal AI tools employ retrieval-augmented generation (RAG) techniques to mitigate hallucinations. Unlike standard LLMs, RAG systems retrieve relevant documents from a knowledge base and contextualize them for model responses. Although RAG serves as a gold standard for reducing hallucinations across various domains, legal queries often lack straightforward answers, complicating information retrieval.
The researchers point out that determining what to retrieve can be problematic, especially for novel or legally ambiguous queries. They define hallucinations as incorrect or misgrounded responses—whether factually inaccurate or contextually irrelevant.
Moreover, document relevance in law extends beyond mere text similarity, meaning that retrieving documents that appear similar but are irrelevant could impair the system's effectiveness.
Evaluating AI Tools for Legal Research
The researchers designed a diverse array of legal queries reflecting real-world research scenarios, testing three prominent AI-powered legal research tools: Lexis+ AI by LexisNexis, Westlaw AI-Assisted Research, and Ask Practical Law AI by Thomson Reuters. Although these proprietary tools utilize RAG, the study found that their performance was not without flaws, as they still exhibited a significant number of hallucinations.
The study highlighted difficulties faced by these systems in fundamental legal comprehension tasks, raising concerns about the closed nature of legal AI tools that limits transparency for legal professionals.
Advancements and Limitations of AI in Legal Research
Despite their limitations, AI-assisted legal research presents value compared to traditional keyword search methods, particularly when used as a starting point rather than the ultimate authority. According to co-author Daniel E. Ho, RAG reduces legal hallucinations compared to general AI, yet errors can still arise from inappropriate document retrieval, emphasizing that legal retrieval remains particularly complex.
The Importance of Transparency
Ho stressed the urgent need for transparency and benchmarking in legal AI. Unlike general AI research, the legal tech sector has maintained a closed approach, providing little technical information or performance evidence. This lack of transparency poses risks for lawyers relying on these tools.
In response to the study, Mike Dahn, head of Westlaw Product Management, emphasized the company's commitment to thorough testing and the complexity of legal questions, suggesting that the research's findings might reflect questions infrequently encountered in AI-assisted research.
Conversely, LexisNexis acknowledged that while no AI tool can guarantee perfection, its focus is on enhancing rather than replacing lawyer judgment. Jeff Pfeifer, Chief Product Officer at LexisNexis, indicated that the criteria used by the researchers may not adequately reflect hallucination rates and pointed to their ongoing improvements.
Looking ahead, LexisNexis and Stanford University are in discussions to establish benchmarks and performance reporting frameworks in AI’s application to legal research, aiming for enhancements that better serve legal professionals and reduce the incidence of hallucinations.
In conclusion, while AI in legal research shows promise, the challenges of hallucinations and the need for transparency remain critical issues that the industry needs to address.
Ranking
