The reliability of large language models (LLMs) is currently under investigation, as recent research delves into the ability of models like ChatGPT to generate factual and trustworthy content. A collaborative study by researchers from institutions such as Microsoft, Yale, and several Chinese universities assessed LLMs across various fields, including healthcare and finance, to gauge their reliability. The research, titled "Factuality in Large Language Models: Knowledge, Retrieval, and Domain-Specificity," highlights critical issues such as reasoning missteps and the misinterpretation of retrieved data as significant contributors to factual inaccuracies.
These inaccuracies can have serious consequences. For instance, a healthcare chatbot may supply incorrect information to a patient, or an AI finance tool could misreport stock information, leading to poor investment decisions. Such errors not only jeopardize user safety but can also inflict reputational harm on the companies that deploy these systems. A notable example occurred when Google’s Bard generated incorrect information during one of its initial demonstrations.
Another challenge impacting the reliability of LLMs identified in the study involves the use of outdated information. Many models rely on datasets with a fixed cut-off date, compelling businesses to frequently update these systems to maintain accuracy.
Researchers warn that factual inaccuracies produced by LLMs can cause significant and lasting damage. They emphasize the necessity for businesses to meticulously evaluate a model's factual reliability prior to its deployment. The study suggests utilizing evaluation techniques such as FActScore, developed by a collaboration of researchers from Meta, the University of Washington, and the Allen Institute for AI. FActScore serves as a metric for assessing the factual accuracy of the content generated by LLMs.
Additionally, the researchers advocate for the implementation of benchmarks like TruthfulQA, C-EVAL, and RealTimeQA, which can help quantify the factuality of LLM outputs. These benchmarks are generally open-source and readily available through platforms like GitHub, enabling businesses to leverage free tools for verifying their models' accuracy.
To enhance the factual accuracy of LLMs, several strategies are recommended, including continual training and retrieval augmentation. These techniques aim to improve the acquisition of long-tail knowledge within the models.
The survey also discusses the reliance of LLMs on historical training data. For instance, the basic version of OpenAI's ChatGPT was originally limited to knowledge up until September 2021, which was subsequently updated to January 2022. This dependency on outdated data risks producing misleading outputs that could hinder effective user experiences. If an AI model is based on outdated information, it may fail to make accurate predictions or could even reinforce historical biases present in the older datasets used during its training.
While there are methods to address this, such as API calls to enhance real-time information access, these solutions are not foolproof for ensuring current data usage. The paper proposes an innovative multi-agent approach, which employs multiple AI systems collaboratively or competitively to generate outputs. Researchers from MIT and Google DeepMind have introduced the concept of a "Multiagent Society," which aligns with this vision.
The benefit of a multi-agent approach is further explored in the study. By utilizing several models in tandem, researchers believe factual accuracy can be enhanced by pooling their individual strengths to tackle problems related to reasoning failures or forgotten information. Concepts such as multi-debate, where different LLMs engage in discussions to refine their answers, could significantly improve their logical and mathematical reasoning capabilities. Additionally, the multi-role fact-checking method, where distinct models generate claims or verify outputs collaboratively, can effectively identify potential inaccuracies.
Moreover, the research highlights the pitfalls of using generalized AI models for specialized fields, such as medicine. While models like ChatGPT excel in general tasks, they may lack the specific factual knowledge needed in niche sectors. In contrast, dedicated models like Harvey (for legal automation) and BloombergGPT (trained on extensive financial data) demonstrate superior accuracy when addressing domain-specific queries.
The study posits that domain-specific LLMs can substantially improve factual accuracy compared to their more generalized counterparts. It promotes methods such as continual pretraining, where models receive a steady influx of relevant domain data, as well as supervised finetuning, which enhances performance on specialized tasks via labeled datasets.
Furthermore, the paper discusses several domain-specific benchmarks that companies can utilize to assess their models, such as CMB for healthcare applications and LawBench for legal scenarios. By focusing on domain-specific training and evaluation, businesses can transform their deployments, exemplified by HuatuoGPT, a medical language model integrating ChatGPT data with input from healthcare professionals for informed clinical decision-making.
Through these strategies, researchers underscore the importance of rigorously evaluating and refining LLMs to enhance their reliability and factuality across various applications.
Ranking
