NVIDIA Launches Open Source Tool to Enhance Safety and Security of AI Models

Since March, NVIDIA has been providing AI Foundations, a service that enables businesses to train large language models (LLMs) using their proprietary data. Today, the company introduces NeMo Guardrails, a tool designed to help developers ensure their generative AI applications are accurate, appropriate, and safe.

NeMo Guardrails allows software engineers to implement three types of restrictions on their in-house LLMs. Specifically, companies can establish "topical guardrails" to prevent their applications from engaging with subjects that fall outside their training scope. For example, a customer service chatbot equipped with this software would be programmed to decline inquiries about the weather.

In addition, organizations can set safety and security measures to guarantee that their LLMs only retrieve accurate information and connect to verified safe applications. NVIDIA asserts that NeMo Guardrails is compatible with all LLMs, including ChatGPT, and emphasizes accessibility—“No need to be a machine learning expert or data scientist,” the company notes. As an open-source solution, NeMo Guardrails integrates seamlessly with the tools that enterprise developers already use.

NVIDIA is integrating NeMo Guardrails into its existing NeMo framework for building generative AI models. Business customers can access NeMo through the company’s AI Enterprise software platform or its AI Foundations service. The launch of NeMo Guardrails follows heightened scrutiny on prominent generative AIs, such as Microsoft Bing and Google Bard, concerning their tendency to “hallucinate” inaccurate information during usage.

NVIDIA developed NeMo Guardrails after several years of research, making it open source to support the developer community's focus on AI safety. "Together, our efforts on guardrails will help companies keep their intelligent services aligned with safety, privacy, and security requirements, ensuring these engines of innovation remain effective," the company states.