Today, Databricks announced the launch of new retrieval augmented generation (RAG) tools within its Data Intelligence Platform. These tools are designed to assist businesses in building, deploying, and maintaining high-quality large language model (LLM) applications tailored to various use cases.
Now available in public preview, these tools tackle significant challenges in developing production-ready RAG applications. They streamline the process of integrating relevant real-time business data from diverse sources with the appropriate models, while also enabling effective monitoring of applications for issues like toxicity that commonly affect LLMs.
Craig Wiley, Senior Director of Product for AI/ML at Databricks, emphasized the urgency of developing RAG apps: “Organizations find it challenging to deliver solutions that consistently produce accurate, high-quality responses while implementing guardrails to prevent undesirable outputs.”
Understanding RAG and Its Challenges
While LLMs are gaining popularity, many existing models rely on parameterized knowledge, which limits their ability to provide up-to-date and context-specific responses, particularly for internal business needs. Retrieval augmented generation (RAG) addresses this by leveraging specific data sources to enhance the accuracy and reliability of model responses. For example, a model trained on HR data can assist employees with various inquiries.
RAG involves several complex tasks, including gathering and preparing structured and unstructured data from multiple sources, model selection, prompt engineering, and ongoing monitoring. This fragmented approach often results in underperforming RAG applications.
How Databricks Is Leading the Way
Databricks’ new RAG tools integrate various processes, allowing teams to rapidly prototype and deploy quality RAG applications. Features such as vector search and feature serving eliminate the need to build cumbersome data pipelines, as structured and unstructured data from Delta tables sync seamlessly with the LLM application. This ensures access to the latest and most relevant business information for accurate and context-aware responses.
“Unity Catalog automatically tracks the lineage between offline and online datasets, simplifying the debugging of data quality issues and enforcing access control settings for better data governance,” noted Databricks' co-founder and VP of Engineering Patrick Wendell and CTO of Neural Networks Hanlin Tang.
Furthermore, developers can utilize the unified AI playground and MLFlow evaluation to evaluate models from various providers, including Azure OpenAI Service, AWS Bedrock, and open-source options like Llama 2 and MPT. This flexibility enables teams to deploy projects with the best-performing and most cost-effective models, while retaining the option to pivot to improved solutions as they become available.
Advanced Monitoring Capabilities
After deploying a RAG application, monitoring its performance at scale is crucial. Databricks offers a fully managed Lakehouse Monitoring capability that automatically scans application responses for toxicity, hallucinations, or any unsafe content. This proactive detection feeds into dashboards, alert systems, and data pipelines, allowing teams to take corrective actions swiftly. The feature integrates with model and dataset lineage, facilitating quick identification of errors and their causes.
Early Adoption Success
Although the new tools have just launched, enterprises such as RV supplier Lippert and EQT Corporation are already testing their capabilities within the Databricks Data Intelligence platform. Chris Nishnick, leading data and AI efforts at Lippert, shared, “Databricks enhances our call center operations by integrating diverse content sources into our Vector Search, ensuring agents have the knowledge they need at their fingertips. This innovative approach significantly improves efficiency and customer support.”
Internally, Databricks is also deploying RAG applications. According to Wiley, the company's IT team is piloting a RAG Slackbot for account executives and a browser plugin for sales development representatives.
Recognizing the increasing demand for specialized LLM applications, Databricks plans to invest significantly in its suite of RAG tools. The goal is to empower customers to deploy high-quality LLM applications on a large scale, with ongoing commitment to research and future innovations in this area.
Ranking
