Databricks is elevating enterprise generative AI developer tools with significant enhancements to its Mosaic AI platform, unveiled at its annual data and AI conference led by CEO Ali Ghodsi. These upgrades aim to empower enterprises to effectively deploy applications powered by large language models (LLMs).
The Mosaic AI platform, a product of Databricks' $1.3 billion acquisition of MosaicML, enhances the company’s capabilities in generative AI, streamlining the development of AI applications. The latest features concentrate on three main areas: 1. building compound AI systems, 2. evaluating these systems across various metrics, and 3. ensuring governance throughout the entire AI pipeline.
This robust end-to-end ecosystem empowers organizations to create dependable generative AI applications that harness their data. It also positions Databricks competitively against Snowflake, which has been expanding its own generative AI offerings, including the recent launch of its enterprise-grade open LLM, “Arctic,” aimed at challenging Databricks’ DBRX.
What’s New in Databricks Mosaic AI?
Organizations enthusiastic about generative AI are eager to harness this innovative technology to develop applications utilizing their internal data. However, many teams struggle to achieve their expected return on investment from large models, often encountering challenges in delivering high-quality outputs within budgetary and privacy constraints.
To address these challenges, companies are increasingly adopting retrieval augmented generation (RAG)-based compound AI systems, which integrate various small models, retrievers, vector databases, and tools for evaluation, monitoring, security, and governance. Databricks has enhanced Mosaic AI to facilitate the creation of these advanced systems.
Recently, Databricks introduced Vector AI search as a serverless vector database integrated into its platform. The latest additions include Mosaic AI Model Training and Agent Framework. The Model Training feature allows users to finetune small, open-source foundation models through a user-friendly API or interface, optimizing knowledge for specific tasks while remaining cost-effective. Meanwhile, the Agent Framework, when combined with Mosaic AI Vector Search and Model Serving, drives high-quality RAG applications using these finetuned models.
“First, the Agent Framework simplifies the measurement and evaluation of app quality through Agent Evaluation,” explained Joel Minnick, VP of Product Marketing at Databricks. “It includes proprietary AI-assisted evaluation tools that automatically assess output quality and an intuitive tracing UI for stakeholder feedback. This allows developers to test hypotheses, iterate quickly, and redeploy applications efficiently within an end-to-end LLMOps workflow.”
The platform also features an AI Tools Catalog that enables organizations to govern, share, and register tools via the Databricks Unity Catalog, recently made open-source. This catalog enhances compound AI systems by equipping them with capabilities such as intelligent code generation, web searching, and API calls. Any registered Python or SQL function within the Unity Catalog is compatible with the Mosaic AI Tools Catalog, thereby enhancing the quality of outputs.
Enhanced Governance with Mosaic AI Gateway
To ensure robust governance and trust in AI applications, Databricks introduces the "Mosaic AI Gateway." This tool provides a unified interface for querying, managing, and deploying both open-source and proprietary models, allowing teams to swap LLMs without extensive application code modifications.
Crucially, the AI Gateway is equipped with built-in governance and monitoring features. It enables usage tracking and guardrails, allowing organizations to monitor model usage and establish rate limits to control costs while filtering for safety and personally identifiable information.
All new offerings for Mosaic AI, except the AI Tools Catalog, are currently in public preview and are set to be generally available in the coming months. The AI Tools Catalog is in private preview, with no announced timeline for broader release. Additionally, Databricks unveiled several other noteworthy products at the event, including Databricks AI/BI for generative AI-powered analytics, Databricks LakeFlow for data engineering, and a specialized image generation model developed in partnership with Shutterstock.
The Databricks Data and AI Summit runs from June 10 to June 13, 2024.
Ranking
