Snorkel AI Announces Major Update to Snorkel Flow Platform
Snorkel AI, a startup formed from the Stanford AI Lab, has introduced significant enhancements to its flagship product, Snorkel Flow. This data labeling, filtering, curation, and AI fine-tuning platform now integrates directly with Google’s Gemini AI model family and Meta’s new Llama 3.
Launched in March 2022, Snorkel Flow is designed to streamline the development and deployment of custom AI solutions for enterprises. It allows organizations to automatically label, annotate, and organize both structured and unstructured documents, transforming them into reliable sources of information for various AI applications.
"Enterprises are facing challenges with off-the-shelf LLMs trained on general-purpose data from the internet," stated Alex Ratner, co-founder and CEO of Snorkel AI, during a media interview. "These models are not tailored to meet the specific needs of organizations. Snorkel Flow addresses this gap by enabling efficient data labeling and development."
For instance, if a company wanted to create an employee chatbot that provides information on internal policies, Snorkel Flow can ensure that relevant documents are labeled correctly for easy retrieval. Similarly, businesses building customer service chatbots can fine-tune the model to recognize specific product names.
Ratner explained that Snorkel AI specializes in “AI data development,” which encompasses labeling, data curation, and the refinement of data sets. "While cloud vendors offer APIs for model tuning, they don’t support the crucial task of preparing data for those APIs, which is often the most challenging part," he added.
At its launch, Snorkel Flow included features like programmatic data labeling and collaborative AI development, which have proven beneficial for enterprises such as Memorial Sloan Kettering Cancer Center and Chubb. These organizations reported improvements in AI model accuracy and efficiency by 10 to 100 times. Additionally, Snorkel has helped major banks automate regulatory compliance data labeling, reducing manual effort from six months to just 24 hours.
With the increasing prevalence of base LLMs, including powerful open-source models like Llama 3, the speed and accuracy of data labeling and curation have become vital for fine-tuning AI models, according to Ratner.
New Features in Snorkel Flow
The updated Snorkel Flow platform allows users to harness their enterprise data—now organized and labeled by Snorkel’s AI—as a reliable source of information compatible with Google Gemini and Llama 3. New integrations with Databricks Unity Catalog, Vertex AI, and Microsoft Azure Machine Learning further enhance data organization and access control for enterprises.
Moreover, Snorkel Flow now supports programmatic labeling of multimodal data, including images, to provide a comprehensive approach to AI insights. Notably, Wayfair has already benefited from the image data labeling capabilities, aiming to reduce the labeling timeline from months to days.
Enhanced Security Features
Snorkel has introduced role-based access controls (RBAC) for account administrators, enabling nuanced control over data access and utilization for AI projects. Administrators can now manage who can upload data and access connected services, similar to OpenAI's new Projects feature, but with the added flexibility of controlling access across multiple models from various vendors.
Additionally, Snorkel Flow supports on-premise and air-gapped access to foundation models, enhancing compliance and data security.
This update aligns with Snorkel’s recently launched enterprise AI accelerator, Snorkel Custom, which assists organizations through all stages of AI model evaluation, tuning, and optimization.
From Demos to Practical Value
Overall, Snorkel aims to empower enterprises to harness generative AI effectively by optimizing their data for fine-tuning models and developing AI-driven applications. "There's been immense pressure to transition from visually appealing AI demos to delivering real production value," Ratner noted.
Both Snorkel Flow and Snorkel Custom are now generally available, with pricing based on specific use cases.
Ranking
