From Data Storage to Intelligent Data Engines: The Evolution of VAST Data's AI Operating System

At the Transform 2024 conference, VAST Data Founder and CEO Renen Hallak shared insights into the company’s vision for AI infrastructure, shedding light on the future of enterprise AI systems.

Hallak unveiled VAST Data's concept of a global operating system for AI, aimed at simplifying the complexities of data management and AI deployment across different regions and organizations. This system is built on three core components: the VAST Data Store, the VAST Database, and the VAST Data Engine.

VAST Data has made notable progress in AI infrastructure, recently raising $118 million in a Series E funding round led by Fidelity Management & Research Company in December 2023. This investment boosted VAST Data’s valuation to $9.1 billion, nearly tripling its valuation from $3.7 billion since 2021.

The VAST Data Store effectively manages unstructured data storage, offering both file and object access for large-scale information from diverse sources, including images, videos, audio, and genomic data. As Hallak noted, “It gives you file access, object access, a lot of large pieces of information… natural information that comes from the natural world.”

Complementing this, the VAST Database enables SQL querying of metadata generated from AI inferences on the stored data, allowing organizations to efficiently extract valuable insights from vast data repositories.

The VAST Data Engine activates the system, triggering functions based on incoming data. Hallak illustrated this with an example: “A genomics file comes in, we run it through that inference function to understand which genes are in which mutations, which then triggers more functions as we deepen our understanding of the underlying natural universe.”

This integrated approach addresses a key issue identified in a recent media analysis of the AI tech stack: the demand for comprehensive, end-to-end solutions that enhance AI infrastructure and streamline operations. VAST Data’s global operating system aims to provide a cohesive platform managing data, AI processing, and analytics across various environments.

Hallak stressed the significance of vertical integration in this system, which facilitates intelligent scheduling based on time and space constraints. He explained, “If you have data centers across the world, you don’t want to be moving that information across the ocean. You want to schedule those serverless functions close to where their data is.”

This capability aligns with the rising trend of semantic layers and data fabrics in enterprise AI infrastructure. By establishing a unified namespace across geographies, VAST Data’s system seeks to simplify data access and processing, potentially unlocking new AI use cases.

To address concerns about data quality and governance, Hallak highlighted that VAST Data’s platform includes tools for smart tagging, anonymization, and metadata management. These features empower enterprises to maintain control over their data while capitalizing on AI capabilities at scale.

VAST Data’s approach also facilitates the integration of AI systems with existing enterprise infrastructures. The platform can connect to data in its original location, eliminating the need for extensive data migrations. This flexibility is essential for organizations wanting to adopt AI without overhauling their entire data architecture.

Looking ahead, Hallak envisions VAST Data positioning itself at the forefront of industry evolution over the next four to five years, addressing emerging challenges in AI infrastructure, including heightened security needs, multi-tenancy, and quality of service in enterprise environments.