Modular, a startup focused on creating innovative solutions for developing and optimizing AI systems, has successfully raised $100 million in a funding round led by General Catalyst, with participation from GV (Google Ventures), SV Angel, Greylock, and Factory. This funding elevates Modular's total capital raised to $130 million, which will primarily support product expansion, hardware enhancements, and the development of Modular’s unique programming language, Mojo, as stated by CEO Chris Lattner.
“Given that we operate in a highly technical realm that demands specialized expertise, this funding will help us grow our team effectively,” Lattner shared in an email interview. “Our budget will be directed towards refining our core products and scaling operations to meet overwhelming customer demand, rather than on AI compute resources.”
Lattner, a former Google engineer, co-founded Palo Alto-based Modular in 2022 alongside Tim Davis, another ex-colleague from Google Brain. They recognized that the potential of AI was significantly constrained by a complex, fragmented technical infrastructure. As a result, Modular was established with the mission of simplifying the processes of building and managing large-scale AI systems.
Modular offers a powerful engine designed to enhance the inferencing performance of AI models on CPUs, and will soon extend its capabilities to GPUs by the end of the year, all while delivering cost efficiencies. The engine, currently in closed preview, integrates seamlessly with existing cloud environments, machine learning frameworks like Google’s TensorFlow and Meta’s PyTorch, as well as other AI accelerator engines. Lattner claims that this engine enables developers to run trained models up to 7.5 times faster than on their original frameworks.
Another flagship product, Mojo, is a programming language that combines the ease of use of Python with advanced features such as caching, adaptive compilation techniques, and metaprogramming. Currently in preview for "hundreds" of early adopters, Mojo is set for general availability early next month.
“Our developer platform empowers our customers and the global community of developers to streamline their AI technology stacks, accelerating innovation and maximizing the value of their AI investments,” Lattner emphasized. “We are tackling the complexities that hinder AI development today by addressing the fragmentation issues that exist between AI software and hardware.”
With around 70 employees, Modular’s ambitions may seem lofty. However, it is not alone in its quest to enhance AI efficiency. Startups like Deci, backed by Intel, and OctoML, which automates optimization and benchmarking for various hardware, are also exploring similar paths.
As the demand for AI technologies surges, it is approaching unsustainable levels. Current generative AI models are significantly larger than previous versions, as highlighted in a recent Wall Street Journal article. Much of the existing public cloud infrastructure was not originally designed to support these expansive systems, especially at scale.
The impact is already evident; Microsoft has reported a severe shortage of server hardware required for AI operations, potentially leading to service interruptions. Additionally, the soaring demand for AI inferencing hardware, primarily GPUs, has propelled Nvidia’s market capitalization to $1 trillion. Notably, the increasing demand has resulted in a shortage of Nvidia’s top-performing AI chips, with reports indicating they are sold out until 2024.
These challenges are reflected in a 2023 poll from S&P Global, revealing that over half of AI decision-makers in leading companies face obstacles in deploying the latest AI tools.
“The compute power needed for current AI applications is tremendous and unsustainable under existing models,” Lattner warned. “We are already witnessing instances where demand exceeds compute capacity. Costs are spiraling while only large tech firms can afford to build such solutions. Modular aims to overcome this hurdle, making AI products and services far more accessible, sustainable, and affordable for enterprises.”
While Mojo brings a fresh take on programming, its widespread acceptance may face challenges due to Python's deep roots in the machine learning sector—where 87% of data scientists reported regular use of Python as of 2020. Nevertheless, Lattner believes Mojo's advantages will ignite its adoption.
“A common misconception is viewing AI applications solely as high-performance accelerator problems,” he explained. “In reality, AI involves complex end-to-end data processes, from loading and transforming data to post-processing and networking. These ancillary tasks are typically managed in Python and C++. Only Modular’s approach with Mojo can unify these components within a single technology base without sacrificing performance or scalability.”
Evidence of growing interest in Modular’s solutions is reflected in its community expansion, which has surpassed 120,000 developers since the product keynote in early May. Leading tech companies are already utilizing the startup's infrastructure, with 30,000 additional developers on the waitlist.
“The greatest challenge for Modular is overcoming complexity—both in software layers that function only under specific conditions and in the low-level nature of high-performance accelerators,” Lattner concluded. “While the transformative nature of AI drives its demand, it also complicates scaling and necessitates significant talent and compute resources. Combined, the Modular engine and Mojo democratize access to AI technology, paving the way for a more efficient future.”
With such a robust funding background and a compelling vision, Modular is poised for significant growth in the AI landscape.
Ranking
