Groq, a startup focused on developing chips designed to accelerate generative AI models beyond the capabilities of traditional processors, announced on Monday that it has successfully secured $640 million in a new funding round led by BlackRock. Other notable participants included Neuberger Berman, Type One Ventures, Cisco, KDDI, and the Samsung Catalyst Fund.
This funding round elevates Groq's total capital raised to over $1 billion and values the company at approximately $2.8 billion—a significant achievement for Groq, which initially aimed to raise $300 million at a lower valuation of around $2.5 billion. This new valuation more than doubles Groq's previous worth of about $1 billion from April 2021, when the company raised $300 million in a round led by Tiger Global Management and D1 Capital Partners.
In an exciting move for the company, Meta's chief AI scientist, Yann LeCun, will serve as a technical advisor, while Stuart Pann, the former head of Intel’s foundry division and ex-CIO of HP, joins Groq as Chief Operating Officer. LeCun's involvement is particularly intriguing given Meta’s investments in its own AI chips, positioning Groq with a formidable ally in the competitive landscape.
Founded in 2016, Groq is pioneering what it terms an LPU (language processing unit) inference engine. The company boasts that its LPUs can execute established generative AI models similar to OpenAI's ChatGPT and GPT-4 at 10 times the speed and with one-tenth the energy consumption.
Groq's CEO, Jonathan Ross, is well-known for his role in the creation of Google's tensor processing unit (TPU), which is used for training and running AI models. He co-founded Groq nearly a decade ago with Douglas Wightman, a former engineer at Alphabet’s X moonshot lab.
Offering an LPU-powered developer platform called GroqCloud, the company provides access to "open" models such as Meta’s Llama 3.1 family, Google’s Gemma, OpenAI’s Whisper, and Mistral’s Mixtral, alongside an API that enables customers to utilize its chips in cloud environments. Groq also launched GroqChat, an AI-powered chatbot playground, late last year. As of July, GroqCloud had over 356,000 developers actively using the platform, and the company intends to allocate a portion of the new funding to scale its capacity and introduce additional models and features.
According to COO Stuart Pann, "Many of these developers are at significant enterprises, and by our estimates, over 75% of the Fortune 100 are represented."
As the generative AI sector continues to surge, Groq encounters rising competition from both emerging AI chip startups and established companies like Nvidia, which commands a dominant 70% to 95% of the AI chip market for training and deploying generative AI models. Nvidia is aggressively advancing its position by committing to annual releases of new AI chip architectures and establishing a business unit dedicated to custom chip designs for cloud computing firms.
In addition to Nvidia, Groq faces competitors like Amazon, Google, and Microsoft, all of which are developing or preparing to release custom chips for AI applications in the cloud. Amazon offers Trainium, Inferentia, and Graviton processors via AWS; Google Cloud provides TPUs and plans to introduce its Axion chip; and Microsoft recently previewed Azure instances featuring its Cobalt 100 CPU, with Maia 100 AI Accelerator instances coming soon.
Competing with companies such as Arm, Intel, and AMD, Groq is positioning itself in a rapidly expanding AI chip market projected to reach $400 billion in annual sales within the next five years. Arm and AMD are notably growing their AI chip divisions, benefitting from increased capital expenditure by cloud providers to meet generative AI demands.
Recent developments in the industry include D-Matrix raising $110 million for a cutting-edge inference compute platform and Etched securing $120 million in funding for a processor tailored for the prevalent transformer architecture in generative AI. Additionally, SoftBank's Masayoshi Son is reportedly aiming to raise $100 billion for a chip venture aimed at competing with Nvidia. OpenAI is also exploring opportunities with investment firms to embark on an AI chip manufacturing initiative.
To solidify its market position, Groq is making significant investments in enterprise and government outreach. In March, Groq acquired Definitive Intelligence, a Palo Alto-based company specializing in business-oriented AI solutions, to establish Groq Systems, which focuses on serving organizations, including U.S. government agencies and sovereign nations, looking to integrate Groq's chips into existing or new data centers.
Most recently, Groq formed a partnership with Carahsoft, a government IT contractor, to distribute its solutions to public sector entities through Carahsoft’s reseller network. The startup has also signed a letter of intent to deploy tens of thousands of its LPUs at European firm Earth Wind & Power’s data center in Norway and is collaborating with Saudi Arabia’s Aramco Digital for future LPU installations in the Middle East.
As it fosters customer relationships, Groq is simultaneously advancing toward the next generation of its chips. Last August, the company announced its partnership with Samsung’s foundry to manufacture 4nm LPUs, which promise enhanced performance and efficiency over Groq's initial 13nm chips.
Groq aims to have over 108,000 LPUs deployed by the end of Q1 2025.
Ranking
