Revolutionary SambaNova Chip Capable of Managing 5 Trillion Parameter Models

Since OpenAI launched ChatGPT last year, buzzwords like "generative AI" and "large language models" have become ubiquitous. However, a deeper look reveals that large language models often depend on costly GPU chips. Today, SambaNova is unveiling a chip designed to significantly cut these costs while efficiently managing a 5 trillion parameter model.

While SambaNova may not yet be as recognized as industry giants like Google, Microsoft, or Amazon, the company has been diligently developing a comprehensive AI solution—hardware and software combined—for years. They have successfully raised over $1 billion in funding from notable investors, including Intel Capital, BlackRock, and the SoftBank Vision Fund. Today, they introduced the SN40L, representing the fourth generation of their custom AI chips.

Rodrigo Liang, the company’s founder and CEO, explains that creating their own chips gives them the ability to optimize hardware for peak efficiency, which is increasingly essential as the demand for processing intensive large language models grows. “We need to move away from the brute force method of deploying more chips for these AI tasks. That’s why we created the SN40L, tailored specifically for very large language models to power enterprise-level AI,” Liang shared.

According to Liang, the new chips are 30 times more efficient as they minimize the number of chips necessary for executing these advanced models. He claims that while competitor chips might require 50 to 200 units to run a trillion parameter model, SambaNova can achieve this with just eight chips. “How many resources are truly required to run a trillion parameter model like GPT-4? I can accomplish it in eight sockets, deliver it on-premises, and provide fully optimized performance with state-of-the-art accuracy,” he stated.

This assertion is ambitious, yet SambaNova offers a complete hardware and software solution that empowers businesses to develop their AI applications seamlessly. “We are in the business of creating AI assets that enable quick model training based on your proprietary data, turning that into a valuable asset for your company,” he asserted. Importantly, SambaNova ensures that customers retain ownership of their data and the models trained on it, even after the training process is complete: “We tell our clients, it’s your data and your model. After we train the model, ownership remains with your company indefinitely.”

By offering this hardware and software suite through a multiyear subscription model, SambaNova provides customers with greater cost predictability for their AI initiatives. The new SN40L chip is available for purchase today and is fully backward-compatible with earlier chip generations, ensuring a smooth transition for existing customers.