As generative AI continues to transform various industries, the companies crafting chips to power these models are reaping significant rewards. Nvidia stands out, controlling an estimated 70% to 95% of the AI chip market. Major cloud service providers, including Meta and Microsoft, are investing billions in Nvidia GPUs, eager to stay competitive in the fast-paced generative AI landscape.
Given this scenario, it's no surprise that generative AI vendors are dissatisfied with the current state of affairs. Their success largely depends on the whims of leading chip manufacturers, prompting them and forward-thinking venture capitalists to seek innovative startups that can challenge the establishment.
One intriguing newcomer in this space is Etched, founded just two years ago by Harvard dropouts Gavin Uberti (formerly of OctoML and Xnor.ai) and Chris Zhu, alongside Robert Wachen and former Cypress Semiconductor CTO Mark Ross. Their mission: to develop a chip specifically designed for running AI models.
While many startups and tech giants are producing inference chips for AI models, Etched’s chips stand out as they are optimized for a specific type of model: Transformers. This architecture, introduced by Google researchers in 2017, has rapidly become the leading framework for generative AI applications.
Transformers power notable models, such as OpenAI’s Sora for video generation, Anthropic’s Claude for text generation, and advanced art generators like the latest version of Stable Diffusion. “In 2022, we bet on transformers becoming the standard,” said Uberti, Etched's CEO. “We are at a point where specialized chips outperform general-purpose GPUs, and industry decision-makers recognize this.”
Etched’s flagship chip, Sohu, is an ASIC (application-specific integrated circuit) designed exclusively for Transformers. Utilizing TSMC’s 4nm manufacturing process, Sohu promises significantly improved inference performance compared to traditional GPUs while consuming less power, according to Uberti. “Sohu is an order of magnitude faster and cheaper than even Nvidia’s next-gen Blackwell GB200 GPUs for running text, image, and video Transformers,” he explained. “Just one Sohu server can replace 160 H100 GPUs, making it a more cost-effective and eco-friendly choice for businesses seeking specialized chips.”
So, how does Sohu achieve these efficiencies? The answer lies in its streamlined hardware-and-software architecture. By focusing solely on Transformers, the Etched team eliminated unnecessary hardware components and minimized the software overhead typically associated with non-transformer models.
Amid mounting concerns regarding hardware efficiency, Etched emerges at a crucial turning point in the generative AI infrastructure race. GPU power consumption poses significant issues; Goldman Sachs predicts AI will drive a staggering 160% rise in data center electricity demand by 2030, contributing to a concerning increase in greenhouse gas emissions. Additionally, researchers at UC Riverside estimate that global AI activities may lead data centers to consume between 1.1 trillion and 1.7 trillion gallons of freshwater by 2027 — a critical concern given that many data centers rely on water for cooling.
Uberti optimistically positions Sohu as a solution to the industry’s consumption challenge. “In essence, our future customers won’t be able to afford to overlook Sohu,” he claimed. “Businesses are willing to invest in Etched because speed and cost are essential to the AI products they are developing.”
Yet, with numerous contenders in the AI chip space, can Etched succeed once Sohu enters the mass market in the coming months? Currently, the company faces limited direct competition, though the AI chip startup Perceive has unveiled a processor with transformer-specific acceleration. Groq is also investing heavily in optimizing its ASICs for Transformers.
However, if Transformers were to wane in popularity, Uberti assures that Etched would simply pivot by developing a new chip. While that’s a valid strategy, it’s a considerable risk, especially given the long journey to unveil Sohu.
Despite these challenges, investor confidence in Etched remains high. The company recently closed a substantial $120 million Series A funding round led by Primary Venture Partners and Positive Sum Ventures, bringing its total funding to $125.36 million. Notable angel investors include Peter Thiel (who has ties to all four co-founders), GitHub CEO Thomas Dohmke, Cruise co-founder Kyle Vogt, and Quora co-founder Charlie Cheever.
These investors likely believe in Etched’s potential to scale its server sales business. Uberti mentioned that unnamed customers have already reserved “tens of millions of dollars” in hardware. The impending launch of the Sohu Developer Cloud, allowing users to explore Sohu through an interactive online platform, is expected to further boost revenue.
Nonetheless, it remains uncertain whether this will be sufficient to elevate Etched and its 35-person team to the future envisioned by its founders. The AI chip sector can be unforgiving; high-profile struggles faced by AI chip startups like Mythic and Graphcore highlight the challenges in this competitive space, especially as investment in AI chip ventures has declined in recent times.
Undeterred, Uberti makes a compelling argument: “Key advancements in video generation, audio processing, robotics, and other future AI applications will hinge on faster chips like Sohu. The capacity for AI technology to evolve depends on whether our infrastructure can scale.”
Ranking
