Arm Highlights Expanding Ecosystem of Sustainable AI Datacenter Silicon Solutions

Arm announced that its Total Design ecosystem has doubled in size within just a year of its launch, significantly advancing global silicon innovation for sustainability.

Data centers are facing ongoing challenges in balancing power demands with the surge in AI workloads, alongside the rising costs and complexities of chip development. In a recent blog post, Eddie Ramirez, Vice President of Go-to-Market for Arm's Infrastructure Line of Business, emphasized that the Arm Total Design initiative was created to tackle these issues by fostering an ecosystem of partners dedicated to accelerating the development of custom silicon. This collaboration aims to build solutions for the future of data centers using Arm Compute Subsystems (CSS).

The ecosystem has rapidly evolved into a multi-vendor, Arm-based chiplet and SoC environment, which now includes over 30 participating companies, among them Alcor Micro, Egis, PUF Security, and SemiFive.

New Arm-Based Solutions Powering AI Data Centers Sustainably

Arm Total Design has facilitated global collaboration, resulting in real-world CSS-powered solutions designed for GenAI computing. Notably, Arm, Samsung Foundry, ADTechnology, and Rebellions have partnered to launch an AI CPU chiplet platform tailored for cloud, HPC, and AI/ML workloads. This platform combines Rebellions’ Rebel AI accelerator with a Neoverse CSS V3-powered compute chiplet from ADTechnology, utilizing Samsung Foundry's advanced 2nm Gate-All-Around (GAA) process technology for production.

The collaboration promises exceptional performance and optimal power efficiency, with an estimated 2x to 3x efficiency advantage for GenAI workloads using Llama3.1 405B parameter LLMs.

“AI and HPC designs require technology solutions that deliver maximum performance, high transistor density, and energy efficiency,” stated Taejoong Song, Head of Foundry Business Development at Samsung Electronics. He expressed excitement about leveraging the flexibility of Arm CSS alongside the Arm Total Design ecosystem to enhance the adoption of cutting-edge technologies for hyperscalers and cloud service providers.

This initiative exemplifies how Arm Total Design and its standards-based compute subsystems accelerate AI silicon development by integrating Arm-optimized EDA tools, global design expertise, and foundry partnerships, facilitating easier integration for AI accelerator designers.

As AI workloads rapidly evolve, tightly coupled CPU compute becomes essential for supporting the complete AI stack. Tasks like data pre-processing, orchestration, and database augmentation techniques such as Retrieval-augmented Generation (RAG) can all leverage the performance efficiency of Arm Neoverse CPUs. These capabilities are integral to the CSS, which is already benefiting the ecosystem.

Setting a New Standard for Purpose-Built AI Infrastructure

Arm Total Design is establishing a robust hardware foundation for sustainable AI data centers. It is accelerating the creation of Arm-based test chips and chiplet products powered by Neoverse N-series or V-series CSS. The diverse range of chiplet solutions, from cloud to edge, is expanding rapidly due to reduced entry barriers and broader access to the latest CSS tools.

Recently, Alcor Micro announced plans to develop a chiplet powered by CSS aimed at AI/ML training and inference. Similarly, Alphawave revealed its advanced compute chiplet built on CSS designed for AI/ML, HPC, data center, and 5G/6G applications. These Arm-based chiplets showcase the flexibility, diversity, and global supply chain capabilities that only the Arm partnership can provide.

Furthermore, partners like Alphawave, Cadence, and proteanTecs are validating their third-party IP products using CSS on advanced nodes to ensure compliance with Arm specifications. This means that partners can build CSS-based custom silicon on cutting-edge nodes and enjoy a seamless software experience.

Software readiness remains crucial for unleashing AI's potential. For over 30 years, Arm has committed to ensuring that software on Arm platforms operates reliably. Today, all major frameworks and operating systems are compatible with Arm, allowing diverse silicon solutions from the Arm Total Design ecosystem to benefit from a vibrant software landscape.

A prime example of this investment is the introduction of Arm Kleidi technology, which optimizes CPU-based inference for open-source projects like PyTorch and Llama.cpp. This optimization is especially valuable for Arm Total Design partners developing CSS-based chiplets for edge AI computing without needing an accelerator.