Elon Musk Unveils World's Largest AI Training Cluster with Plans to Add 100,000 GPUs

Elon Musk's AI computing cluster, Colossus, is officially online and set to double its processing power in the near future.

On September 3rd, Musk, the CEO of Tesla, announced on the X platform that xAI's latest creation, the super AI training cluster called Colossus, has been launched. The team completed the setup in just 122 days and plans to add 100,000 GPUs in the coming months, with 50,000 utilizing the more advanced Nvidia H200, which is expected to further enhance its processing capabilities.

The announcement of Colossus has garnered significant attention in the industry. Cathie Wood, CEO of the venture capital firm ARK Invest, congratulated Musk, calling the achievement "impressive" and hinting at more major developments from xAI in the future.

Reports from early last year indicated that Musk was procuring a considerable number of GPUs for Tesla. Colossus was initially revealed in May and began operation in July. During a speech on May 25th, Musk mentioned an ambition to create a supercomputer dubbed the "super factory of computing power," expected to be four times more powerful than its closest competitors, utilizing Nvidia's H100 GPUs.

On July 22nd, Musk announced that the xAI team, along with other partners, had commenced training on the Memphis Supercluster. Composed of 100,000 liquid-cooled H100 GPUs running on a single RDMA structure, Colossus is touted as "the most powerful AI training cluster in the world." Musk aims to train "the world's strongest AI, measured across various metrics," by this December.

The Grok large model from xAI will be trained on Colossus. Founded in July of last year, xAI's mission is to "understand the true nature of the universe" while collaborating closely with companies like X and Tesla. In November, xAI released its first large model, Grok-1, and in an August 13th beta release, it introduced image generation capabilities with Grok-2. Musk has indicated that Grok-3 will be trained on Colossus, with a planned release by the end of the year.

In May of this year, xAI completed a $6 billion Series B funding round, raising its post-money valuation from $18 billion to $24 billion. While advancing AI technology, Musk has also been vocal about the potential safety risks of AI, calling for the establishment of regulatory measures.

On August 27th, Musk posted in support of California Senate Bill SB 1047 on X, which mandates that AI models with development costs exceeding $100 million report safety risks. This proposal has sparked controversy in the tech industry, with many expressing concerns over its potential to stifle innovation in the U.S. The bill has met resistance from xAI's competitor, OpenAI.

In summary, as Musk drives forward in AI development, he remains deeply invested in addressing its potential risks and advocating for appropriate regulatory frameworks.