As ChatGPT gains popularity, leading tech companies like Microsoft, Google, Meta, and now Intel are joining the AI landscape. During the recent International Supercomputing Conference (ISC) in Hamburg, Germany, Intel unveiled its collaboration with Argonne National Laboratory to develop a groundbreaking generative AI model called Aurora genAI, which will feature up to 1 trillion parameters. For context, this makes Aurora genAI at least five times larger than ChatGPT, which has 175 billion parameters.
Aurora genAI: Powered by the Aurora Supercomputer
Intel's Aurora genAI model is built on two robust frameworks: Nvidia's Megatron and Microsoft's DeepSpeed.
- Megatron: This architecture optimizes distributed training for large language models, particularly Transformers, supporting both data and model parallelism.
- DeepSpeed: This framework enhances the training of large deep learning models, improving scale, speed, cost, and usability, enabling models of up to 100 billion parameters.
The Aurora genAI model will utilize the Aurora supercomputer, specifically designed for Argonne National Laboratory. After several delays, the supercomputer is now operational. Publicly available details reveal that it features Intel's Xeon CPU Max and GPU Max chip series, with 10,624 nodes, 63,744 Ponte Vecchio GPUs, and 21,248 Sapphire Rapids Xeon CPUs, among other components.
Intel recently reported that the Aurora supercomputer doubles the performance of AMD's MI250 GPUs for scientific and engineering workloads and outperforms the H100 in quantum mechanical applications by 20%. It promises linear scalability across hundreds of nodes.
Looking ahead, the Aurora supercomputer is anticipated to deliver over 2 Exaflops of double-precision floating-point computing power, surpassing the current leading supercomputer, Frontier, which operates at 1.194 Exaflop/s.
A Focus on Scientific Applications
Given its immense computing power, the Aurora genAI model is strategically designed for large-scale scientific initiatives. Argonne National Lab is spearheading international collaborations for the project. Rick Stevens, Deputy Lab Director at Argonne, emphasizes that the goal is to fully exploit the capabilities of the Aurora supercomputer to support the Department of Energy's scientific projects and beyond.
The Aurora genAI model will train on diverse data, including general text, code, and scientific datasets in disciplines such as biology, chemistry, materials science, physics, and medicine. With up to 1 trillion parameters, it aims to address various scientific challenges, including systems biology, cancer research, climate science, and polymer chemistry. Its applications also extend to finance, natural language processing, machine translation, image recognition, and voice recognition.
Expected Completion in 2024
Intel aims to finalize the Aurora genAI model by 2024. This ambitious announcement has generated significant interest, especially as Intel sets a new standard in AI development with the 1 trillion parameter target, heightening competition with existing models like GPT-4.
Experts have weighed in on this initiative:
- "One trillion parameters could signal a threshold, though it may be just another notable milestone. If the model shares traits with GPT-4, it will enhance the dialogue around AI capabilities. As various companies make announcements, we might see a peak interest around mid-year."
- "Currently, companies are leaning heavily on H100 GPUs for new systems, but clearly superior AI options are emerging. If this trend persists, NVIDIA may need to quickly introduce new graphics cards to stay competitive."
- "I foresee that GPT-4 may soon relinquish its State of the Art status in several benchmarks, as newer models will likely be trained on the most advanced supercomputers. In terms of GPU power, OpenAI's setup features around 10,000 GPUs, contrasted with Aurora's 63,744 GPUs."
The emergence of Intel’s Aurora genAI marks a significant shift in the AI landscape, promising transformative insights across various scientific and technological fields.
Ranking
