Google Unveils Cloud TPU v5p: The Most Powerful AI Accelerator to Date

Google has officially launched its new Gemini large language model (LLM) alongside its latest Cloud TPU v5p, an upgraded version of the previously released Cloud TPU v5e. The v5p pods feature an impressive total of 8,960 chips, enhanced by Google’s fastest interconnect technology, offering speeds of up to 4,800 Gbps per chip.

Google boasts that these new chips deliver significant performance improvements over the previous v4 TPUs, with a claimed 2x increase in FLOPS and a 3x enhancement in high-bandwidth memory. This advancement is reminiscent of the leap from the older OpenAI GPT-3.5 model to the current innovation in AI. Notably, Google has continuously pushed the boundaries of performance beyond the TPU v4; however, the v5e pods did present some limitations compared to the v4 pods, featuring only 256 v5e chips per pod compared to 4,096 in the v4, and delivering 197 TFLOPs in 16-bit floating point performance per v5e chip, versus 275 for the v4 models. With the introduction of the v5p, Google promises up to 459 TFLOPs of 16-bit floating point performance, powered by the faster interconnect.

According to Google, this upgrade enables the TPU v5p to train large language models, like GPT-3 with 175 billion parameters, 2.8 times faster than the TPU v4. Additionally, it does this in a more cost-effective manner, although the TPU v5e, despite being slower, offers better relative performance per dollar compared to the v5p.

Jeff Dean, Chief Scientist at Google DeepMind and Google Research, stated, “In our initial testing, Google DeepMind and Google Research have observed 2X speedups for LLM training workloads using TPU v5p chips compared to the TPU v4 generation. The solid support for various ML frameworks—I.e., JAX, PyTorch, TensorFlow—coupled with orchestration tools, allows us to scale more efficiently with v5p. The second generation of SparseCores also brings significant improvements for embedding-heavy workloads. TPUs are essential in driving our large-scale research and engineering projects on advanced models like Gemini.”

Currently, the TPU v5p is not generally available, and developers interested in accessing it will need to contact their Google account manager to be added to the list.