Google Cloud Unveils 5th Generation Custom TPUs for Enhanced Performance and Innovation

At its annual user conference, Cloud Next, Google Cloud unveiled the highly anticipated fifth generation of its tensor processing units (TPUs) designed for AI training and inferencing. While the fourth generation was announced in 2021, it became accessible to developers only in 2022.

This latest version of the chip emphasizes efficiency, boasting remarkable advancements over its predecessor. Specifically, this generation offers a 2x improvement in training performance per dollar and a 2.5x enhancement in inferencing performance per dollar.

“This is our most cost-effective and accessible cloud TPU to date,” stated Mark Lohmeyer, VP and GM for compute and ML infrastructure at Google Cloud, during a press conference prior to the announcement. He highlighted the significant scalability advancements that users can expect from this TPU iteration.

“We empower our customers to effortlessly scale their AI models beyond the limitations of a single TPU pod or cluster," Lohmeyer explained. "In essence, a large AI workload can now be distributed across multiple TPU clusters, effectively utilizing tens of thousands of chips in a cost-efficient manner. By offering both cloud GPUs and cloud TPUs, we are providing our customers with diverse choices and great flexibility to address the wide range of emerging AI workloads.”

In addition to the new TPUs, Google also revealed plans to make Nvidia’s H100 GPUs generally available to developers next month, featuring them in the A3 series of virtual machines. For more details, you can explore further information here.