Nvidia has made a significant impact in the chipmaking industry by shipping nearly half a million H100 and A100 GPUs in the third quarter of this year. This remarkable achievement translates to an impressive $14.5 billion in revenue from data centers, marking a nearly fourfold increase compared to the same quarter last year. This growth is detailed in a recent Market Snapshot Report by Vlad Galabov and Manoj Sukumaran, leading analysts at Omdia, a research firm specializing in cloud and data center insights.
The majority of GPU server shipments in Q3 were directed toward hyperscale cloud service providers, with Meta emerging as one of Nvidia's largest clients. Microsoft also made substantial orders for H100 GPUs, likely to support its expanding array of AI products and Copilots. Other top clients include Google, Amazon, Oracle, and Tencent, the latter facing challenges due to strict export restrictions from the Biden administration that limit access to H100 GPUs. Omdia's analysts predict that Nvidia's GPU shipments will exceed half a million by the end of Q4, driven by ongoing high demand for robust hardware.
However, many server manufacturers, including Dell, Lenovo, and HPE, are experiencing delays in fulfilling their H100 server orders. These setbacks stem from insufficient GPU allocations from Nvidia, resulting in estimated wait times of 36 to 52 weeks.
Market Dynamics
Galabov and Sukumaran project that the server market will reach a staggering $195.6 billion by 2027, more than doubling from a decade earlier. This growth is fueled by an increasing reliance on server processors and co-processors as organizations shift toward hyper heterogenous computing—server configurations optimized for specific applications that utilize multiple co-processors.
For AI training and inference tasks, the leading server setups include the Nvidia DGX server, equipped with eight H100/A100 GPUs, and Amazon's servers tailored for AI inference, featuring 16 custom-designed co-processors known as Inferentia 2. In the realm of video transcoding, Google's server with 20 video coding units (VCUs) and Meta's video processing server with 12 scalable video processors have emerged as market favorites.
The authors noted, “We anticipate this trend will grow, as the demand for certain applications has reached a scale that makes it financially beneficial to create optimized custom processors.” They highlighted that while media and AI sectors are currently reaping the benefits of hyper heterogeneous computing, other fields, such as databases and web services, are likely to embrace similar optimization strategies in the near future.
Infrastructure Growth
According to Omdia's findings, the rise of these advanced servers for AI applications is propelling growth in physical data center infrastructure. For instance, rack power distribution revenue in the first half of the year surged 17% compared to the previous year, slightly ahead of Omdia’s anticipated growth of 14%. Additionally, data center thermal management revenue is on track to see a 17% increase in 2023, driven by greater rack densities that necessitate innovative liquid cooling solutions.
As professional services for generative AI expand, fostering greater enterprise adoption in 2024 and beyond, the authors suggested that the primary constraint on the current pace of AI implementation could be the availability of power. They emphasized that their forecast remains unimpeded by financing constraints and highlighted an intriguing trend: companies are increasingly leveraging sought-after GPUs as a form of debt collateral.
This landscape underscores the vital role of innovative hardware and optimized server configurations in the ongoing evolution of data centers, particularly as demand for AI capabilities continues to escalate across various sectors.
Ranking
