Nvidia GPU Access: A Growing Business to Satisfy Rising Demand

Feeding the escalating demand for Nvidia GPUs, which captured the attention of Silicon Valley last summer, has transformed into a lucrative venture within the AI industry.

For instance, Lambda, a GPU cloud company utilizing Nvidia technology, recently announced a new round of financing, raising $320 million at a $1.5 billion valuation. This funding will help expand its AI cloud services.

In related news, Salesforce has invested in Together AI, which is now valued at over $1 billion, according to a recent report from The Information. Additionally, in December 2023, GPU cloud provider CoreWeave achieved an impressive $7 billion valuation following a $642 million minority investment led by Fidelity Management and Research Co.

Nvidia's stock has tripled as AI startups scramble for GPU access. A year ago, experts identified Nvidia, dominating over 80% of the high-end AI chip market, as a key player poised to benefit from the booming generative AI sector. Recently, its market capitalization surpassed that of Amazon and Alphabet.

Startups are increasingly desperate for Nvidia’s ultra-expensive H100 GPUs, crucial for training large language models (LLMs). Investor and former GitHub CEO Nat Friedman revealed the creation of a platform resembling Craigslist for GPU clusters, offering listings like “32 H100s available from 02/14/2024 to 03/31/2024.”

Moreover, Forbes reported that Friedman and his partner Daniel Gross developed their own supercomputer, the Andromeda Cluster, which comprises over 4,000 GPUs. They provide these computing resources to companies in their portfolio at below-market rates.

Friedman shared that he has become primarily a computer chip broker for emerging AI firms, stating, “There were weeks where I spent most of my time finding GPUs for people. Asking founders what they needed help with became problem number one.”

In light of this GPU demand, OpenAI CEO Sam Altman suggested a transformative project aimed at redefining the AI chip landscape, potentially costing trillions and influenced by geopolitical factors.

However, not everyone agrees with this approach. Databricks CEO Ali Ghodsi, for instance, anticipates a significant drop in AI chip prices and a realignment of supply and demand within the next year. He likened the situation to past concerns over internet bandwidth, predicting a similar resolution for GPU shortages.

The pressing question remains: Will this shift occur soon enough to alleviate the GPU starvation faced by AI startups?