xAI Completes Installation of 100,000 Nvidia H100 AI Data Center
Despite Meta and OpenAI receiving their chips first, xAI managed to complete the installation of its 100,000 Nvidia H100 AI data center before them. The main chip installation and build were finished in just 19 days, with the entire project completed in 122 days. According to Nvidia CEO Jensen Huang, a project of this scale typically takes a year for most companies. The time saved by xAI has allowed it to catch up for Grok 3 to be trained and ready by the time GPT5 is released and before Llama 4 hits the market. The upcoming 200,000 H100/H200 xAI system is expected to be operational by December or January, enabling faster training for Grok 4 compared to GPT6. If xAI continues to accelerate, the 300K B200 Chip system could be 6-18 months ahead of its competitors, which would be utilized for Grok 5.
Meta and Microsoft's AI Data Center Deployments
Meta's AI data centers collectively have the capacity to hold around 600,000 NVIDIA H100 GPUs, as revealed by NVIDIA CEO Jensen Huang during a discussion at SIGGRAPH 2024. Microsoft collaborates with OpenAI, and both companies had over 150,000 Nvidia H100 chips delivered at least 6-12 months before xAI. Meta is on track to complete a 100,000+ Nvidia H100 AI data center by late October or November. Microsoft and OpenAI were utilizing 50,000 Nvidia H100s for GPT5 training until April 2024, and it is speculated that their data centers are expanding to accommodate 100,000 to 200,000 H100 and H200 chips.
Latest Developments at Meta Platforms
Meta Platforms is finalizing an over 100,000 Nvidia H100 server chip cluster in the US to train the next iteration of its generative AI model, Llama 4. The estimated cost of the chips alone could exceed US$2 billion. The cluster is expected to be operational soon, showcasing Meta's commitment to advancing AI technologies.
Source: Tweet
