Elon Musk is putting his AI chips to work — and he's catching up with ...
Despite distractions such as the Brazilian Supreme Court's ban on X, Elon Musk is forging ahead with his AI ambitions. Musk recently announced that xAI, the company he founded in 2023, has launched a colossal new training cluster of chips, dubbed Colossus, in Memphis, Tennessee. This system, powered by 100,000 Nvidia chips, is hailed as the most powerful AI training system globally, with plans to double in size in the coming months.
Playing Catch-Up with Zuckerberg
While Musk confirmed the cluster's size back in July, bringing it online marks a significant milestone for his AI aspirations. It also allows him to keep pace with his Silicon Valley rival, Mark Zuckerberg. Both tech giants rely on high-performance GPUs to fuel their AI endeavors, with the demand for Nvidia GPUs skyrocketing since the advent of ChatGPT in late 2022.
Companies are vying for these GPUs, despite their hefty price tags and supply shortages. Meta, led by Zuckerberg, aims to amass 600,000 GPUs by year-end, a substantial portion being Nvidia H100s. On the other hand, Musk's xAI has acquired 100,000 H100 GPUs, while Tesla holds 35,000.
The AI Arms Race
Meta's undisclosed GPU count notwithstanding, the company has made strides in AI development, training its large language model on 16,000 H100 GPUs. Meanwhile, xAI's latest training cluster dwarfs Meta's efforts, comprising 100,000 H100 GPUs — a feat that has garnered industry acclaim.
Nvidia's data-center account lauded Colossus as the world's largest GPU supercomputer, commending its swift deployment. Greg Yang, xAI's cofounder, celebrated the achievement with a reference to a popular rap song, underscoring the magnitude of the milestone.
Shaun Maguire, a partner at Sequoia Capital, attested to xAI's access to cutting-edge technology for its Grok chatbot development, signaling competitiveness with industry leaders. However, the path to commercializing AI remains a challenge for many companies, including xAI, as Musk aims to unveil a groundbreaking version of Grok post-training on 100,000 H100s.
The Road Ahead
As Musk gears up to unveil the next iteration of Grok, the AI landscape is set for further evolution, with xAI and Meta poised to push the boundaries of AI innovation. Stay tuned to witness how these developments position Musk against Zuckerberg in the realm of artificial intelligence.
