Four co's are hoarding billions worth of Nvidia GPU chips. Meta has 350K of them

Meta has released Llama 3.1, a new large language model that reportedly outperforms OpenAI's ChatGPT 4o on certain benchmarks. The model, particularly its largest version with 405 billion parameters, was trained using up to 16,000 Nvidia H100 GPUs, which are valued between $20,000 and $40,000 each. This indicates that Meta has invested up to $640 million in hardware for this model alone, part of a larger goal to amass 350,000 H100s, totaling over $10 billion in Nvidia chips. Other companies, including venture capital firm Andreessen Horowitz and Tesla, are also stockpiling H100s for AI training. Andreessen Horowitz reportedly has over 20,000 GPUs, while Tesla aims for 35,000 to 85,000 H100s, with Elon Musk stating that xAI's training cluster consists of 100,000 H100s. The demand for these GPUs is so high that there are reports of individuals being paid to smuggle them into China to evade U.S. export controls. OpenAI's GPU strategy remains less transparent, but it is known to rent significant processing power from Microsoft and Oracle. Meanwhile, the California Supreme Court upheld Proposition 22, allowing gig companies like Uber and Lyft to classify drivers as independent contractors, which has implications for worker protections and company costs. This ruling has been met with mixed reactions, as it maintains the current business model for these companies.