AI's $600B Question

Meta has about 350,000 of these GPUs and a whole bunch of A100s. This means the ability to train 50 GPT-4 scale models every 90 days or 200 such models per year.

This level of overkill suggests to me that the core models will be commoditized to oblivion, making the actual profit margins from AI-centric companies close to 0, especially if Microsoft and Meta keep giving away these models for free.

This is actually terrible for investors, but amazing for builders (ironically).

The real value methinks is actually over the control of proprietary data used for training which is the single most important factor for model output quality. And this is actually as much an issue for copyright lawyers rather than software engineers once the big regulatory hammers start dropping to protect American workers.

I put this as equivalent to investing in Sun Microsystems and Netscape in the late 90s. We knew the internet was going to change the world, and we were right, but we were completely wrong as to how, and where the money would flow.

Highly debatable.

When we look back during the internet and mobile waves it is overwhelmingly the companies that came in after the hype cycle had died that have been enduring.

> A huge amount of economic value is going to be created by AI. Company builders focused on delivering value to end users will be rewarded handsomely.

Such strong speculative predictions about the future, with no evidence. How can anyone be so certain about this? Do they have some kind of crystal ball? Later in the article they even admit that this is another one of tech's all-too-familiar "Speculative frenzies."

The whole AI thing just continues to baffle me. It's like everyone is in the same trance and simply assuming and chanting over and over that This Will Change Everything, just like previous technology hype cycles were surely going to Change Everything. I mean, we're seeing huge companies' entire product strategies changing overnight because We Must All Believe.

How can anyone speak definitively about what AI will do at this stage of the cycle?

The biggest takeaway from this piece is the stark realism of this article (maybe a bit too bearish, imo) compared to the usual Sequoia VC-speak. Maybe FTX did teach them something, after all.

1. That all datacenter GPUs being purchased are feeding AI. You might be able to argue that some are or a lot are, but you don't know how many just looking at Nvidia sales numbers. I know of at least two projects deploying rows of cabinets in datacenters full of GPUs for non-AI workloads.

2. The assumption that pay-for-an-API is the only AI business model. What we now call "AI" has been driving Google's search and ad businesses for nearly a decade, sooo AI is already doing $300B/yr in revenue? There is no way for this guy to quantify how AI is solving problems that aren't SaaS.

David Chan, if you are reading this feel free to email me if you want a fact check for what will surely be the third installment in the series.

$600 billion is $434 per person, $36 per month per person, 1.2% percent of GDP.

If 75% of the spending goes to increasing productivity, I could see it. Get rid of $450 billion in labor costs (12 - 15 million work years). Cut worked hours in call centers, customer service, many services, menial programming jobs, ...

But I don't see it happening fast enough to pay for current investments.

So why aren't there more entrants in the CPU cloud area? The technology is a commodity. Google and Amazon don't make CPUs.

I'm reminded of Bitcoin/crpyto, which in its early history was all operated on GPUs. And, then, almost overnight, the whole thing was run on ASICs.

Is there an intrinsic reason something similar couldn't happen with LLMs? If so, the idea of a bubble seems even more concerning.

What if these AI investment were only to protect or strengthen their current business? AI in Windows, macOS, Adobe, iPhone, Facebook, Instagram may not bring in any additional revenue. But it add additional value to their current product line, making competition harder, further hardening their moat.

Nvidia or Jensen is also smart to play the national security card. Does the European want their model to be all US based. Are the answer culturally correct? Just like how every single country invested in their own Telecom or Internet infrastructure, if this pitch were even half as successful, do these numbers we are looking at even matter when it is spread out across G7 or G20?

While I believe we are still far, or at least 10+ years away from AGI, the current form of AI still has a lot of improvement incoming and are already bringing in real world benefits and value to a lot users. The adoption curve will accelerate once it is integrated into Windows, Office and Mac. So even if we are in a bubble, I still think we are very early in the curve before it burst.

So we have this bearish piece and the previous bearish Goldman Sachs piece. While I agree with their analysis in this case, there is a lingering doubt that some banks might just want to tank Nvidia a little in order to go long. Or something like that.

But other than FOMO why would someone buy better chips when they don't actually know what to do with their old ones?

So while I initially thought customer-facing roles would be front and center in the "AI revolution." Today, I tend to think they'll be bringing up the rear, with entertainment/smut applications at the forefront along with a few unexpected applications where LLMs operate behind the scenes.

#1 Just like the intranet there are a lot of productivity gains that firms like Tesla, Meta, Google and Amazon can gain internally by optimizing their own workflows. That in itself should justify the investment. Granted some of these optimisations will use their own chips instead of Nvidia, but Nvidia will get a lion-share of this.

#2. Then there are other verticals - pharma, oil and gas, logistics who can optimize their internal workflows to gain productivity. It just helps them improve their margins. No end user benefit may be realized and that’s fine.

#3. Nation states are buying GPUs too. Ex: Falcon2 was trained on a cluster owned by a middle eastern country. Nation states see something larger at stake here than just releasing an app. This does not have to even be a profitable endeavour.

The returns are going to chip-makers and employers, including single founder startups, who don't have to hire a lot of people, and additionally get productivity they never thought possible. A surgeon I know uses AI every day - to translate, to explain, to figure out problems, to write. They wouldn't have paid someone to do that, but now they get that output in seconds. This is a time to solve all kinds of problems we didn't think possible - because AI has made the enterprising among us instantly smarter.

All the fodder about AGI being a next step is smoke and mirrors - for everyone using OpenAI knows they don't need any more niche tools as their one $20 subscription is doing more for them every day. AGI is here. Experts can correct AI generated mistakes, but those are getting less and less too. The real benchmark is: Name how many people you know who can out-do ChatGPT on a question. You won't bother to check LinkedIn for that.

The gains are aggregating towards Chips, Clouds and Entrepreneurs. The VCs, since A16Z's original AI blog post (all expense, little return, echoed this Sequoia post but did it but years ago), know they are not needed as much anymore. Fewer VCs will beat the market when founders can grow startups without raising too much money (they don't need to hire as many people). Hiring needs lead to PR waves which require VC funding. Valuation is not a big deal for founders making money either, so they may not even disclose how successful their companies are. Bragging about your gains only invites competition. So other than ponzi-type ventures where you need to attract the dinner to serve dinner, you won't hear much about the good ones.

A different era indeed. The tech giants are in for a lot of change as well. Those who have distribution may try to push their models to the masses to be the point of reference, but that can get expensive, especially for those who don't charge. AI will help improve AI performance as well and that means cheaper better performance with time.

What's most needed in this era are people who know what the world needs that hasn't been invented yet. They need to be inventing and monetizing it. Little stopping you now.

But in which sector will these extemely important companies be active? Adtech? Knowledge management / productivity tools? Some completely new category?

What is an undeniable fact is the drastic commodization of the hardware / software stack for certain classes of algorithms. How is this technological development going to be absorbed and internalized by the economy feels still rather uncertain.

Modern web is full of examples of platforms that really don't make money...

Sell! Sell! Sell now before it's too late!

Most AI startups aren't building massive data centers so they're unaffected. Most money isn't spent on compute in most startups. Only a few companies spend big.

It's obviously a terrible idea to invest massive amounts into compute when Nvidia's profit margins are so astronomical if you need ROI in the long term. The massive corporations won't get their money back for these investments; but they don't have to.

Investors first need to ask what that ratio might look like in 10 years. 10-to-1? 100-to-1? Inference-to-Training

Assuming for each NVDA training GPU sold there are 100 open source / commodity GPUs doing inference, who owns and supplies those data centers and hardware?

This is one of the first time in the tech industry where the value was fully reaped by the hardware itself and not by the differentiated software that ran on top of it

The author of this blog makes a great argument that there is a risk of AI investments not paying off because if a revenue gap: The author argues that the gap between the revenue expectations implied by the AI infrastructure build-out and actual revenue growth in the AI ecosystem has increased from $200B to $600B. This is due to factors such as the subsiding of the GPU supply shortage, growing GPU stockpiles, and the dominance of OpenAI in AI revenue. The author also notes that the $125B hole in AI revenue has now become a $500B hole.

However, my experience with previous AI winters is not relevant here because now is the first time in history that there is a possibility of what we used to call “real AI” and now call AGI. No investor wants to miss out completely on the creation of near limitless wealth.

Assuming startups like Etched (with its recent massive funding) could shrink CapEx quite a bit (and make it not such a large revenue shortfall)

Just two words to describe all the idiocy of the current wave of AI offerings.

> A huge amount of economic value is going to be created by AI. Company builders focused on delivering value to end users will be rewarded handsomely. We are living through what has the potential to be a generation-defining technology wave.

No. It's crap no matter how much money you throw at it. It will end in tears, because there is no standard, no operating system, no protocol, only a handful of APIs controlled by a couple of companies. They are not building networks, but hubs with a single point of failure--the API provider. The internet is such a world-changing force because it is built on top of TCP/IP, which allows the rest of the internet to survive even if a part of it goes down. When an AI API provider shuts down, all those bullshitters repackaging LLMs will be left holding the bag, or rather their investors will be. In a way, the coming AI bubble burst is going to be a self-fulfilling prophecy--AI will make its creators redundant.

> But we need to make sure not to believe in the delusion that has now spread from Silicon Valley to the rest of the country, and indeed the world. That delusion says that we’re all going to get rich quick, because AGI is coming tomorrow, and we all need to stockpile the only valuable resource, which is GPUs.

No, the delusion is that Gen AI is good for anything. It is not.

Soon, the $600B question is going to be, "where's the money gone?"

What do you get if you multiply six by seven