Goldman Sachs says the return on investment for AI might be disappointing

I agree with the folks at GS. Every non-tech corporation in the world is trying to figure out some kind of "AI strategy," but most executives and managers have no clue as to what they're doing or ought to be doing with respect to "AI." They're like patsies joining a very expensive game of poker they don't understand, and they are driven by FOMO, ready and eager to put down gobs of cash on the table, because they don't want to miss their chance of winning. They are on track to learn predictable lessons that cannot be learned in any other way.

I’m confused by the evidence they use:

> to justify those costs, the technology must be able to solve complex problems, which it isn't designed to do

Planning and reasoning are the two greatest areas of research in AI right now, with an OOM more researchers devoted to it than there were to the first generation of generative AI architectures

> In our experience, even basic summarization tasks often yield illegible and nonsensical results

Summarization with current generation models is excellent. I can get a summarization of a several-hour-long-call with better recall than I could have had myself, for less than $2 in inference costs.

> even if costs decline, they would have to do so dramatically to make automating tasks with AI affordable

We’ve seen a literal 10x decrease in cost from gpt-4-32k to gpt-4o in a single year of AI development (3:1 cost blend). And that ignores that sonnet-3.5 is 50x cheaper than gpt-4-32k while getting better scores on pretty much all benchmarks?

> the human brain is 10,000x more effective per unit of power in performing cognitive tasks vs. generative AI

Patently false, we’re not untethered brains floating around and require shelter, food, and a ton of other energy intensive requirements to live, and an AI system can perform a task that it is designed to do easily 10-20x faster than a human could.

If anything this makes me more bullish about AI systems having a positive ROI; the criticisms they have are based on extraordinarily (if not nefariously) dumb assumptions.

Every investment company out there knew this from day one. They were riding the hype and gains. Now the late and individual investors will pay for the losses while the big investors start moving cash to the next hype.

In real time we’ve seen “LLMs are the future” becomethe AI research into AGI is the future”. What’s the next movement of the goal post?

Seeing that AI is being subsidized by investors I wonder how long it'll take for said investors to want a return on their investments and if AI is too expensive for anyone to actually want to use it. Because I don't think a lot of people would pay the actual cost it takes to run AI.

Is this not expected? Every hype encounters a “Trough of Despair”. I think costs need to decrease dramatically before the current state of AI is worth the money that is being dumped into it. Both in terms of GPU costs and electricity because it’s insane.

It's quite likely to be like the 1999 dot com boom and bust. Investors put a lot of money into things like Webvan which went bust in 2001 while a lot of the value ended up in the likes of Google which didn't float till 2004.

The return on investment on anything that’s over-bid due to hype is usually quite poor.

I don’t see strong reasons to think AI will be different than tulips or South Sea investments in that regard.

Heresy!

Kind of a little surprised that they’re coming right out and saying it at this point; I didn’t think we were at that point in the hype cycle just yet.

I know that we "want" investment in AI, but I'd be perfectly happy if Wallstreet stays the heck out of this whole situation. Every time investment money gets into a topic - not company, not an industry, a topic - it becomes an absolute mess.

Investment feels like a micromanager that won't let you do your work.

I'm sort of dreading the next year or so at big corps, in which engineers will hear from their management: "We've made a big investment in AI, and need you to make this work".

When you’re selling shovels it’s easy to pump and dump and then say weeellllllll it might not be as big of a gold deposit as we thought it was.

Integration of LLMs with existing compute and data systems will have an excellent return on investment if done correctly.

The current brain dead spitball method of shoehorning a chatbot interface on top of every single existing GUI application is not that.

Here I am still waiting for all those amazing innovations that the blockchain will bring us.

If we replace bankers with AI the returns, the returns may be better.

AI winter is coming! Fascinating how the bust part "boom-bust" cycle invariably comes for every promising tech.

On a good side, we have finally have the first generation of AGI. Give it another ten years of improvements before we reach the next AI boom.

What a waste of humanity's resources.