Gen AI: too much spend, too little benefit?

You know you're in trouble when even GOLDMAN SACHS is like "maybe this thing is a bubble"

It could end up like Chatbot (pre-Gen AI) and 3D Printers. I started a company a few years back, literally with the name "bot" in it, and bought myself a 3D printer. Both are left in the dust.

I don't think Gen AI will be totally bust, but it won't be as promised (anytime soon). Just like in a software project, the last 10% is another 90%.

Perhaps the link should be changed to https://www.goldmansachs.com/intelligence/pages/gs-research/...

A whole lot of large tech companies are proudly shipping their “it demoed well” hackathon projects. Investors are far more eager for these things to get released than customers are to use them.

AI does have some legitimate uses (i.e., Grammarly and GitHub Copilot). It’s just when people say they are going to replace a whole operating system with AI that they are full of shit lol

I had no idea it had reached US$ 1tn Capex, that's an insane amount of money. That's 1/62th of what a Stanford study said would take to move the whole global population completely away from fossil fuels for our energy demands...

There's no way that that amount of projected spend over the next 10 years doesn't result in a lot of new efficiencies in chip production costs and energy usage, particularly a move away from graphics cards to more purpose built chips and algorithms designed for purpose-built chips.

It seems like the killer apps for generative AI right now are:

1) Automating boring reading and writing tasks. Think marketing copy, recommendation letters, summarizing material, writing proposals, etc. LLMs are pretty good at this stuff but these are not many people's core job responsibilities (though they may take up a lot of their time). Consider it a productivity booster for the most part. Some entry level jobs will be eliminated, and this may create problems down the road as the pipeline of employees to oversee LLMs erodes.

2) Code writing tools a la Copilot for certain "boilerplate" code in commonly used languages. I think the impact is similar to (1) where entry level jobs erode and this may impact employee pipelines.

The core problem (as I see it) is that LLMs don't produce outputs good enough to be used without human oversight except on a small subset of tasks. So you end up needing humans (maybe fewer of them) to check the LLM output is headed in the right direction before you let it out into the world.

Consider voice interface LLMs for customer service. When will they get good enough to do the job with real money on the line? If your airline help desk keeps giving away free flights or on the flip side infuriating passengers by refusing allowed changes, can you really use it in production? My sense is they aren't good enough to replace the usual phone tree just yet.

When accuracy doesn't matter that much, LLMs will really shine because then they can be used without a human in the loop. Think some marketing/advertising and especially, especially propaganda.

I think the existing killer apps don't yet have enough money/savings in them to justify the spend. If generative AI technologies can get good enough on the accuracy front to remove humans from the loop in more contexts, we will be talking about much more dramatic value.

What is lacking in AI is a concept of pain and suffering. Those are key to the human experience. If we want a truly capable AI, or even an AGI, we need to add in a suffering feature to these models.

Is there any way to short Open AI and this nonsense that AGI is round the corner? They might prove to be useful to some degree (niche areas) but as soon as I give the LLMs information or problems they haven't seen before they completely struggle to do anything sensible. Try getting them to compare two different JSON documents, they are very confident and produce absolute garbage if these have not been seen at the training stage.

If they can't handle new ideas humans will always be much more useful and these systems are good for references and human learning are not good for creating something new and of value. I've noticed for text LLMs are quite weirdly repetitive and have an empty style that requires a lot of editing to get it into a shape humans would craft.

People will say the improvements are coming but I think most of them have come from more data which is running out. I think one of the most profound things about real intelligence is being able to define and update concepts within your own mind… how to add new information to LLMs in realtime and have that reflected across the board seems intractable given the training and refinement these things would sit upon. There is no clear unit of information about a concept that links to all the other ideas. LLMs seem quite limited by this.

The brain is so much more complex than these algorithms too and so much more flexible, I don't see how a very good encyclopaedia with some fuzzy AI concept extraction capability is in any way the same as the human brain being able to apply and adapt concepts from all around art, science, literature and the human experience.