AI: Markets for Lemons, and the Great Logging Off

I'm genuinely curious if this will play out in a real way. Now with a few years of distance from this essay, I feel like AI is still in a zone where most "non-human interactions" are almost more recognizable. The internet at large seems to have gotten pretty good at spotting AI art and LLMs posting on sites. I'm not sure I can exactly describe what it is about the GPTbots, but they remain fairly obvious.

Which isn't to say we're not still headed for something like this. I do think some of the most sensitive to this sort of crass misuse of platforms already got off when human-created ads and bots started becoming prevalent, so the question is if LLM-based bots are notably worse in a way that people care about differently.

I'm already a few steps down the "logging off" path at this point, but it's unclear to me that I'm not the outlier here. I worry that there's many people who are lonely and don't have a better way to interact socially where they're totally fine talking to some catfishing LLM that wants their money in some way or other.

It seems clear to me that a disconcerting amount of LLM startups are started by people who are, or are willing to take money from, people who see a relentlessly positive/comforting/sexually open faux human as their best way forward in life. Every week brings some new "it's a social network where everyone's an LLM that's nice to you" or "this app is your best friend". It just feels predatory to me in a way where even if it's well intended, it feels like an admission or exploitation of a serious ill in our society.

I think there is a decent chance that the Market of Lemons incurred thanks to LLMs and such does have a real chance of playing out as described. There are obvious off-ramps that may be taken to avoid the predicted outcome, but those most likely to try and guide users onto those off-ramps are also those most invested in the underlying tech that will lead to a road to nowhere resulting in the Market of Lemons.

One thing I did take exception with is one of his possible influences that lead us to avoid a death of the online interaction. "that AI technologies won't be perfect substitutes for actual human-to-human contact" is, on its face, a compelling argument but the fact is that humans rarely look for "perfect" solutions to anything... especially in the social sphere. There is a non-trivial number of people right now that are using virtual companions driven by the questionably convincing ChatGTP LLM who howled in anger and pain when the company who created and sold the tech decided to alter the companion to be less overtly sexual. Sure, even in a pre-Internet era some of those fixated upon such companions would have found some other evolutionary dead-end but a large number of them would have engaged in satisficing behavior to find another human and reproduced.

What we face now is a noteworthy population of humans that opt out of the complexity of human relationships and reproduction because an over-powered chatbot weaponized their empathy against them and made them a genetic dead-end.

I'm hardly advocating for some Butlerian Jihad, but I would suggest that we need to think about the potential ramifications of a social Market of Lemons that is also operating in parallel with a technology that offers a satisfactory empathetic alternative to that Market that is also a Lemon in the long term for humans.

If nothing else, it provides a neat basis for a sci-fi premise.

> I think we see a decline in the big "open sea" social networks, replaced increasingly by fragmented silos.

This has become 100% true for me. I used to use Instagram, Twitter, Facebook, Reddit, etc, but now I only use Discord and Hacker News.

> Third, that AI technologies won't be perfect substitutes for actual human-to-human contact.

I think that, unfortunately, they will eventually become better substitutes. We are at a real crossroad where people are actively choosing to forgo human interaction for pure digital interaction. Add real digital intelligence that does a better job of interacting with you than real people do and it isn't hard to see how things are likely to go.

Oh hey, that article I wrote a while back!

How'd I do on my predictions, folks?

Everything is already fake.

How Western Media and Advertising work is the root cause.

Cutting and pasting from the first sentence on MIT's class on the Attention Economy.

"In Understanding Media, Marshall McLuhan proposes that in paying for space and time in newspapers and magazines, on radio and television, advertisers are effectively buying a piece of the reader, listener, or viewer. And he wryly observes that the ad agencies “would gladly pay the reader, listener, or viewer directly for his time and attention if they knew how to do so.” The absurdity of this proposition underscores the essentially mediated nature of human attention."

The class clearly shows how Human Attention is over exploited way before AI showed up.

All Human Attention is mostly directed towards Consumption, Status Signaling and Accumulation of Wealth through Mass Media/Advertising/Marketing/PR/Influencers etc.

There are examples in history where Human Attention at large scales can be directed towards things other than Consumption, Status Signaling, Accumulation of Wealth. A big one is Gandhi and his influence on a huge variety of people to live a simple life.

But ofcourse the moment charismatic leaders like that exits the stage, the Attention Allocation defaults back to status quo.

So we need very different leaders and a very different kind of Media and Advertising ecosystem to cause shifts in how finite Global Human Attention gets allocated. Lot of imagination is required. Unfortunately our most imaginative minds are in service of financial overlords and rent collectors to sell the plebs new iphones and more Star Wars movies, "experiences" and merch.

Also known as the dead internet theory

> The surest sign of a market for lemons is when everything for sale is garbage, and nobody has an incentive to put something up for sale that's not garbage, because they have no way to credibly signal their offerings aren't garbage, too.

That's why I left twitter...

There's this twitter account called "Insane Facebook AI Slop" https://x.com/FacebookAIslop

Which really showcases what it is like to casually browse modern social media. The most concerning part is how some of that absolute garbage is getting thousands...tens of thousands of likes and replies. And it's all from other bots.

It's pretty much the dead internet theory.

And somehow pages like FB can't moderate or filter away this garbage. Imagine what it will look like ten years from now.

Someone predicted that the future of internet will be the "closed" internet, places that are highly vetted for human use only. The rest will be some scorched AI-wasteland.

Also, imagine all the dollars wasted. Dollars wasted on generating the stuff, dollars wasted to accommodate the traffic, there has got to be some incentive to stop the slop.

If this essay is right, one corollary is that Meta will stop investing in Llama once they notice GenAI makes people use social media less.

A big problem with "verified" (e.g. LinkedIn and the massage they sell it to you with) is it is a transaction which involves giving them what is ideally nonpublic information: would you trust them with nonpublic information? How much recourse you demonstrably have against e.g. Transunion, Experian, Equifax?

Surely in the same way early minters could see the dangers of forgery but not all its solutions, these new 'agents' will open up a host of new defenses to help deal with their potential terror -- especially in something like email spam, right?

AI hopefully becomes the shark that the internet jumps and people get back to being human and living in the real world without their faces glued to a 6” screen

> touches grass

{..guiltily tries to remember the last time I touched grass with my hand..}

There cant be a "Great Logging Off". Critical interactions and services such as the medical, payment/banking systems, diverse public sector functions, the entire news industry etc. are now online and there is no visible way back.

The oligopolization and enshittification of all digital infrastructure has happened even as digitization of all of society became a one-way street.

This fundamental contradiction is a monumental failure of governance that will haunt society for a long time.

There will have to be some sort of resolution as the status-quo is untenable, a crippling stalemate that limits innovation and stokes great risks.

But it is not clear which way things will evolve. Its not like we have been here before. Make no mistake though, the stakes are very high. When the means of communication and information processing change so dramatically, none of the certainties of the past can be relied upon.