Chatbots Are Primed to Warp Reality

I can definitely see people becoming overly trusting of LLM output, but I have a feeling it might be a self-correcting problem over time.

Initially when working with an LLM if you start with a problem it knows well, it's likely to give good results and if some minor hallucination creeps it you may well not notice and accept it based on earlier results being right.

However it's quite likely you'll hit a wildly wrong statement at some point, and that tends to break the illusion, and hopefully people who have that experience will start being more skeptical of what they're being told by LLMs.

> suggests that the solicitous, authoritative tone that AI models take—combined with them being legitimately helpful and correct in many cases—could lead people to place too much trust in the technology.

Haven't there been reports lately that people don't trust the news? I'd think that the search engines' AI models would suffer the same fate given similar levels of accuracy.

> No one person, or even government, can tamper with every link displayed by Google or Bing.

Well, Google or Bing can.

Has The Atlantic written any articles about how Google's skewed top results also warps reality?

Chatbots are warping reality. There is a growing number of people who use them as confirmation bias machines because most LLMs still do not disagree very well. And people enjoy being told they are right in an authoritative tone. We now get really angry if an LLM is "patronizing". We expect that it will tell us what we want to hear. And some of that anger is perhaps justified in the most egregious cases of information censorship for the sake of "Silicon Valley ethics", but not all of the anger.

As a software developer, I meet clients nowadays that dismiss all actual implementation issues because an LLM told them their idea is good. They will send screenshots from ChatGPT and shut down any meaningful discussion about the reality of the situation. I've also seen the older generation fall prey to many blogspam websites pumping out conspiracy content with LLMs, and sometimes even quite young people. I think we have all seen the blogspam situation.

I think this and echo chambers, or more generally — seeking unnatural levels of validation — is turning into something pathological. Either in the sense that it's pathology to seek only validation and nothing else, or also in the sense that this leads to stunted growth and inability to see nuance. We need some disagreement to properly come of age, to gain wisdom, and to understand the world around us. Developmental psychologists like Erik Erikson place conflict of ideas[0] at the center of a person's mental growth. But many people these days insulate themselves as much as they can from such conflicts. If this continues, it will be transformative for humanity, and very likely not for the better.

[0] https://www.simplypsychology.org/erik-erikson.html

Paywall. We need a flair for pay walled articles

I tried to ask my amazing google AI to send a text message today, and it couldn't fucking do it

The tech still sucks, and everyone loves to ignore that it is constantly wrong

ask an AI to help you with a Makefile to see what I mean lmao

People so strongly wish that AI exists that they will believe anything. Pretty sad from a sociological and, much later, historical point of view.