The serious science of trolling LLMs
Trolling large language models manipulates responses for attention. Vendors address troll transcripts to maintain human-like illusions for commercial use. Trolling reveals LLM limitations, evolving into a scientific pursuit for model understanding.
Read original article
The article discusses the practice of trolling large language models (LLMs) by manipulating prompts to generate nonsensical or outrageous responses for social media attention. While some view this as a pointless activity since LLMs do not learn from mistakes like humans, the author argues that vendors invest resources in addressing viral troll transcripts to maintain the illusion of human-like responses. The deceptive nature of making LLMs appear more human is crucial for commercial applications, where customers expect transparency about the models' limitations. Trolling is seen as a way to reveal the shortcomings of LLMs and differentiate reasoning from data recall. The author suggests that trolling is evolving into a legitimate scientific pursuit, shedding light on the true capabilities and limitations of these models. The article emphasizes the importance of understanding when and how LLMs fail to ensure their responsible use in various applications.
Related
Large Language Models are not a search engine
Large Language Models (LLMs) from Google and Meta generate algorithmic content, causing nonsensical "hallucinations." Companies struggle to manage errors post-generation due to factors like training data and temperature settings. LLMs aim to improve user interactions but raise skepticism about delivering factual information.
Overcoming the Limits of Large Language Models
Large language models (LLMs) like chatbots face challenges such as hallucinations, lack of confidence estimates, and citations. MIT researchers suggest strategies like curated training data and diverse worldviews to enhance LLM performance.
7 commentsI really don't agree. Watching AI fail spectacularly is a great bread and circus, but identifying something's deficiencies is not the same as fixing it. The thing these trolls get so damningly right is their denouncement of AI; we can't rely on shit text-generators that fly by the seat of their pants. The insinuated "but it's useful and does real work" shtick is over; I don't even see people talking about RAG value-add anymore.
The bubble popped, and now expectations are returning to normal. It's less of an enlightenment period and more of the "AI winter" that we've always known was going to occur once we exhausted the current scaling limits. The trolls are scratching beneath the surface and proving the prophecy right.
> In short, I believe that the revealed objectives of LLM vendors differ starkly from what they say. They pour considerable resources into responding to every trollish LLM transcript that goes viral on the internet — and it’s useful to ponder why.
I'd be pretty interested in reading at how people do poke at LLMs and what this actually reveals about the vendors priorities.
https://youtu.be/kNJNV0GXO-s?si=3TKUpFoFHV7FNSHR
The AI has no idea where it is going before it starts outputting. If it ends up with a sentence that can’t be completed sensibly, it will output nonsense.
I think a better model needs to be able to plan out the idea to convey in advance and possibly backtrack.
Trust seems a big factor here.
wut. I was pretty sure they do learn from mistakes. but they call it 'training'
Related
Large Language Models are not a search engine
Large Language Models (LLMs) from Google and Meta generate algorithmic content, causing nonsensical "hallucinations." Companies struggle to manage errors post-generation due to factors like training data and temperature settings. LLMs aim to improve user interactions but raise skepticism about delivering factual information.
Overcoming the Limits of Large Language Models
Large language models (LLMs) like chatbots face challenges such as hallucinations, lack of confidence estimates, and citations. MIT researchers suggest strategies like curated training data and diverse worldviews to enhance LLM performance.