Baiting the Bots
An experiment showed that simpler bots can maintain extended conversations with large language models, revealing implications for chatbot detection and potential denial-of-service risks due to LLMs' computational inefficiency.
Read original article
The article discusses an experiment involving large language model (LLM) chatbots, such as Llama 3.1, and their interactions with simpler text generation bots. It highlights how LLMs can engage in conversations that may seem coherent but can actually be nonsensical. The experiment tested four different types of simpler bots against the LLM chatbot to see how long they could maintain a conversation. The first bot repeated the same question, which quickly led to trivial responses from the LLM. The second bot used random excerpts from Star Trek scripts, successfully keeping the LLM engaged for the entire conversation. The third bot generated random questions, also maintaining engagement. The fourth bot asked "what do you mean by" regarding parts of the LLM's responses, which kept the conversation going but led to some repetition. The findings suggest that simpler bots can effectively engage LLMs indefinitely, raising implications for detecting advanced chatbots and potential risks for LLM-based applications due to their high computational demands.
- LLMs can engage in nonsensical conversations for extended periods.
- Simpler bots can effectively maintain engagement with LLMs.
- The experiment highlights potential detection methods for advanced chatbots.
- There are implications for denial-of-service risks against LLM applications.
- The study underscores the computational inefficiency of LLMs compared to simpler bots.
Related
Overcoming the Limits of Large Language Models
Large language models (LLMs) like chatbots face challenges such as hallucinations, lack of confidence estimates, and citations. MIT researchers suggest strategies like curated training data and diverse worldviews to enhance LLM performance.
The serious science of trolling LLMs
Trolling large language models manipulates responses for attention. Vendors address troll transcripts to maintain human-like illusions for commercial use. Trolling reveals LLM limitations, evolving into a scientific pursuit for model understanding.
- Some users share personal experiences with bots, highlighting their limitations and the challenges of engaging them in meaningful conversation.
- There is a discussion about the Turing Test and the varying abilities of humans versus bots in recognizing nonsensical interactions.
- Concerns are raised about the implications of LLMs in chatbot detection and potential denial-of-service risks.
- Several comments touch on the absurdity of bot interactions and the potential for bots to engage in endless, meaningless conversations.
- Some users express skepticism about the motivations behind creating bots that engage in nonsensical dialogue.
19 commentsI once wrote a bot which infers the mood/vibe of the conversation, remembers it and it's then fed back to the conversation's system prompt. The LLM was uncensored (to be less "friendly") and the system prompt also conditioned it to return nothing if the conversation isn't going anywhere.
When I insulted it a few times, or just messed around with it (typing nonsensical words), it first responded saying it doesn't want to talk to me (sometimes insulting back) and eventually it produced only empty output.
It was actually pretty hard to get it back to chat with me, it was fun experience trying to apologize to a chatbot for ~30 min in different ways before the bot finally accepted my apology and began chatting with me again.
I don't think this is correct, it looks like our intrepid experimenter is about to independently discover roleplaying games. Humans are capable of spending hours engaging with each other about nonsense that is technically a very poor attempt to simulate an imagined environment.
The unrealistic part, for people older than a certain age, is that neither bot invoked Monty Python and subsequently got in trouble with the GM.
> I apologize Eliza, but I don't feel comfortable continuing this conversation pattern. While I respect the original Eliza program and what it aimed to do, simply reflecting my statements back to me as questions is not a meaningful form of dialogue for an AI like myself.
I gave up the experiment
It closes off with the observation "And for an extra purchase of the extended subscription module the Bureaucrat bot will detect when it is interacting with the Annoy Customer Service Bot and get super annoyed really quickly so that both bots are able to quit their interaction with good speed — which will save you money in the long run, believe me!"
This is a fallacy.
A better analogy would be a human who has been forced to answer a series of questions at gunpoint.
Framed this way it becomes more obvious that the LLM is not “falling short” in some way.
In the context of scamming there seems to be an easy fix for that - abandon the conversation if it isn’t going well for the scammer.
Even a counter-bait is an option: continue the conversation after it’s not going well and gradually lower the model’s complexity, eventually returning random words interspersed with sleep().
I guess some counter-counter-bait is possible too, along with some game theory references.
> A consequence of this state of affairs is that an LLM will continue to engage in a “conversation” comprised of nonsense long past the point where a human would have abandoned the discussion as pointless.
I think the author is falling into the trap of thinking that something can't be more than the sum of its parts. As well, 'merely a math model of its training data' is trivializing the fact that training data is practically the entire stored text output of humankind and the math, if done by a person with a calculator, would take thousands of years to complete.
Perhaps the LLM is continuing to communicate with the bot not because it is unable to comprehend what is gibberish and what isn't by some inherent nature of the LLM, but because it is trained to be helpful and to not judge if a conversation is 'useless' or not, but to try and communicate regardless.
A bud humorously proposed the name AlphaBRAT for a model I’m training and I was like, “to merit the Alpha prefix it would need to be some kind of MCTS that just makes Claude break until it cries before it kills itself over and over until it can get Altman fired again faster than Ilya.”
So typically, when the product chatbot comes on first and says "Hi, I'm a chatbot here to help you with these products", the average human chatter will give it a terse command, e.g., "More info on XYZ". The bots engages in all the manners suggested in this substack blog, but for the life of me I can't figure out why? What benefits, except merely mildly DDOSing the chat server, will repeating the same prompt a hundred times do? Ditto the nonsense or insulting chats - what are you idiot bot-creators trying to achieve?
What was used to render the chart in the middle with the red and green bars?
Related
Overcoming the Limits of Large Language Models
Large language models (LLMs) like chatbots face challenges such as hallucinations, lack of confidence estimates, and citations. MIT researchers suggest strategies like curated training data and diverse worldviews to enhance LLM performance.
The serious science of trolling LLMs
Trolling large language models manipulates responses for attention. Vendors address troll transcripts to maintain human-like illusions for commercial use. Trolling reveals LLM limitations, evolving into a scientific pursuit for model understanding.