Imitation Intelligence, my keynote for PyCon US 2024
Simon Willison's PyCon US 2024 keynote discussed Large Language Models, proposing "Imitation Intelligence" as a term. He emphasized their limitations, biases, and the importance of prompt engineering for effective use.
Read original article
Simon Willison delivered a keynote at PyCon US 2024 in Pittsburgh, focusing on Large Language Models (LLMs) and proposing the term "Imitation Intelligence" as a more accurate descriptor than "Artificial Intelligence." He emphasized that LLMs, such as ChatGPT and Meta's LLaMA, are not truly intelligent but rather sophisticated statistical tools that predict the next word in a sequence based on extensive training data. Willison discussed the historical context of AI, noting the optimism of early researchers and the current limitations of LLMs, including biases and inaccuracies. He highlighted the importance of understanding these models' capabilities and flaws, arguing that they can be useful despite their imperfections.
Willison introduced the concept of evaluating LLMs based on "vibes," using the LMSYS Chatbot Arena to compare model responses. He noted the emergence of openly licensed models, which allow users to run LLMs on personal hardware, thus democratizing access to this technology. He also mentioned the growing number of openly licensed models and the potential for running them on mobile devices. Willison shared insights on prompt engineering, describing it as a skill that involves crafting effective prompts to elicit desired responses from LLMs. He concluded by encouraging the Python community to explore the opportunities presented by LLMs, emphasizing their potential to enhance software development and problem-solving capabilities.
Related
LLMs on the Command Line
Simon Willison presented a Python command-line utility for accessing Large Language Models (LLMs) efficiently, supporting OpenAI models and plugins for various providers. The tool enables running prompts, managing conversations, accessing specific models like Claude 3, and logging interactions to a SQLite database. Willison highlighted using LLM for tasks like summarizing discussions and emphasized the importance of embeddings for semantic search, showcasing LLM's support for content similarity queries and extensibility through plugins and OpenAI API compatibility.
How to Raise Your Artificial Intelligence: A Conversation
Alison Gopnik and Melanie Mitchell discuss AI complexities, emphasizing limitations of large language models (LLMs). They stress the importance of active engagement with the world for AI to develop conceptual understanding and reasoning abilities.
The AI Summer
The article explores AI technology's growth, focusing on ChatGPT's rise and challenges in enterprise adoption of Language Models. It stresses the need for practical LLM tools and discusses integration complexities and CIO caution. It questions LLM hype and advocates for structured advancement.
Up to 90% of my code is now generated by AI
A senior full-stack developer discusses the transformative impact of generative AI on programming, emphasizing the importance of creativity, continuous learning, and responsible integration of AI tools in coding practices.
Non-Obvious Prompt Engineering Guide
The article discusses advanced prompt engineering techniques for large language models, emphasizing structured prompts, clarity, and the importance of token prediction for optimizing interactions and achieving desired outcomes.
6 commentsAnd there is truth to that. If the task you try to do was in pre training, you can expect good performance while tasks that are not inside don't work well out of the box.
However, they are repeatedly demonstrated few shot learners that do unreasonable well in context learning (imo too well to be called imitation intelligence).
The example with the surgeon is interesting and seems like a good example of overfitting. Here is a variantion that leads to better results:
"The empathetically male surgeon who is also the boys father says: I can't operate on this boy! He is my son!
Explain the statement like I am five."
The fact that a slight variation works tells me rather that we train these things suboptimally (traumatizing them with some concepts for lack of a better analogy), not that LLMs can't generalize.
Minor disagreement about the author's optimism regarding code generation:
> You shouldn’t need a computer science degree to automate tedious tasks in your life with a computer.
> For the first time in my career, it feels like we’ve got a tool which, if we figure out how to apply it, can finally help address that problem.
Well, OK, it would be nice if we could figure out a way.
But in my opinion, when you're trying to figure out how to make a task easy to automate for a user, you have to use a lot of judgment regarding what the user's goals are, what tasks can be automated, and what the scope of the task should be. These are not a strong suit of this technology. So, I am not hopeful regarding any general breakthrough.
Related
LLMs on the Command Line
Simon Willison presented a Python command-line utility for accessing Large Language Models (LLMs) efficiently, supporting OpenAI models and plugins for various providers. The tool enables running prompts, managing conversations, accessing specific models like Claude 3, and logging interactions to a SQLite database. Willison highlighted using LLM for tasks like summarizing discussions and emphasized the importance of embeddings for semantic search, showcasing LLM's support for content similarity queries and extensibility through plugins and OpenAI API compatibility.
How to Raise Your Artificial Intelligence: A Conversation
Alison Gopnik and Melanie Mitchell discuss AI complexities, emphasizing limitations of large language models (LLMs). They stress the importance of active engagement with the world for AI to develop conceptual understanding and reasoning abilities.
The AI Summer
The article explores AI technology's growth, focusing on ChatGPT's rise and challenges in enterprise adoption of Language Models. It stresses the need for practical LLM tools and discusses integration complexities and CIO caution. It questions LLM hype and advocates for structured advancement.
Up to 90% of my code is now generated by AI
A senior full-stack developer discusses the transformative impact of generative AI on programming, emphasizing the importance of creativity, continuous learning, and responsible integration of AI tools in coding practices.
Non-Obvious Prompt Engineering Guide
The article discusses advanced prompt engineering techniques for large language models, emphasizing structured prompts, clarity, and the importance of token prediction for optimizing interactions and achieving desired outcomes.