Large language models don't behave like people, even though we expect them to

Large language models (LLMs) are powerful tools capable of performing a wide range of tasks, but they do not behave like humans, despite common expectations. Researchers from MIT have proposed a new framework for evaluating LLMs based on how humans perceive their capabilities. This approach, termed the human generalization function, examines how users form beliefs about an LLM's performance after interacting with it. The study found that users often misjudge LLMs, leading to overconfidence or underconfidence in their abilities, particularly in high-stakes situations. The researchers conducted a survey to analyze how people generalize LLM performance across various tasks, revealing that users struggle to predict LLM responses accurately compared to human responses. This misalignment can result in larger models performing worse than smaller ones in critical scenarios. The findings suggest that as users gain more experience with LLMs, their understanding may improve. The researchers aim to further explore how human generalization can be integrated into LLM development and performance measurement, emphasizing the importance of aligning model capabilities with user expectations to enhance real-world application.

- Large language models do not behave like humans, leading to misjudgments in their capabilities.

- A new framework evaluates LLMs based on human perceptions of their performance.

- Users often overestimate or underestimate LLM abilities, especially in high-stakes contexts.

- Experience with LLMs may improve user understanding over time.

- The study highlights the need for better alignment between LLM capabilities and human expectations.