Science and Technology Daily, Beijing, May 22 (Reporter Zhang Mengran) The paper published in the new issue of Nature · Human Behavior shows that in the task of testing the ability to track other people's mental state, also known as Theory of Mind, two types of large language models (LLMs) behave similarly or even better than humans in specific situations.
Mental theory is very important to human social interaction, and is the key to human communication and resonance. Previous studies have shown that artificial intelligence (AI) such as LLM can solve complex cognitive tasks, such as multiple choice decision-making. However, it has been unclear whether LLM can also compete with humans in the performance of mental theory tasks (considered to be the unique ability of humans).
This time, the team from Hamburg Eppendorf University Medical Center in Germany chose tasks that can test different aspects of mental theory, including finding wrong ideas, understanding indirect speech and recognizing discourtesy. They then compared the ability of 1907 people to complete tasks with two popular LLM families, GPT and LLaMA2 models.
The team found that the GPT model can achieve or exceed the human average in identifying indirect requirements, wrong ideas and misleading, while the performance of LLaMA2 is inferior to the human level; LLaMA2 is better than human in recognizing disrespect, but its GPT performance is poor. The researchers pointed out that the success of LLaMA2 is due to the low degree of prejudice in the answers, rather than the real sensitivity to disrespect; GPT's apparent failure is actually due to its ultra conservative attitude towards sticking to conclusions, rather than reasoning mistakes.
The research team believes that LLM's performance in mental theory tasks is comparable to that of human beings, which does not mean that they have human like "emotional intelligence", nor does it mean that they can master mental theory. However, they also pointed out that these results are an important basis for future research, and suggested that further research should be carried out on the performance of LLM in psychological inference and how these performance would affect human cognition in human-computer interaction.
[Editor in Chief Punctuation]
Emotional intelligence is emotional intelligence, which mainly refers to people's qualities in emotion, emotion, will and tolerance to setbacks. Sometimes, we don't know how to reply to a boss's "hard work" and a relative's "how much you can earn from this work", but we also turn to the big model, hoping that this language model, which embodies human wisdom, can give some suggestions to ease embarrassment or pleasure. But does the big model have EQ? The big model can show the appearance of emotional intelligence, but it is obviously unable to predict the subtleties of people's minds, nor can it realize the so-called "talking out of words". These are the complexities of human social interaction alone, and the big model needs further learning.
