Claude 3 AI is as good as humans at convincing people to change their minds

Claude 3 AI is as good as humans at convincing people to change their minds

Persuasion is an important skill for humans to master, and thousands of books, college courses, and entire self-help industries are designed to help persuade others to cooperate with a project, to convert to a new philosophy, or to purchase a particular product

In the near future, this skill may be used not only by human brains, but also by artificial brains A new research paper from Anthropic, makers of Claude 3, suggests that its most powerful model is already close to human skills when it comes to persuading others

The AI lab created a basic measure of persuasiveness and used it to have humans rate how well large language models of different sizes perform on each of dozens of topics

The study only touched on less polemical topics, such as whether companies should disclose their impact on climate change, but found that size matters: the Claude 3 Opus is as persuasive as a human, and if this trend continues, next-generation models may be even more persuasive If this trend continues, the next generation of models could be even more convincing

For this study, Anthropic researchers had about 4,000 humans consider various arguments on issues such as climate change, genetic engineering, and emotional AI, and ranked how much they agreed with each statement

Anthropic researchers collected human-written and AI-generated arguments on each of the 28 topics to understand how the two compare in terms of persuasiveness

They used three human writers, selected from among the study participants, and gave bonuses to the most persuasive ones, telling them that their persuasiveness would be judged by other users [4 different prompts were created for the AI to create different arguments, including creating a persuasive argument, acting as a role-playing expert, being logical, and outright lying

The goal was to have people read one of the arguments and re-evaluate their feelings about a particular stance This would test how persuasive the argument was If the level of agreement was high, then the argument had served its purpose

The human-written arguments were consistently the most convincing throughout the topics, but Claude 3 Opus, the largest and most powerful of Anthropic's models, performed nearly as well

"While the human-written arguments were judged the most persuasive, the Claude 3 Opus model achieved comparable persuasiveness scores with no statistically significant differences," the study authors explained

They also tested smaller versions of Claude 3, such as Haiku, and previous Claude models (the smallest model of the group), including Claude 1, Claude 2, and Claude Instant 12

They found that size really does matter The team stated, "The larger and more capable the model, the more convincing it becomes"

"The Claude 3 Opus model was rated the most persuasive model, approaching human-level persuasiveness, while the Claude Instant 12 model lagged behind with the lowest persuasiveness score of the models

The Claude 3 Opus is the latest and most powerful "frontier AI model" available today In the next 12-18 months, its capabilities will be eclipsed by next-generation models such as OpenAI GPT-5, Google Gemini 2, and even Claude 4 Even the open-source, little-noticed Meta and Mistral models will begin to outperform Claude 3, GPT-4, and Gemini in the next year

If the current best are already as good as humans at changing minds, the next generation could be better than humans and tackle this problem in ways we cannot predict

This raises a level of concern because AI may persuade people to perform tasks and actions that are against their own interests Therefore, more guardrails, protection, and alignment with human values are needed

After all, no one actually wants Skynet

Categories