News of new AI capabilities is now a steady drumbeat accompanying our lives.
Another benchmark broken by the latest model, another AI-generated video that you cannot tell from real footage, or another paper announcing how AI systems are getting pretty good at hacking their way around the internet or passing as human in extended conversations. There is no particular need to invoke transformative AI or superintelligence to agree that the most recent AI models can now do things long assumed to be decades away.
As an academic studying the societal impacts of AI at Oxford, one area that I pay a lot of attention to — and where models have become really very good it seems — is persuasion: the act of changing people’s attitudes, beliefs and even their actions with nothing more than a conversation. Such persuasion is about changing people’s minds by giving them reasons they can inspect (and of course reject), and is quite different from manipulation, which forgoes that process by deceiving people — even though the two are often and easily conflated.
That AI systems can be persuasive is… pretty big news. If someone had told me five years ago that chatbots could be better than seasoned campaigners at getting people to part with their money, as a recent preprint found, I would have been pretty skeptical. And yet, that is exactly what you can do with the latest models — at least in an experimental setting.
These results have, unsurprisingly, prompted people to come up with all sorts of ideas, as well as quite a bit of excitement and fear. Platforms, governments, or public bodies could move people away from harmful conspiracies or could nudge them toward choices that serve them better, for example, quitting smoking. But, of course, every light side comes with a dark one: persuasive systems could allow politicians to steer how people vote, talk someone into joining a cult, make it easier to scam and swindle people out of money, encourage decisions that put their health at risk, and much more. The most apocalyptic of these scenarios imagine a new dawn of uber-persuasive machines against which we pitiful mortals stand no chance.
But is it really game over? I don’t think so. Persuasion is possible, but I doubt that it will make as much of a difference as some claim. Persuasiveness is actually the “easy” part. What is difficult is getting hold of someone’s attention in the first place. Shifting a view once someone has one is harder still. There are, in other words, a range of bottlenecks that will limit the persuasive power of these systems for the foreseeable future — for good and ill.
Persuasion through AI is real
So, what do we actually know? Our best evidence for AI’s persuasiveness comes from experiments in which researchers pay people to sit down with a chatbot and measure if their views have shifted.
In probably the largest public study so far of its kind, conducted by researchers from the UK’s AI Security Institute (AISI) and elsewhere, more than 42,000 UK participants were asked to argue about various political topics with 19 different language models on several hundred issues. Afterward, the research team measured how far people’s attitudes had moved compared to a control group. The answer was around 10 points on average on a scale running from zero to 100. That’s roughly equivalent to walking into a bar mildly opposed to the idea of beer, having a chat with the barman, and then walking out feeling lukewarm about beer, but overall still preferring white wine. That conversation was more effective than single, static messages — the kind of argument or claim you regularly see in a social media post or TV ad — by about 41% to 52% depending on the AI model involved.
While having a bigger model helped, post-training — the fine-tuning AI firms do once a model has been built to shape its behavior — towards better persuasion mattered even more. So did a specific persuasion strategy: prompting the model to aim for information density. Models that rammed more facts into their answers were better at persuading people, while personalizing answers to people’s attitudes and demographics — a recurrent worry ever since Cambridge Analytica — did not do very much.
Meanwhile, in a study published last year in Nature, participants were randomly picked to talk to an AI that supported one of the leading candidates during election campaigns in the US, Canada and Poland. The study found that AI conversations had a stronger effect on changing people’s candidate preferences than traditional video ads — and they found a similar mechanism as the AISI study, with relevant facts and evidence being the thing that persuaded people.
The immediate implications of such a finding, beyond the fact that AI models can do something long thought to be the prerogative of humans, are twofold. For one, persuasion is now, at least in theory, dirt cheap. You no longer need as many messy and costly humans to do it. In a recent paper on AI in political campaigns, Zhongren Chen and colleagues estimated that once you have accounted for the price of actually reaching people, LLM-based persuasion costs between $48 and $75 per persuaded voter compared with $100 for traditional campaign methods. Instead, you have persuasion on tap, anytime, in any language and at whatever scale someone is willing or able to pay for — quite different from the persuasion happening where people simply interact with a model for things like work or pleasure.
The AISI study also had another significant finding: Those models tuned hardest for persuasion also made more false claims as part of the process — in the most persuasive setting, nearly a third of the model’s claims were inaccurate, something that is obviously no problem if you are a scammer or propagandist, but does matter quite a lot if you have good intentions. However, the authors did note that inaccuracy seemed to be a byproduct of greater information density, not a cause of persuasion. Telling a model to make things up did not make it more persuasive.
Why the lab isn’t the real world
The AISI study sparked a big brouhaha when it first came out, but experts — including some of the study’s authors — have since pointed out that the findings need to be treated with a pinch of salt when applied to the real world. The biggest bottleneck is deceptively simple: for such persuasion to work, you need to get people to pay attention.
Experiments solve this via forced exposure: people are being paid to engage in back-and-forth conversations, otherwise it would be difficult for researchers to reliably detect effects. In the AISI studies, people had to engage in at least two conversation turns to receive an incentive, although on average participants talked to the chatbot for about seven turns (or roughly nine minutes). Yet, this is a form of exposure that you just cannot assume in the real world. In the wild, people’s attention is voluntary and scarce.
To see why this is, just remember how little time is left (in your life) to actively pay attention, accounting for the factors that shape the humdrum of our day-to-day existence.
Take the average adult in a highly online country. Depending on their quality of sleep, they will be awake for roughly 16 hours a day. And these 16 hours are chock-full of… well, things, things that command a lot of time and attention. This person eats, cares about their personal hygiene multiple times a day (hopefully), they run errands, they socialize, they work, or — lo and behold, and please look away now my dear Americans — they might simply do nothing. Depending on how you count, you are very quickly at 10 to 14 hours of those 16 already used up, before we even get to discretionary media use as part of people’s leisure time. According to the latest American Time Use Survey, people aged 15 and over averaged about 5.2 hours of leisure and sports per day in 2025, TV alone accounted for 2.6 of these hours, with other activities such as messaging, listening to music also falling in that category.
Of course, some media use — where we would typically encounter chatbots or messages generated by AI — is fitted around or layered on top of people’s other activities. You listen to something while traveling, you scroll your phone as you wait for your doctor’s appointment, you might check social media as you eat your meal or sit on the toilet. But a lot of this is partial, interrupted, or incidental. In other words, even if people’s total screen time might be four to five hours, the time available in a given day for direct, intentional, and, most importantly, attentive information consumption is a lot smaller than that. By my own back-of-the-envelope calculation perhaps just 30 to 60 minutes on an average day.
The competition for your attention within that tiny window — in a world where information is cheap and abundant — is, to put it mildly, insane. And it’s unlikely people will voluntarily spend such time mostly “with a chatbot that keeps trying to get them to change their minds and who wants to tell you that you are wrong,” as cognitive scientist Hugo Mercier put it recently in a talk in Brussels.
Or as Ben Tappin, one of the lead authors on the AISI study and an assistant professor of psychology at the London School of Economics and Political Science, told me: “Most people in the real world are not going to spare 10 minutes for a very information dense political conversation with a chatbot. I think those studies are probably overestimating persuasive impact by a fairly large margin for these exposure reasons.”
The same challenge — truly getting people’s attention — is also true for static messages. No matter whether you are a brand advertising a product on social media or a politician hoping to win someone’s vote by buying ads on television, no matter if you throw AI at it to make it more persuasive, at the end of the day these messages are competing in a crowded environment. People need not only to see or hear them but also pay sufficient attention to take them in. And then you still run the risk that all the money and effort are wasted because your message reached people who simply cannot be swayed. A very persuasive ad for cat food is utterly wasted on someone who doesn’t have a cat.
Competition, competition, competition
Of course, there will be people who end up in a conversation with a persuasive chatbot or end up being exposed to content meant to persuade them and who are at least partly receptive to the message. For these, then, surely the AI magic will work, no?
Well, it depends. Here, bottleneck number two kicks in: We generally don’t consume information or hear persuasive arguments in isolation or against a blank slate. In a lab setting, persuasion happens largely free of other influences. Outside of such sheltered settings this is not the case.
We constantly get bombarded by conflicting messages that want to persuade us in some way or the other. Ads and influencers want to sell us things (“Take more vitamin pills to LiveYourBestLife™️”), while experts try to keep our worst impulses in check (“Thou shalt not take too many vitamin pills”), news media may have their own agenda (“Science says vitamin pills kill” or “Vitamin pills will make you woke”), while family members (“Just go out into the sun”), friends (“I swear by homeopathy”), and random strangers (“I will not put this big pharma stuff in my body”) will all present us with more or less convincing, sometimes plainly contradictory messages of their own. Given this cacophony of voices, it is not guaranteed that persuasive effects from AI systems will necessarily always last very long.
Politics, arguably one of the areas where persuasion matters most, is a case in point. Any persuasion at scale here is bound to meet attempts of counter-persuasion — in fact, this is already how politics works, with candidates trying to make a case for why they are right in their diagnosis of what needs to change and why they should be allowed to take things in a different direction.
People are different
A third major bottleneck which we shouldn’t lose sight of is the fact that people and topics are not homogeneous.
With some beliefs it is cheaper and easier to change someone’s view. Not a lot is at stake for most people when I try to convince them that white clothing will be better in hot weather because it better reflects the sun. If you didn’t already know this, you might simply say “Sure, this sounds logical and I don’t mind white clothes” (unless you are Batman). Alas, there are some areas where changing one’s view does incur a cost — which might be personal (accepting that your favorite politician’s latest policy cost lives might be painful for you to admit) or social (accepting that your favorite politician’s policy was disastrous marks you out as a pariah in your circle of friends).
And not only is it harder to change people’s views on some topics than on others, even if someone changes their mind, it does not follow automatically that they will also change the way they act, something that researchers know as the attitude-behavior gap.
People are different, too. Not only do we not all pay attention equally, we are also not all equally easy to persuade. After all, we are not just empty vessels waiting to be filled with other people’s views. The knowledge and attitudes we accumulate like sediment over a lifetime shape how receptive we are to having our views changed.
It depends on how set our attitude is and what we find convincing. We are not just influenced by the strength of an argument but also by who is speaking and how this makes us feel. This might also be said for the evidence-dense, highly structured style that turns out to be so successful at persuasion in experimental settings. Some people like this mode of arguing, but it might actually be a bit of a turn off in the wild. And we humans can be a suspicious bunch, too.
Where and how does all this matter?
All of which leaves us with a final question: How much should we care?
I am not arguing that AI cannot be persuasive — it can be and the evidence base is pretty solid by now — but we fool ourselves if we think that a model capability on this front is all that matters. In experiments, exposure and attention are bought and as such are fixed, with persuasiveness the only element that can meaningfully change. In real life, persuasion is malleable, and exposure — getting to people — matters far more. As Tappin has argued in one of his essays, a piece of content that is mediocre in terms of persuasiveness but nevertheless receives high exposure often has greater impact than something highly persuasive that receives only mediocre exposure.
But there is a different way to look at all this.
Automated AI scam calls, AI-optimized political ads, or people using AI to make their own arguments more persuasive all describe situations where things are pushed at people — these are all expensive to carry out and are limited by people’s time and attention. It’s also where our mechanisms of epistemic vigilance usually work reasonably well.
“People are very averse to being persuaded when it seems like an overt persuasion attempt,” Tappin said. A robocall with a persuasive AI attached to it might make very good arguments about why you should buy an insurance policy, but if you don’t like getting random calls, especially from an AI trying to sell you something, that won’t matter very much. And whether or not the purpose is ethical, it seems to really piss some people off because it’s too much (“I had five calls already”), feels intrusive (“Where the heck did they get my number?”) or because people might not particularly enjoy talking about something very personal with a machine that they — looking at public attitudes toward AI — often have mixed feelings about.
Yet, the exposure argument also cuts the other way. Increasingly large numbers of people now use these systems on a daily basis for all sorts of things, including for information — including in the run-up to elections. In those kinds of situations the exposure problem has been “solved” by the user.
While regulators take aim at persuasion that is forced on to people — the US Federal Communications Commission (FCC) ruled AI-voiced robocalls illegal and the EU’s AI Act now stipulates that people need to know when they are talking to a machine — both measures likely do not do much in situations where people voluntarily engage with a chatbot and where no one is fooled by what they are doing.
But if a model is skewed slightly in one direction or the other, as AI systems tend to be, and makes generally persuasive arguments over a longer stretch of time — not because it was manipulated to do so but just because it writes and argues better than your uncle Howard over family dinner — we are faced with a situation where we might get to see effects that matter in the real world. For now, these seem to be depolarizing, but that might not always be the case.
This is well-reasoned - thanks for this. I think the key (beyond the point you make several times, rightly, about *capturing attention* being the hard part) is also that the vectors for capturing attention are likely to be highly orthogonal to the vectors for what to do with attention once it's captured. Machine intelligence is likely to get very good at both, I'd conjecture, over quite a long T — but humans are still going to be very good at it too, with likely 'countermeasures pressure' priced in along various timelines (e.g. the way that AI 'tics' weren't as costly as they are now that people look out for them).
I don't think this refutes those who say that once there's ENOUGH raw intelligence, none of this matters, but it does update the relevant timelines accordingly.
The persuasion and manipulation line drawn here is also a legal line in the EU. Article 5(1)(a) of the AI Act prohibits systems that deploy subliminal, purposefully manipulative or deceptive techniques that materially distort behaviour by appreciably impairing informed decision-making and cause significant harm. That prohibition has applied since 2 February 2025. Article 99(3) sets fines up to EUR 35 million or 7 percent of worldwide annual turnover, whichever is higher, with the lower figure applying to SMEs under Article 99(6). What nobody has yet is a decided case marking where measured persuasion ends and material distortion begins.