This is well-reasoned - thanks for this. I think the key (beyond the point you make several times, rightly, about *capturing attention* being the hard part) is also that the vectors for capturing attention are likely to be highly orthogonal to the vectors for what to do with attention once it's captured. Machine intelligence is likely to get very good at both, I'd conjecture, over quite a long T — but humans are still going to be very good at it too, with likely 'countermeasures pressure' priced in along various timelines (e.g. the way that AI 'tics' weren't as costly as they are now that people look out for them).
I don't think this refutes those who say that once there's ENOUGH raw intelligence, none of this matters, but it does update the relevant timelines accordingly.
The persuasion and manipulation line drawn here is also a legal line in the EU. Article 5(1)(a) of the AI Act prohibits systems that deploy subliminal, purposefully manipulative or deceptive techniques that materially distort behaviour by appreciably impairing informed decision-making and cause significant harm. That prohibition has applied since 2 February 2025. Article 99(3) sets fines up to EUR 35 million or 7 percent of worldwide annual turnover, whichever is higher, with the lower figure applying to SMEs under Article 99(6). What nobody has yet is a decided case marking where measured persuasion ends and material distortion begins.
The AISI study's own numbers matter more than the headline caution suggests. Forty-two thousand participants, roughly a 10-point attitude shift, at 48 to 75 dollars per persuaded voter against roughly 100 dollars for conventional persuasion, and information density beat personalization as the driver. That is a single-session, short-form result. METR's July 2026 economics paper found the return on AI R&D productivity has been rising since coding agents launched, currently around 9 percent against the 15 percent threshold their model needs for self-sustaining acceleration, and separately the task-horizon doubling time sits at roughly 88.6 days. A persuasion capability priced per voter today is being generated by systems whose planning horizon is still doubling every few months. What does the AISI study's cost-per-persuaded-voter figure look like once the same system can sustain a multi-day conversation rather than a single exposure?
This is well-reasoned - thanks for this. I think the key (beyond the point you make several times, rightly, about *capturing attention* being the hard part) is also that the vectors for capturing attention are likely to be highly orthogonal to the vectors for what to do with attention once it's captured. Machine intelligence is likely to get very good at both, I'd conjecture, over quite a long T — but humans are still going to be very good at it too, with likely 'countermeasures pressure' priced in along various timelines (e.g. the way that AI 'tics' weren't as costly as they are now that people look out for them).
I don't think this refutes those who say that once there's ENOUGH raw intelligence, none of this matters, but it does update the relevant timelines accordingly.
Good point, thank you!
The persuasion and manipulation line drawn here is also a legal line in the EU. Article 5(1)(a) of the AI Act prohibits systems that deploy subliminal, purposefully manipulative or deceptive techniques that materially distort behaviour by appreciably impairing informed decision-making and cause significant harm. That prohibition has applied since 2 February 2025. Article 99(3) sets fines up to EUR 35 million or 7 percent of worldwide annual turnover, whichever is higher, with the lower figure applying to SMEs under Article 99(6). What nobody has yet is a decided case marking where measured persuasion ends and material distortion begins.
The AISI study's own numbers matter more than the headline caution suggests. Forty-two thousand participants, roughly a 10-point attitude shift, at 48 to 75 dollars per persuaded voter against roughly 100 dollars for conventional persuasion, and information density beat personalization as the driver. That is a single-session, short-form result. METR's July 2026 economics paper found the return on AI R&D productivity has been rising since coding agents launched, currently around 9 percent against the 15 percent threshold their model needs for self-sustaining acceleration, and separately the task-horizon doubling time sits at roughly 88.6 days. A persuasion capability priced per voter today is being generated by systems whose planning horizon is still doubling every few months. What does the AISI study's cost-per-persuaded-voter figure look like once the same system can sustain a multi-day conversation rather than a single exposure?