The AI safety movement needs normies
A broader base may be the only way for the AI safety field to get what it wants
If you look past the cryptic AI billboards lining Highway 101, San Francisco is still a city shaped by civil disobedience. For decades, young people, queer people, and weirdos of all stripes flocked west and settled here, building co-living spaces and resisting the powers that be. There’s plenty of anti-establishment angst to go around.
On a partly sunny Saturday this March, protestors gathered outside Anthropic’s office to rally against the AI race, a few blocks southwest of Market Street, the city’s historic artery of dissent. AI slop coverage of the protest illustrated a dense crowd framing a pink-haired woman in front of a “SAVE OUR JOBS” banner, screaming into a megaphone. San Francisco stuff.
In reality, Stop the AI Race pulled between a few dozen and a couple hundred people — mostly men, very earnest, and nearly all white — carrying more esoteric signs: “IT’S SMART ENOUGH,” one said. “MAY YOUR GPUs CHIP AND SHATTER,” said another. “PAUSE IS DEMANDED if you aren’t CONSISTENTLY CANDID.” Despite the city’s appetite for nonviolent protests and growing antagonism toward AI companies, the entire crowd could have comfortably fit in a couple of BART cars.
Then on April 10 a Molotov cocktail was thrown at Sam Altman’s San Francisco mansion sometime between 3am and 4am. Twenty-year old Daniel Alejandro Moreno-Gama was arrested for the attack outside OpenAI’s headquarters, where he was allegedly trying to break in. In his backpack, officers reportedly found a manifesto listing the names and home addresses of other AI executives. Earlier this year, he wrote Substack posts about death, destiny, and existential risks, or “x-risk,” posed by artificial intelligence.
(Within 48 hours, two others were arrested for shooting at Altman’s house before being released pending investigation — they reportedly had no connection to Moreno-Gama.)
While AI-informed types were quick to condemn the violence, many outside the Silicon Valley bubble seemed thrilled. “One does have to admire the skills of someone who can pour a good cocktail in this weather,” someone posted to Reddit. Instagram users — many with full legal names publicly displayed in their profiles — reacted similarly. “Where can we support their bail fund? ✨” one said. “New love language just dropped 😍,” replied another.
The vibe mismatch between the AI crowd and outsiders was unsettling, but similar splits over violence against corporate targets have happened before. In December 2024, #FreeLuigi went viral as users — mostly young people — painted Luigi Mangione as a folk hero after he killed UnitedHealthcare CEO Brian Thompson. The handsome suspect became the protagonist of a buzzy musical and dozens of steamy fanfics. Fans even bought merch.
Silicon Valley loves to say we can “just do things,” but when it comes to meaningfully changing the arc of AI development, most of us can’t do anything at all. Committing violence against tech CEOs and their families is not forgivable, but it’s certainly agentic. To a radicalized young man who believes that if anyone builds superintelligent AI, everyone dies, burning down a CEO’s house and company headquarters to save the human race might seem like the lesser of two evils. After all, if a trolley is approaching a fork in the tracks, utilitarian logic says you should pull a lever to send it running over a single victim, if it would save many others who’d otherwise die as a consequence of your inaction.
Over the last couple years, Moreno-Gama reportedly posted 34 messages to anti-AI activist group PauseAI’s public Discord under the Dune-inspired handle “Butlerian Jihadist,” referencing the book’s fictional crusade against thinking machines. PauseAI, whose US branch founder Holly Elmore gave a speech at last month’s Stop the AI Race protest, stated that it “unequivocally condemns this attack and all forms of violence, intimidation and harassment,” and that “violence against anyone is antithetical to everything we stand for.”
This attack and its aftermath — celebrated by anti-AI normies, rebuked by AI safety insiders, motivated by the same “doomer” canon that inspired Altman to found OpenAI in the first place — may have been a foreseeable consequence of a technical, existential-risk-focused community sounding the alarm before building a broadly-appealing coalition to channel people’s anxiety and anger into political action.
At least for now, protests against future superintelligence, focused on hedged demands such as “CEOs should commit to pause AI development if everyone else does too,” can only rally so many people. But even those who don’t spend time on LessWrong love Gen Z dudes who attack CEOs. This narrative transcends AI safety, argued tech journalist Brian Merchant: it “identif[ies] billionaire AI executives as uniquely powerful actors, who are all but unaccountable to democratic constraints and society’s best interests.”
While it’s easy to dismiss memes and Instagram comments as nihilistic noise, the legitimate frustration behind them is more widespread than the AI safety community seems to realize. A recent NBC News poll found that, netting positive views against negative ones, more voters feel worse about AI than about ICE (i.e., really bad). Mainstream anxieties about job loss, cyberattacks, and mass surveillance — which all rank relatively high on the public’s list of concerns about what AI might do — tie into x-risk-pilled concerns such as gradual disempowerment and loss of control.
The AI safety community has historically worried that addressing normie concerns would come at the expense of x-risk, and possibly knock it off potential legislation altogether. But these pressing, present socioeconomic issues may be the gateway that gets x-risk on the table. “These are the things that people are feeling right now,” said Alex McCoy, Head of Left Coalition at political advocacy group Humans First. “It doesn’t mean that they don’t believe in Skynet.”
Politicians respond to what they think their constituents want, and the vast majority of Americans do not want AI to continue along its current trajectory. The momentum is there — people are beginning to take action, however imprecisely, driven by deeply-rooted feelings of unfairness and demoralization. Traditional AI safety advocates may just need to cede enough control of their narrative to harness it.
It all comes down to insularity
Traditionally, the work of AI safety has excluded the public by design. For years, AI safety discourse has largely unfolded behind closed doors, between researchers, executives, and the policymakers they have direct access to. This has often been prioritized over broader public engagement, for what seemed like good strategic reasons. Long, information-dense blog posts, closed-door meetings and money carry more weight in this world than mainstream media, anyway. Why waste time explaining “AGI” to normies, when they’re not drafting policy proposals?
The AI safety field was built on the idea that, with enough compute, money, and brainpower, a small group of very smart people can save the world — no help, input, or permission required. They reasoned that preventing tech companies from building a deadly machine god should be done as quietly as possible, without attracting the kind of public or political attention that might inadvertently spark an ill-fated race towards superintelligence. In hindsight, this wasn’t paranoid: philosopher Nick Bostrom’s 2014 bestseller Superintelligence: Paths, Dangers, Strategies surfaced concerns about x-risk from the depths of the rationalist blogosphere to the New York Times best seller list — and partially motivated OpenAI’s founding.
But in choosing to operate largely behind the scenes, the AI safety community created a vacuum that’s now being filled by industry lobbyists, populist politicians, and radicalized individuals. Leading the Future, a pro-AI super PAC network backed by venture capitalists and AI executives, has reported raising more than $75m, and claims to have raised $140m. Meanwhile, Bernie Sanders, Gen Z influencers, and data center NIMBYs are leading the populist backlash against the industry. Existential risk has only very recently entered the conversation, and it comes with baggage.
While AI safety is not the same thing as effective altruism, the two are deeply entangled. EA, a movement that attempts to maximize human flourishing through quantitative reasoning, funneled a lot of talent and money toward early AI safety research. Today, many of the field’s biggest names speak at EA conferences, share donors and shape AI policy decisions. At this point, it’s very hard to disentangle the public’s perception of “AI safety” from EA itself.
The community’s manufactured insularity and inclination for “Secret Congress”-esque policymaking stoked an aura of opacity, leading to public distrust from all sides. “In Silicon Valley, the EAs are viewed as one step to the right of Elizabeth Warren,” a former Biden official told Politico. “Conversely, in DC, on the left they think EAs are the devil.” Everywhere else, the public’s response is mostly “Who are you guys?,” said Akshyae Singh, co-founder of The Frame, an accelerator for AI safety content creators.
This problem isn’t completely lost on insiders. Two years ago, a survey asked 17 prominent AI safety experts what big mistakes the AI safety community was making. Their biggest gripes: “overly theoretical argumentation” and “being too insular.” A couple of respondents explicitly called out the EA urge to dismiss public outreach in favor of technical problem-solving as the root of the problem. Richard Ngo, an independent researcher who previously worked at DeepMind and OpenAI, argued that, at least at the time, AI safety overemphasized fundraising and back-room deals at the expense of the field’s public image. “From the perspective of an external observer,” he responded, “it’s difficult to know how much to trust stated motivations, especially when they tend to lead to the same outcomes as deliberate power-seeking.” Another respondent, METR researcher Daniel Filan, said, “If the three biggest oil companies were all founded by people super concerned about climate change, you might think that something was going wrong.”
Others, including prominent commentator Anton Leicht, reject the idea of building a popular movement altogether. One of Leicht’s primary concerns, echoed across the upper echelons of Silicon Valley, is that in the process of trying to address everyone’s concerns, a populist AI safety movement will wind up dropping the one thing that ought to be its central concern: existential risks. The line items that rally a crowd — data center moratoriums, for instance, or child safety legislation — don’t necessarily make for great policy. Leicht believes that the AI safety community’s strongest assets are high expert credibility, and the fact that most people seem to vaguely support AI regulation. He worries that a poorly-executed popular movement could ruin both.
Even so, Leicht acknowledges that there are tradeoffs to playing inside baseball. “There is a huge epistemic gap between the small community of people who think they’re on the inside, and the rest of the world,” he told me. Like many others in the field, he’s wary of communicating to the “lowest common denominator” about existential risk without feeling confident that non-experts have enough background knowledge to contribute productively. “I don’t have a solution to it,” he said. “I think people are just not that good at it.”
McCoy described this perspective as “emblematic of the sort of anti-politics” of the AI safety community. Congress will address AI one way or another, he said, and whatever organized constituencies show up will shape what that legislation will ultimately look like.
“If the AI safety community does not take seriously the necessity to engage in capital-P politics,” he cautioned, “its concerns will be left out.”
Who’s missing?
One big problem, Singh argues, is that “people in this field don’t tend to do most things unless there’s absolute concrete proof” that it will be effective. In the absence of a previously-successful effort to increase demographic diversity within the AI safety community, there are no hard numbers saying that decentering the concerns of a tiny, homogenous group of people will make passing AI regulation easier. But a mass movement with the power to pressure governments and AI companies, by definition, has to include a lot of people — including those who don’t currently fit inside the AI safety bubble.
In the world of left-wing community organizing, there’s a common refrain: center the most impacted. “That’s not out of some kind of virtue signaling,” McCoy said. “It’s because those are the people that are going to fight the hardest.” But those most directly feeling the real-world impacts of AI today — first-generation college graduates struggling in an increasingly-nonsensical job market; women humiliated by nonconsensual deepfakes; content moderators in Nairobi exposed to traumatic content for a couple dollars an hour — are those least represented within the AI safety community.
And there is data to back this up. A 2025 Seismic Report, for instance, found that women are over twice as concerned about AI than men. On a global level, a UN report found that female-dominated occupations are almost twice as likely to have high automation potential than male-dominated ones, And, relative to men, they’re over three times more likely to have their jobs disrupted by AI. Yet, women are few and far between in the AI safety field, particularly in technical roles — perhaps in part because near-term socioeconomic concerns often wind up lower on the priority list than more abstract concerns about alignment, interpretability, and the governance of AI systems that don’t exist yet.
The dominant AI safety discourse “primarily serves the interests of technological institutions and stakeholders in high-income nations, often privileging abstract future scenarios over pressing sociotechnical harms that disproportionately affect marginalized communities,” the Brookings Institution wrote last year. Growing up in India, this deeply frustrated Singh. “If brown people don’t have a voice, there is no way in hell that you’re making a solution that benefits me,” Singh said. “Like, how do you know how I feel?”
Getting abstract future scenarios onto the policy agenda may, perhaps counterintuitively, depend on addressing the concrete harms already shaping people’s lives — and, by extension, informing who they vote for and which AI products they use and pay for. The AI safety community already tried relying on a handful of well-connected experts to regulate the AI industry behind closed doors, and it’s not working.
The AI safety field is mostly talking to itself, and it’s created an information void that’s being filled by populist anger. Outside Silicon Valley, most people don’t experience AI as a powerful coding tool or existential threat. Rather, it’s a symbol of the machine we’re meant to be raging against — not an extinction risk, per se, but something billionaires are using to forcibly strip humans of their humanity.
“I don’t think it’s about persuading people that superintelligence is bad,” said John Sherman, president of the AI Risk Network. “It’s about persuading people that they can make a difference.”
How to (hopefully) not screw up the AI safety movement
Sherman proudly introduced himself to me as a Baltimore resident who, until a couple years ago, “didn’t know anything about AI.” After decades of working in TV and video production, “I can edit in Adobe Premiere,” he joked. “That’s about as technical as I get.”
Then he stumbled across Eliezer Yudkowsky’s 2023 article in TIME Magazine, in which he wrote: “If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.”
Today, Sherman also runs a nonprofit called GuardRail Now, focused on communicating AI x-risk to normies. “My primary concern is x-risk,” he said, but “I think we have to take side roads to get to the destination. And a lot of people in AI safety are unwilling to consider that — but they’re not getting anywhere. So like, how’s it going? You’re stuck.”
Sherman’s theory of change is simple: people are feeling a visceral sense of unfairness. AI companies are reshaping society without anyone’s consent, so “we need to make unsafe AI bad for business.” This can be framed in terms of existential risk, he said: “We’re building systems that we don’t know how to control, that we don’t understand how they work, that the experts say can kill everybody. Why would we do that?”
Imagine a family in Ohio, where everyone is experiencing AI differently. Dad is uneasy about new AI policies at his white collar job, and a data center is being built behind the neighborhood school. His teenager is doing god-knows-what on ChatGPT, and his college kid is talking about how she wants to drop out and work in construction. “To build a real movement,” Sherman said, AI safety advocates “need to run full speed ahead towards people who are concerned about their kids, towards people who are concerned about data centers … the whole thing, all of it.”
This approach makes many in the AI safety community uneasy, reasonably so. A little over two weeks after hiring Sherman as its Director of Public Engagement, the Center for AI Safety parted ways with him after clips surfaced of Sherman telling podcast listeners that the “proper reaction” to the AI race was to “walk to the labs across the country and burn them down.” CAIS announced that the “connotation of statements like this do not reflect CAIS’s values,” to distance itself from this kind of fiery rhetoric — which, while in this case hypothetical, could radicalize people who may already be angry enough to act. (Sherman said he regretted using the language, clarifying that he meant “when the general public finds out their lives are being risked for AI, the reasonable reaction is to shut it down.” However, he has continued to describe AI in rather intense, hyperbolic terms.)
But building a movement doesn’t mean encouraging violence. The climate movement, for example, managed to grow from scientists expressing concern amongst each other to a global issue, without linearly increasing the risk oil executives faced from potential attackers. Decades of climate activism, including conveying the existential stakes of climate change to the public, hasn’t led to the kind of violence one might expect if x-risk messaging were a reliable radicalizer. Arguably, political organizing gives individuals a more structured outlet for their righteous frustration than attempted murder. The alternative — an AI safety community that stays silent while populists take up space around them —opens the door to unstructured acts of radical violence and worse policy.
Even Yudkowsky, Machine Intelligence Research Institute (MIRI) president Nate Soares, and the broader Berkeley rationalist scene they helped build — who have arguably shaped the conversation around existential risk more than anyone — recently pivoted hard toward public outreach. Soares, who co-authored If Anyone Builds It, Everyone Dies, has been on tour for the book. Last month, he spoke at the Stop the AI Race protest in San Francisco.
In February, Soares and Yudkowsky sat down with Bernie Sanders at MIRI’s Berkeley offices. After decades of treating persuading the public at large as a distraction, some of the biggest players of the inside game have, however begrudgingly, realized they can’t do this alone.
It’s not an accident that Sanders was the first to get there. Perhaps more than anyone else in DC, his politics is centered around pushing back against unaccountable billionaires and their friends in government — which, stripped of jargon, is what preventing superintelligence ultimately requires. Given the Trump administration’s damage to American soft power abroad and the unimaginable amount of money AI companies have to throw around, an international treaty to pause the race looks relatively unattainable.
But AI companies are still businesses. And the people with the most leverage over large corporations are customers, workers, investors and voters — not researchers writing alignment papers.
“AI safety as a persuasive cause will never have more power than the industry’s hard power in dollars and political influence,” McCoy said, “unless it is allied to constituencies who can lend their power together.” Conversations about extinction risk among rich tech guys in San Francisco, he added, are “not the message that is going to get hundreds of people to show up to a protest.” The protests that matter will be about jobs, surveillance, kids, data centers — what McCoy calls the “symptoms.” But the disease, in his framing, is exactly what x-risk advocates have been trying to address, seen from another angle: “an unaccountable set of billionaire investors and executives who have no guardrails and are seeking to concentrate an incredible amount of power in their companies.”
One could argue (and Leicht does) that by making AI safety an “omnicause” addressing everyone’s prosaic concerns, it will get elbowed out of whatever legislation ends up passing. But existential risk is already last on the public’s list of AI concerns across every demographic, according to last year’s Seismic poll. It might need to join a coalition of other causes to get on the table at all.
“AI safety right now is islands,” Sherman told me. “We need an ocean to connect the islands.”
Silicon Valley speaks its own language, and most people outside the bubble don’t understand it. “The best thing that the AI safety movement could do would be to build an army of surrogates who are regular people, going into their own communities and talking about this stuff,” Sherman said — “not strangers from a foreign land speaking a different language.”
Singh agrees. The catastrophic threats posed by AI aren’t hard to grasp. “Like, my dad, my mom — I can explain it to a lot of people, and they’ll get what’s going on,” Singh said. But they’re often made unintelligible by people who treat communicating to people without technical backgrounds as an inconvenience rather than a necessity. So Singh launched the Frame Fellowship, an eight-week incubator for content creators to bring AI safety discourse to the masses via YouTube, TikTok, and Instagram (Michaël Trazzi, who organized the Stop the AI Race protest, was also a fellow).
Long-term existential concerns about AI aren’t separate from near-term populist anxieties. Loss of control and gradual disempowerment are natural extensions of power concentration and job displacement. When microinfluencers talk about AI in “Get Ready With Me” TikToks, or neighborhoods band together at town hall meetings to voice their concerns, the case for existential risk becomes the endpoint of what people are already afraid of.
The AI safety community has a window of opportunity to make its case to the broader public, and some already are. While accelerationists such as Marc Andreessen have dismissed efforts to communicate about AI risks as “propaganda” fueled by “unaccountable dark money,” pro-industry leaders are also investing in communicating beyond the Bay Area bubble. The majority of people in the US feel uncomfortable about the current trajectory of AI, and this discomfort will likely turn into action. Whether that manifests as voting power or bottles of gasoline flying over San Francisco depends on building a movement.
Lots of good stuff here, but it falls into a different trap, I think, than the common AI-safety trap of refusing public and political engagement. A lot of the strategies discussed in the article, if scaled up and intertwined with AI safety more deeply, risk turning the fight to stop the AI race into an explicitly politically polarized issue. We need to engage not just the left, but also the right. I'll be writing more about this soon, hopefully in the next week or two, but this is not an impossible task, far from it. Both mass disemployment/perma-UBI with no means to better your own situation through your own hard work, and x-risk itself, are very powerful horrors to especially the populist/MAGA right if framed in their own languages and their own priorities rather than making it seem like just one new piece of a broader lefty-activist omnicause, as well as avoiding falling into certain very avoidable traps. Much more on this and other topics here: https://connorsscratchpad.substack.com/p/strategic-considerations-for-pausing
This is especially true given the current administration. A purely lefty movement is not going to move a right wing administration. In my linked essay above I talked about how the mass MAGA participation in the pushback against Mike Lee's land sale efforts was crucial.
Geoffrey Miller has also written on this, on X and elsewhere, much more eloquently than myself.
This op-ed has several problems that stem from either a conflation between AI safety and AI governance (the latter of which is much broader), or a failure to take existential AI risks seriously.
The section "Who's Missing" contains both of these mistakes. For example, the second paragraph implies that unemployed people deserve more representation within AI safety advocacy. Why? I understand that gen-AI screwed up the job market, but that's not a *safety* problem; it's a governance problem.
The section also recommends "centering the most impacted", but that doesn't make sense as a response to an extinction threat, which maximally impacts everyone. This universality also applies to lesser concerns, including unemployment. The ILO report claims that >70% of jobs have no exposure to automation, but that's ridiculous to anyone who takes ASI seriously. AI is coming for everyone's job, regardless of one's race or sex. (Tangential observation: the ILO report's so-called "global" data excludes the U.S.A., Canada, Australia, and New Zealand.)
Contrary to a different part of the op-ed, loss-of-control and gradual disempowerment are not extensions of power concentration, and loss-of-control certainly isn't a an extension of job displacement. A loss-of-control scenario involves an AI acting against its overseers. This is another conflation between AI safety and AI governance.
https://securityandtechnology.org/virtual-library/report/ai-loss-of-control-risk-indications-warning/
Regarding the critique that AI safety advocacy relies on overly theoretical arguments: that may have been true two years ago, when the survey was taken, but it's not true anymore. Both Anthropic and independent organizations, such as Redwood Research, have concretely demonstrated the problems that early AI safety researchers warned about.
https://www.lesswrong.com/posts/b8eeCGe3FWzHKbePF/agentic-misalignment-how-llms-could-be-insider-threats-1