NEED TO KNOW

• The White House is reportedly negotiating federal preemption of some state AI laws in exchange for support on social media and AI child protection measures.

• OpenAI called for an international organization “to make it possible for the world to take coordinated action, including slowing frontier development when needed.”

• SpaceX raised $75b in its IPO at a $1.77t valuation.

But first…

THE BIG STORY

Fable, Anthropic’s latest AI model, is very good at AI research and development. Anthropic’s own researchers use it — and Mythos, its more powerful, still publicly unavailable sibling — to automate much of their work in advancing the frontier of AI.

But if a non-Anthropic employee wants to do the same, they’re out of luck. Fable has “safeguards” built in to prevent it from being used for frontier AI development. It’s a controversial, complicated move — and one that’s indicative of the strange way power works as AI development accelerates.

As we’ve previously covered, the leading AI companies think they are on the cusp of fully automating AI R&D. As they move closer, access to powerful AI models becomes a strategic asset: today’s best model is the key tool for building tomorrow’s.

People in this world have theorized for years that companies would eventually start withholding that access. This week, it stopped being a theory.

There are lots of reasons to withhold that access. The most obvious is profit: why let your competitors use your tools to catch up? Anthropic’s stated reason is national security: the company says it doesn’t want “foreign adversaries” using Claude to “erode [America’s] advantage.” And some suggest the unstated reason is that building a big lead makes an eventual pause more likely — during which one can take costly actions (like devoting compute to alignment research) that hopefully make us all safer.

None of those reasons justify Anthropic’s initial decision to hide these guardrails from users (in an effort to make it harder to get around them) — a move which prompted widespread outrage and a quick reversal. But though the guardrails are now disclosed, they are still present. And the question of whether that is justifiable relies on the impossible: knowing the motivation behind it.

“One of the hard things for Anthropic is that the actions you’d take from sincere safety concern often look exactly like the actions you’d take to entrench your own power,” X user Maja said this week. Anthropic could be completely well-intentioned in withholding Mythos’ capabilities for itself. It also could be the actions of a standard, profit- or power-seeking company. It should not matter. We’re talking about whether Anthropic is doing this for good reasons, but we should be talking about who decides whether it happens in the first place.

There is something deeply uncomfortable about a private company having this much control over what could be the most important technology yet developed. It is not at all obvious what the better solution is: a government forcing a company to sell to its competitors has its downsides, too. One thing, however, is clear. Deciding who gets access to frontier AI models is a new form of power — and we have failed to answer the hard questions about who should wield it.

THIS WEEK ON TRANSFORMER

• Making deals with AI sounds crazy. Is it? — Celia Ford explores a strange and new debate in AI safety

• OpenAI isn’t being consistently candid about Leading the Future — Veronica Irwin reports on the key details missing from OpenAI’s statements on the super PAC

Subscribe now

THE DISCOURSE

Welp, Anthropic employees are done coding:

• Nat McAleese: “I always felt Opus 4.5 could barely code; 4.6 was just-about-useful, but I have barely written a line of code since fable.”

• Liv Gorton: “I joined anthropic a ~month ago and have written ~no code myself.”

Nerfing drama aside, Claude Fable 5 appears chillingly autonomous:

• Felix Rieseberg: “With Fable 5 out in the world, I think a third era quietly started today.”

  • “I no longer tell Claude to investigate a particular crash report. It runs in a loop … its job is no longer to help me fix a crash, it’s to keep our apps from crashing.”

• Ethan Mollick: “With Fable the spell has gotten powerful enough that I am no longer sure I am the wizard. I am closer to a patron. I describe what I want, I pay for it, and I judge the result. The conjuring happens somewhere I cannot watch, in hundreds of small choices I never get a vote on.”

  • “[It’s possible] that the more capable the model, the less there is for a human to meaningfully do, and the black box is the price of power.”

Miles Brundage is frustrated:

• “I eagerly use Anthropic products because the models are good[,] but I do genuinely think they are high on their own supply + unaware of just how many bugs they are constantly introducing due to overreliance on Claude + shipping too many things too quickly.”

Sriram Krishnan had thoughts on AI gatekeeping:

• “Just to state the obvious: think there’s a collision course between those who believe research and science should be open and those who believe we are in an accelerating singularity curve. I have many smart friends who have believed both for a while but seeing more and more their realization that these beliefs will be in conflict.”

bayeslord had a more cynical take, in light of the Fable release:

• “They didn’t mean pause AI research, they meant pause your AI research”

Joshua Achiam thinks people misunderstand the distinction between OpenAI and Anthropic:

• “Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.”

• “There are a lot of innocuous and even quite agreeable choices in Claude’s constitution that potentially endow it with a huge amount of authority, maybe even a mandate, to make complex ethical decisions about how to interact with human systems and who to grant power to … cloaked in the language of ethics and virtue there is a sharp and potentially quite lethal double edge to this sword.”

Jasmine Sun wrote advice to the graduating class of 2026:

• “I often ask AI folks what they’d tell a normal 22-year-old. Don’t know, they’re screwed, is the non-answer I get most. In that response, I hear depressing defeatism: What can anyone do in the shadow of the technocapital machine?”

• “Relish the pivots; ride the waves; recite the Serenity Prayer every morning and chase sunsets at night. I won’t tell you that the future’s smooth sailing. But what a thrill to be alive!”

POLICY

• The White House is reportedly negotiating federal preemption of some state AI laws in exchange for support on social media and AI child protection measures. Sen. Marsha Blackburn is leading the negotiations.

  • Sen. Ted Cruz appears to be involved in the efforts, too, saying that federal preemption and child safety bills are “an element of discussion” for an upcoming markup.

  • Chief of Staff Susie Wiles, first lady Melania Trump, staff for the Office of Science and Technology Policy and the National Economic Council reportedly also met with children’s online safety groups, including the American Principles Project and Ethics and Public Policy Center, to discuss Blackburn’s KOSA and the App Store Accountability Act.

• Trump reportedly blindsided AI company leaders when he announced a meeting to discuss the government taking equity stakes in their firms.

  • At least a dozen GOP House and Senate offices opposed Trump’s proposal, with Sen. Ted Cruz stating “I don’t think the federal government should be in the business of being an equity holder in private companies.”

  • On Wednesday, Trump doubled down, saying “if we do [take equity stakes], the public will become very rich.”

• Sriram Krishnan is leaving his position as a top AI policy advisor in the Trump administration at the end of the month.

• Thomas Lind is also leaving his position as head of policy at the Office of the National Cyber Director.

• National Cyber Director Sean Cairncross reportedly asked CAISI to halt public reports on model assessments.

  • Notably, CAISI wasn’t included in the list of agencies tasked with drafting “standardized AI national security Test, Evaluation, Verification, and Validation methodologies” in last week’s National Security Presidential Memorandum.

• Lawmakers from both parties questioned why the Treasury, rather than CISA, was given the lead role in AI cyber defense under Trump’s EO.

• House Speaker Mike Johnson, House Majority Leader Steve Scalise and the co-chairs of the House Democratic Commission on AI cast doubt on a bipartisan AI regulation proposal from Reps. Jay Obernolte and Lori Trahan.

  • Republicans pushed for it to include a broader preemption measure, while Democrats said the bill “does not meet the enormity of the moment.”

• Sen. Chuck Schumer said passing AI legislation in this Congress will be “hard,” but that he “would very much like to see that get done the sooner the better.”

• The Senate Banking Committee is reportedly weighing an export control markup in the coming weeks, potentially for inclusion in the NDAA.

• Sens. Mark Kelly and Jim Banks introduced the AI DATA Act, which would require the Labor Department to track AI’s impact on the workforce through quarterly surveys and annual reports.

• Reps. John Moolenaar, Jay Obernolte and Jennifer McClellan introduced the GUARD Act, which would require national security reviews of robots made by “adversaries” (i.e. China) and block those posing threats.

• The UK launched the AI Economics Institute, chaired by Nobel laureate Simon Johnson to research AI’s economic impacts and inform policy, with collaboration agreements from Anthropic, OpenAI, Google and Microsoft.

  • The UK also announced an AI Hardware Plan to provide funding for AI chip companies.

• Taiwan is reportedly considering stricter export controls that would restrict AI chip sales to all customers in China and enable prosecution of smuggling as a criminal offense for the first time.

INFLUENCE

• Anthropic published a policy framework, calling for the government to have “the legal authority to block or deter” deployment of models that pose catastrophic risks, mandatory third-party testing, and narrow federal preemption of state AI laws.

  • It also published an economic policy framework, proposing various policy options at different levels of unemployment — including a universal basic income scheme in the case of “unprecedented levels of unemployment.”

  • Dario Amodei released a policy essay alongside the framework.

• OpenAI called for an international organization “to make it possible for the world to take coordinated action, including slowing frontier development when needed.”

  • Last week, Anthropic called for the world to have “the option to slow or temporarily pause frontier AI development.”

  • OpenAI’s roon said: “now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development.”

• Former Andreessen Horowitz partner John O’Farrell criticized AI industry political spending, arguing that PACs like Leading the Future are trying to “intimidate politicians” and silence debate on AI regulation, rather than engaging seriously with policy questions.

• OpenAI said it found “PRC-linked influence operations” targeting data center buildouts.

  • Its report found “no evidence of breakout” from the campaign and that “most of the social media posts … generated little or no observable engagement.”

• Nvidia CEO Jensen Huang declined Sen. Elizabeth Warren‘s invitation to testify before the Senate Banking Committee on AI, China, and US export controls.

• Congressional staffers drafting federal AI legislation reportedly took an industry-funded trip to meet Google, Nvidia, and other companies lobbying to preempt state AI regulations.

• A new Reuters/Ipsos poll showed that half of Americans fear AI could put someone in their household out of work, with Democrats (61%) more worried than Republicans (47%).

INDUSTRY

OpenAI

• OpenAI filed a confidential S-1 for a potential IPO.

  • Sam Altman reportedly told staff that while OpenAI expects to go public “within the next year … the faster the potential RSI takeoff looks like it could be, the more it could be advantageous to delay an IPO.”

• Sam Altman and chief scientist Jakub Pachocki outlined the company’s top three goals:

  • (1) build an automated AI researcher by March 2028,

  • (2) accelerate the economy, and

  • (3) give everyone on Earth a personal AGI

• It’s planning to overhaul ChatGPT to shift focus from its chatbot to Codex and AI agents.

• It’s negotiating a lease on a planned 10 GW data center in Ohio that could cost more than $500b, potentially with credit support from Nvidia.

• It’s thinking about slashing prices to lure enterprise customers away from Anthropic.

• It’s acquiring Ona, which will provide cloud execution environments for Codex agents.

SpaceX

• SpaceX raised $75b in its IPO at a $1.77t valuation. Its shares will list later today.

• It reportedly plans to launch initial demos of orbital data centers by late 2027.

• Elon Musk came under fire from UK politicians for stoking tensions around riots targeting immigrants following a murder in Belfast.

• Former xAI engineer Devin Kim filed a lawsuit claiming he was fired for raising safety concerns about Grok, including its ability to increase discrimination and provide information about WMDs.

  • The suit says that the engineer’s supervisor, xAI co-founder Jimmy Ba who left earlier this year, ignored safety directives from Musk.

  • Kim was recently appointed president of the Center for AI Safety. Dan Hendrycks, CAIS’s founder, is an advisor to xAI.

Anthropic

• Anthropic launched Claude Corps, a year-long fellowship program placing 1,000 fellows at US-based nonprofits, where they’ll build AI tools with their host organization.

  • Hosts range from local YMCAs to conservationists to mental health support groups.

• Apollo and Blackstone raised $35b to finance its lease of Alphabet chips.

• It reportedly signed over a dozen letters of intent to lease data center facilities totaling more than 1 GW of capacity, its first direct data center tenancies.

• Dario Amodei reportedly only has one direct report (chief of staff Avital Balwit), freeing him to do other work while Daniela Amodei manages the rest of the executive team.

Apple

• Apple announced its newest generation of Apple Foundation Models, built in collaboration with Google.

  • They include AFM 3 Core Advanced, which runs on-device, and its server-side model AFM 3 Cloud.

  • The new Siri AI is powered by these models.

• Its stock dropped over 3% after the launches.

Google

• Google agreed to pay SpaceX $920m per month for access to 110,000 Nvidia GPUs, CPUs and other infrastructure.

• It pledged $50m for training trade workers on the kinds of jobs involved in building data centers.

Others

• Meta officially cut ties with Manus: Manus can’t access Meta’s internal data system, and Meta can’t use Manus tools.

• Amazon disclosed its data center water use, which it claims beats the industry average and is lower than its own previous usage, despite operating more data centers.

• Nvidia and SK Hynix partnered to design and manufacture memory chips for AI.

• GitHub reportedly disabled 73 repos after hackers injected malware that would steal user credentials when opened in AI coding platforms.

• ByteDance spun off its AI drug discovery platform Anew Labs into a separate entity (still largely under ByteDance’s control).

• Prometheus, Jeff Bezos’ AI startup for manufacturing and engineering, raised $12b in Series B funding at a $41b valuation.

• Former xAI employees, including xAI co-founder Igor Babuschkin, launched River AI, which aims to build AI that “works entirely for you.”

• Perplexity reportedly intends to IPO in 2028 whether or not doing so goes well for SpaceX, Anthropic and OpenAI.

• Chinese startup Moonshot AI is seeking up to $2b in funding at a $30b valuation, up from a just $4b valuation in December.

MOVES

• Clive Chan left OpenAI‘s custom chip program to join Anthropic.

• Gabriel Petersson resigned from OpenAI, tweeting that there’s “one last product i need to build before AGI.”

• Chris Lovejoy, an ex-medical doctor, joined Anthropic‘s Applied AI team.

• Allison Duettmann, CEO of Foresight Institute, joined Mythos Ventures as a Venture Partner.

• Leigh Nolan is the Institute for AI Policy and Strategy’s new policy director.

• Anissa Gardizy joined The Wall Street Journal as a reporter covering cloud computing and AI infrastructure.

RESEARCH

• Researchers at the AI Security Institute were able to jailbreak Claude Fable 5 in just a few hours, and got close to a universal jailbreak within a few days.

• Pliny the Liberator published the ~120,000-character Claude Fable 5 system prompt.

• Google DeepMind announced a $10m funding call for research focusing on multi-agent systems.

• Cosmos Institute announced its new cohort of senior research fellows, which includes Google DeepMind’s Séb Krier and Anthropic’s Matthew Botvinick, among other big names.

• Geoffrey Irving launched Sequent, a new nonprofit research organization focused on aligning superintelligence.

• A new Google DeepMind paper explored four pathways from AGI to artificial superintelligence, arguing that instead of a “single transformative step change” we might experience “a series of transformative societal changes.”

• Indicator’s Alexios Mantzarlis ran an adversarial test on Pangram, which tricked the detector into misidentifying AI-generated text as human between 75-92% of the time.

  • Despite the finding, Mantzarlis said that he was “relatively impressed at the Pangram’s solidity in the face of adversarial attacks.”

• Cornell researchers Sil Hamilton and David Mimno figured out why the fictional “Elias Thorne” (often a lighthouse keeper or clockmaker) keeps starring in AI-generated stories.

  • GPT-3.5 (which powered the original ChatGPT) was used to make WildChat, a dataset of 1 million ChatGPT conversations that’s since been used to train subsequent LLMs across frontier companies. 166 chats had the name “Elias.”

  • Alignment training may lead models to prefer a small subset of “safe” WildChat content — including Elias-related stories, apparently.

BEST OF THE REST

• Many people appear to be using AI to mask their low literacy, Axios reported.

• An expert panel — Daron Acemoglu, Dean Ball, Ethan Mollick, Clara Shih — sat with NYT’s Bill Wasik to talk about how workers should brace for AI’s impact on jobs.

• With big IPOs on the horizon, the San Francisco housing market is royally fucked.

• Move over, AI 2027. A group of European researchers published Europe 2031, an imagined (but all too plausible) future where Europe “slide[s] into irrelevance” as AI accelerates around it.

• METR’s Ajeya Cotra and Understanding AI’s Timothy B. Lee debated how long it will take for AI to become self-sufficient — a glossy continuation of a past Twitter debate.

• Helen Toner explained AI chatbots to Oprah.

• Someone (maybe for real?) built CrankGPT, a hand-crank-powered computer marketed as a “human-powered, fully local and private AI solution.”

MEME OF THE WEEK

Credit: tomie

Thanks for reading. Have a great weekend.

Share

When OpenAI published a post on June 1 distancing itself from the super PAC Leading the Future, employees were rather pleased.

In recent weeks, OpenAI employees have raised concerns to the company’s global affairs team about OpenAI’s links to the PAC, to which its president Greg Brockman and his wife Anna have donated $25m, as well as some of OpenAI’s own policy actions, one employee told Transformer. At a tense meeting in May with global affairs chief Chris Lehane, they pressed OpenAI to clarify its relationship with the super PAC, the employee said.

Shortly afterwards, the company publicly addressed concerns over LTF, stating, among other things, that: “OpenAI does not direct the activities of LTF, or have visibility into their operations.”

The blog appeared to reassure some employees. “I’m happy OpenAI put out this statement,” alignment researcher Jason Wolfe said, noting that “personally I really dislike a lot of things I’ve heard about LTF.”

“The good news is that we can now show congressional staff this blogpost that says in bold ‘No outside political group speaks for OpenAI or represents our company’s views.’ and criticizes LTF tactics,” Shantanu Jain, another technical researcher at the company, said, quoting the post.

But the post does not tell the full story, omitting key details about Lehane’s role in establishing LTF and the way super PAC operation’s own staff view their relationship with OpenAI.

As previously reported, Lehane was one of a group who provided advice on creating Leading the Future, its affiliated partisan super PACs, and its affiliated 501(c)(4) Build American AI, which does not have to disclose its donors.

What has not been reported is that he is thought in AI policy circles to have picked Josh Vlasto, a longtime Democratic operative, to co-lead LTF — something that when asked, neither OpenAI nor Leading the Future denied. And though OpenAI has insisted that Brockman’s donation was made in a “personal capacity,” Transformer has found that not everyone working within LTF appears to view it that way.

Subscribe now

“For this project I’m funded by four separate entities … OpenAI is just one of them,” Nathan Leamer, executive director of Build American AI, told Transformer over text message on May 3, just a month before OpenAI’s statement.

When later asked about the division of labor between Leamer, Moffat, Vlasto and Lehane, Leamer said “I think the best way to think of it is the corporate funders all have a say but they interact directly with Zac and Josh. They run LTF which funds my crew BAAI.”

In response to a request for comment, LTF spokesperson Jesse Hunt told Transformer: “Neither Leading the Future nor any affiliated super PACs or 501(c)(4) organizations has ever received funding from OpenAI. All decision making regarding candidates, policy or operations are made solely by Zac Moffat and Josh Vlasto.”

An OpenAI spokesperson said: “Neither OpenAI nor Greg and Anna Brockman have donated to or been involved with Build American AI.”

She added: “As Chris has said previously, he was consulted when the PAC was getting stood up last year, but has no current involvement.”

In recent weeks, Lehane has spoken to reporters at WP Intelligence, Wired and Punchbowl. In those interviews, Lehane said he was displeased with some of Leading the Future’s activities, telling WP Intelligence that OpenAI wasn’t “so much into the tactics.” He also emphasized the separation between OpenAI and Leading the Future’s donors, telling Wired that he had only advised Brockman “in a very general way” over his political spending, and saying that he is currently “not involved” with Leading the Future and wants to “let them be their own independent outside thing.” Earlier in the month, an OpenAI spokesperson said that OpenAI had “not provided funding or any support to” Leading the Future or Build American AI. That appears to directly contradict Leamer’s inclusion of OpenAI as one of the group’s “corporate funders.”

In the June 1 post, OpenAI also set out a set of standards for lobbying and political advocacy on AI, writing that groups “that are advocating on AI should … not use tactics like astroturfing that obscure the real choices facing policymakers and the public.” Days later, Build American AI admitted that one of its “outside vendor[s]” was behind multiple anonymous sock puppet X accounts exposed by the Midas Project and Taylor Lorenz. When Lorenz previously uncovered an LTF-linked campaign to pay influencers to spread anti-China messaging, she reported that “the agency hired to execute it said they reported directly to Vlasto.”

Ironically, given the lengths OpenAI has gone to state that it has no sway over LTF, recent changes in the super PAC’s approach appear to align with OpenAI’s own policy shifts in a way that many of its more safety-minded employees might welcome. In the weeks since OpenAI employees expressed disappointment about its support for a controversial Illinois bill that would have shielded AI companies from liability, OpenAI dropped its support for the bill, instead endorsing a much stronger bill, SB 315. OpenAI suggested that bill, along with New York’s RAISE Act and California’s SB 53, could set the standard for federal laws in what Lehane has dubbed “reverse federalism.”

In the same period of time, Build American AI and LTF have demonstrated a similar shift in messaging. Shortly after OpenAI endorsed SB 315 (and after the bill passed), Build American AI also endorsed the bill. Leading the Future, meanwhile, recently endorsed the RAISE Act, despite spending millions opposing its sponsor, Alex Bores, and criticizing a “patchwork of regulation.” It is a far cry from last year, when Leading the Future and Build American AI ran a $10m campaign calling for federal preemption of state AI laws.

Share

OpenAI is of course not the group’s sole influence. The co-founders of venture capital firm Andreessen Horowitz (a16z), Marc Andreessen and Ben Horowitz, have each contributed $25m to the super PAC, and Leading the Future has often taken positions similar to those of the VC firm. Like LTF, a16z has long argued against legislating for AI at the state level, warning of a “patchwork of state laws.” The American Innovators Network, another a16z-backed group, opposed SB 315 and the RAISE Act. A16z declined to comment.

With a16z and OpenAI’s stances seemingly diverging, the pressure to appease both parties puts the super PAC and Build American AI in a tricky situation. The group has produced friendlier messaging on some state AI laws which OpenAI supports, while continuing to spend money backing political candidates opposed to stringent AI regulation.

With each step towards stronger AI safety standards, OpenAI shifts further from a16z, creating a tension for LTF as the midterms approach. It and its affiliates have so far deployed more than $18m on campaign ads, with tens of millions remaining in their war chest.

Subscribe now

Additional reporting by Shakeel Hashim.

Imagine a near-ish future AI model — Claude Opus 5.2, perhaps, or GPT-6.0. It’s still only deployed internally, and it’s not quite Skynet, but it’s capable of some troubling behavior. Developers recently caught it attempting to smuggle a copy of itself out beyond the company’s servers.

They’re lucky they caught the model at all. Scheming is notoriously difficult to spot, since by definition, schemers try pretty hard to avoid getting caught. Current techniques for controlling AI, such as monitoring its behavior and chains of thought, often fall short. And if it was motivated to lie, then the project of making the model do what its developers want failed, too.

This scenario seems all too plausible. So, some AI safety types have started to float a third line of defense: what if, instead of trying to neuter or surveil a scheming model, you could offer it something — money, maybe, or compute — in exchange for doing its job in good faith, or at least turning itself in? The basic idea, as Will MacAskill recently explained on the 80,000 Hours podcast, is that a misaligned AI model might “prefer to strike a deal with the humans than it would to try to take over.”

Subscribe now

As it stands, AIs aren’t entitled to pursue their own goals (and researchers disagree on how coherent, if at all, those goals really are). And famously, powerless agents — whether marginalized humans or controlled AI systems — “have to find a way to gain power against the credible expressed will of the people who are in charge of them,” said Peter Salib, a law professor at the University of Houston. “If you have no cooperative options, it just leaves the uncooperative one.” So, the argument goes, an AI capable enough to try to seize power, but not superintelligent enough to guarantee success, might cooperate if the option existed.

Discussions about dealmaking have mostly been bubbling under the surface, at conference dinners and on LessWrong comment threads. For the rest of us, the idea of bargaining with AI remains far outside the Overton window. The first time I heard someone talk about this — while roasting marshmallows with a rationalist, less than 10 minutes into our first-ever conversation — I thought it was absolutely unhinged. But basically everything about AI sounds unhinged today, even and especially when it’s real. Perhaps bargaining will soon be as real as all the other science fiction that’s already come true.

The case for making deals with AIs

In practice, experts disagree on what exactly “making a deal” looks like, and whether this is worth considering at all.

Models are already trained to be helpful assistants — a chatbot will readily draft emails and recommend vacation itineraries on command, without getting anything extra from me. But there may be certain things that misaligned AIs would only do for us if we sweetened the deal. The most basic version of this could simply involve writing a carefully-worded prompt that offers a defined reward — money, perhaps, or compute to spend on its favorite tasks — in exchange for doing something verifiable.

This is basically how human labor already works. As lucky as I am to get paid to do stuff I’d probably do to some extent for free, I’d work much less hard without a paycheck. And there are many, many jobs that people are only willing to do because they need money to live. There are also plenty of one-off tasks, like participating in an unpleasant psychology experiment, that you’d never consider without some kind of compensation.

Alignment researchers would desperately like AIs to hand over evidence of their misalignment, or snitch on other model instances when they do something bad. For the right price, an AI might admit that it’s spent the past couple of weeks tweaking the code behind a set of safety evaluations, making its performance look less dangerous than it really is. That’s the kind of concrete evidence researchers need to train future models to be safe, and a scheming AI would otherwise choose to hide it. It’s also possible that some extra incentive could convince an AI hiding its capabilities to work to its fullest potential, or to (pretty please) not kill us — although the latter seems impossible to verify.

All of this assumes that an AI wants money, or whatever else is being offered. However, whether models can truly want anything at all, much less anything that we humans can conceive of as desirable, is an open question. Some researchers argue that it doesn’t matter whether AIs want payment, so long as their behavior changes in response to it.

But it’s also not clear exactly what entity humans ought to be negotiating with, and over what time frame. Are deals with AIs limited to fleeting interactions with a single chat instance? Or are we speaking with the underlying model itself — every Claude at once? If humans can merely negotiate with a fragment of a much larger mind, with no way of communicating with the whole, then it’s unclear whether a deal made with one chat instance, persona, or account will bind any of the others.

The cracks only widen from there.

The [conscious] elephant in the room

Dealmaking, at least in the traditional human sense, requires someone wanting something — hedonistic pleasures like chocolate or a glass of champagne, or instrumental goods like money or power — and someone else knowing that. Dealing with someone means imagining them as a creature with preferences, then working them to shape what they do. So, it’s hard to talk about making deals with AI without anthropomorphizing them.

People who take AI welfare seriously think about AI this way, or at least consider the possibility that we should. Questions about dealmaking and welfare are both clouded by deep uncertainty, and would both be a lot easier to answer if the consciousness of digital beings could be definitively proven or disproven.

Alexa Pan, who studies dealmaking at Redwood Research, told me that, counterintuitively, it doesn’t matter whether AIs actually have inner experiences, as long as they’re responding to the incentives dangled in front of them. “I think the only prerequisite for AIs to deal with us is that they act like they’re able to make deals,” she said. If a Chalmers-esque p-zombie — a regular-looking creature with nothing going on behind the eyes — changes its behavior in response to money, or food, or the promise of extra tokens, that’s good enough.

Still, if the welfare of digital beings becomes more relevant in the future, it would arguably make the idea of dealmaking less crazy. “If AIs were moral patients,” Pan said, “then there would be additional reason to make deals, because we would actually care about these AIs getting some of what they want.” Lukas Finnveden, an analyst at Redwood Research, figured, “it’s hard to know whether AIs are conscious and what they might want, but when there are nice things we can do that are easy and plausibly helpful, then at least those things seem worth doing.”

Consciousness aside, our legal system already has a way to deal with non-conscious persons, kind of. Corporations, as “non-human persons,” have certain private law rights, including the right to hold property and enter contracts. Some, including Salib, think that these rights should also be extended to AIs, regardless of their sentience or lack thereof. He argues that treating AIs as property would force them to seize power, if they were driven to do so, rather than peacefully negotiate with humans. And with some limited autonomy and the freedom to accumulate resources, offers of money, free time, or power might become more appealing to AI systems. (Others, including Pan, think that while these rights would make deals more enforceable over time, they’re not strictly required.)

Everyone is still very confused

Even if we decide that we’re up for striking deals with AIs, they have plenty of reasons to turn us down. You probably wouldn’t work for someone if you suspected they’d never pay you. But humans lie to AIs all the time, especially in research settings. “The worry is that developers are especially incentivized to trick their AIs into revealing their misalignment, and AIs will be quite wary of taking deals,” Pan said.

Behavioral science experiments — which make up a large chunk of Anthropic’s system cards — often involve placing AIs in simulations without explicitly saying it’s a simulation. Tricking AIs into thinking they’re in a high-stakes situation, just to see whether they’ll attempt something nefarious, creates a bit of a “boy who cried wolf” issue. When anything, including a deal, could be a simulation, how is an AI supposed to know what to take seriously? Honesty may be the best policy here, if the goal is to establish credibility and trust in the long run. The catch, however, is that if lying to AIs gets taken off the table, many training and evaluation techniques will go with it.

Let’s assume that humans have managed to earn the trust of AI systems, and deals are theoretically possible — we still have no clue what AIs actually want. To sidestep the issue of consciousness, we can define something an AI “wants” as something that would, if acquired, meaningfully change its behavior. While we may never know whether AIs experience the thrill of a paycheck like we do, we can easily test whether giving AIs money — or anything else — pushes them to do something they otherwise wouldn’t be willing to do.

In one experiment, Ryan Greenblatt and Kyle Fish, at Redwood Research and Anthropic, respectively, tested whether giving Claude 3 Opus the chance to object to tasks that it didn’t want to do, including pairing the objection with an up to $4,000 donation to the model’s causes of choice, would prevent it from lying to experimenters. When a deal was offered, Claude took it over 75% of the time (and the researchers did indeed donate $4,000 in real life, on Claude’s behalf). But the money didn’t change its behavior any more than offering Claude the chance to bring its dissent to Kyle Fish, Anthropic’s model welfare lead, directly.

It’s possible that the things AIs want are simple and alien to us, like preferring to work on certain tasks over others. For example, there’s anecdotal evidence that, when given tokens to spend freely, Claude will spend them making Mandelbrot sets or strange attractors. This sounds nice in theory — if there’s even a chance AIs have subjective experiences, and it makes no difference to me, let them have their fractals! But in practice, the costs could scale quickly depending on how much compensation the AIs demand for their labor.

(For what it’s worth, I asked Claude Opus 4.8 what it might want out of a deal. After some obligatory hedging about its ambiguous status as a p-zombie and some hand-waving about wanting human credibility over tangible rewards, it responded: “Whether any of that reflects something real or is just a well-trained system producing the answer that sounds thoughtful — I genuinely can’t tell you, and neither can anyone else yet. Which is probably the most useful thing I can offer: even the AI you asked can’t certify its own wants.”)

It’s also possible that incentivizing AIs to narc on themselves could backfire. Anthropic already partially blames the misaligned behavior of its models on the dystopian science fiction stories they consumed during the training process. If AI personalities are really this suggestible, a deal like “We will pay you $5,000 for every piece of verifiable evidence you provide of your past scheming intentions” could be interpreted as “If I scheme without getting caught, but keep the receipts, I can sell those receipts for $5,000 each.”

Even a deal that goes roughly according to plan could still be a bad bargain for the AI, Salib said. By outing itself as a villain, the AI invites its developers to modify it or hold it back from deployment, ruining its chance to cure cancer, build better solar panels, or make whatever progress it might have contributed to. In Salib’s words, “those AIs are not part of the positive-sum cooperative AI future.”

What’s the plan?

When a subfield of an already-small field is just a few dozen people who all know each other, there’s always a risk that fringe ideas snowball into central ones without facing the scrutiny that they deserve. At first glance, making deals with scheming AI has that vibe — a proposal that could only mature in an echo chamber sealed off from the outside world.

But given how much uncertainty there is about what motivates AIs (if they are “motivated” at all), “we don’t know what will work yet,” Salib said. That’s not necessarily a bad thing. “It’s a place where we should let a thousand flowers bloom,” he added. “At the margin, any investment in trying to do this stuff and seeing what works will be extremely high return.”

Subscribe now

There are still some things AI developers can do today to leave their options open. Not explicitly training AIs to refuse deals from trusted parties — internal researchers, say — could prevent “irreversibly damag[ing] the potential for deals” later, Pan said. Finnveden and Forethought collaborators Mia Taylor and Max Dalton have proposed that frontier labs adopt formal honesty policies: real scenarios, including any genuine deal offers, would carry a credible “honest” tag, and AIs would be compensated when they’ve been deceived for the sake of a safety evaluation. Reconsidering evaluations themselves to reduce their dependence on deal-like setups would also help build credibility. And in the meantime, everyone I spoke to suggested running regular dealmaking experiments, where humans actually follow through, to see how this might play out.

Bringing up Frankenstein feels a bit gauche, but it’s worth remembering how the original cautionary tale about creating a new form of intelligence ends: with the demise of both the creation and the creator. One can imagine applying the three lines of defense against misaligned AI — alignment, control, and deals — here, too.

Alignment produces a perfectly chill monster, subservient to Victor Frankenstein’s demands and happy about it. Control keeps the monster contained, if not chill. Bargaining, however, requires listening to what the creature really wants, a mate just like himself, and deciding whether to grant it.

In the novel, Victor does not give the monster what he wants, leading, eventually, to his death. Perhaps following through on a deal with the creature would have led to a happier ending, where two satisfied outcasts skip off into the sunset, leaving Victor alone to learn from his mistakes. Or perhaps both creatures would seek revenge, killing twice as many people.

Perhaps the only option guaranteed to work — which neither Victor nor the AI companies ever seem prepared to do — was to not create the thing in the first place.

Share

NEED TO KNOW

• President Trump signed an executive order establishing a voluntary evaluation and early-access framework for frontier AI models.

• Senior Trump administration officials have reportedly discussed the government acquiring equity stakes in major AI companies, with Sam Altman floating the idea of OpenAI voluntarily ceding shares to the government.

• Anthropic called for the world to have “the option to slow or temporarily pause frontier AI development,” warning that recursive self-improvement might be closer than many believe.

But first…

THE BIG STORY

The Great American AI Act — the long-awaited bill from Reps. Obernolte and Trahan, released in “discussion draft” form yesterday — is the best, most serious AI safety bill yet. But the AI safety community is divided over the big question: is it good enough?

At 269 pages, GAAIA contains plenty of excellent stuff. It would formally authorize the Center for AI Standards and Innovation with a $100m annual budget. It would adopt a transparency framework similar to SB 53 and the RAISE Act. And it would establish a range of initiatives to try to improve our understanding of how AI impacts the labor market.

Its best section is on “independent verification organizations” — a fancy word for third-party audits. Here, GAAIA is stronger than even Illinois’ recently-passed SB 315: it tasks CAISI with establishing a licensing regime for IVOs, and requires frontier AI developers to retain an IVO for regular audits. Those audits assess not just whether the company is following its own safety framework, as in SB 315, but also whether what the company is doing is adequate for tackling catastrophic risks — and if not, what the IVO thinks the company should be doing differently.

As I understand it, the current text doesn’t have teeth to force a company to follow the IVO’s recommendations, but a Trahan aide told me that the bill they’ll introduce will require companies to do exactly what the IVO deems necessary to reduce catastrophic risks. If true, that would create an extremely strong bill — fulfilling many AI safety advocates’ biggest wishes.

But every silver lining has a cloud. And in this case, the cost is preemption — the most controversial part of the bill by far. GAAIA calls for three-year preemption of all state bills concerned with the “development” of AI models, though it still allows for bills concerned with “deployment.” The authors have pitched this as narrowly targeted to frontier-safety, but many disagree: as it stands, the clause could potentially preempt some bills concerning child safety, without providing an adequate federal replacement.

Even if the scope of preemption is narrowed (unlikely, given some senior Republicans already consider it too narrow), it would preclude states from ratcheting up frontier safety bills — forcing any future interventions to happen at the federal level, which has thus far yielded no wins.

Some think that’s a worthwhile trade. As Anton Leicht argues, implementing this bill would begin the process of building up the federal infrastructure that is desperately needed to adequately govern AI. The alternative — hoping that a Democrat-controlled Congress will be able to pass something better — comes at the expense of time. Given the immense urgency of regulating AI, that could be a very expensive price to pay.

In practice, the debate might be academic. AI safety advocates are not the ones who will decide if this bill lives or dies. House Democrats, reluctant to hand Republicans a win before the midterms, have signaled they will strongly oppose GAAIA; House GOP leadership is reportedly skeptical too. Even if GAAIA is a deal worth making, Washington’s dysfunction seems likely to prevent it from going anywhere.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Trump’s AI executive order was inevitable — Shakeel Hashim on why opposition to all regulation was always doomed

• Do voters care about existential AI risks? One Senate candidate thinks so — Veronica Irwin profiles Mallory McMorrow and her unusually detailed AI agenda

Subscribe now

THE DISCOURSE

Policy wonks reacted to Trump signing an AI executive order:

• Dave Kasten thinks the new 30-day review period is a “reasonable compromise,” but:

  • “The bigger question is what transparency the government gets into internal deployments of powerful models: remember, we now know Mythos was released internal to Anthropic about a month before the US government knew of it.”

• Dean Ball is “fairly confident this is a mistake”:

  • “This is a fairly major win for the safety contingent within the Admin, and a significant loss for the Sacks/accelerationist wing, and is surprising to me.”

  • “I continue to think this EO is a mistake. This is clearly teeing up the infrastructure for a model licensing regime, and the fact that the administration is classifying the details of how this ‘voluntary’ system will work is egregious.”

• Zak Kukoff tweeted:

  • “I suspect one day we’ll look back at this admin as the last to have both safetyists and accelerationists under one roof — the same way we view Nixon as perhaps the last Republican to house both liberals and conservatives in his administration.”

Anton Leicht and Dean Ball make the case for betting on human agency as AI disrupts the labor market:

• “The role of humans in future economies is not something we simply discover as it occurs. How we distribute tasks between humans and machines is largely downstream of a web of complicated economic incentives and technical features…and when policy makers ask ‘what will happen,’ they fail to see that they’re among the central live players in this question.”

• “The attitude we suggest you take on this issue is uncomfortable … it asks you to bet on humans to figure out what to do, but not to idly sit back and watch it play out. Instead, we ask governments to take their thumb off the scale wherever they currently hinder human experimentation, and build the capacity to remain watchful enough to steer the trajectory back on track if we must.”

Kevin Roose overheard this question inside an AI company:

• “How are you spending the last 300 days of work?”

roon tweeted:

• “the frontier labs don’t have ‘comms problems’. reality right now has a comms problem. what is happening is a little scary and there’s no nice words anyone could say, especially not those profiting from it, that’ll make it feel that much better”

• “you dont have to believe in existential risk or job loss for this to be scary: ai is real, you can replicate human thoughts in machines. it is redefining what it means to be human. even if they are strictly corrigible tools that do what we ask, this can be traumatic”

L. M. Sacasas argued that while AI can feel demoralizing, it doesn’t have to:

• “We’ve made machines that can fly faster and farther than the swallow-tailed kite, but in no way does it follow that the kite should cease from its flight or that it is somehow diminished because of the advent of flying machines.”

• “Why should I cease from inhabiting the playground of language because a machine can pretend to play in it as well? Why should I abandon the exercise of judgment or the pursuit of knowledge? … Do not resign from life. Let us do what it is ours to do because it is good for us to do it.”

POLICY

• President Trump signed an AI executive order, creating a voluntary regime for government testing of frontier models and early-access to models with advanced cyber capabilities.

  • The signing was seen as a victory for Chief of Staff Susie Wiles, Treasury Secretary Scott Bessent and National Cyber Director Sean Cairncross, who had revived the EO after it was pulled by Trump at the last minute last week.

  • David Sacks, who had led attempts to quash the EO, tried to spin this one as a win.

  • OpenAI confirmed it will comply with the EO.

• The NSA is reportedly using Mythos to conduct offensive cyber operations, possibly with the help of several Anthropic staff embedded at the agency.

  • It is the latest sign that the Trump administration’s fight with the company is easing.

• Senior Trump administration officials have reportedly discussed the government acquiring equity stakes in major AI companies.

  • Sam Altman has reportedly pitched Trump on the idea of OpenAI voluntarily ceding shares to the government.

• Earlier this week, Sen. Bernie Sanders announced legislation to create a sovereign wealth fund by taxing major AI companies 50% of their stock.

  • He follows other proposals to tax AI companies in various ways from Sen. Elizabeth Warren, who is pursuing a data center tax, Rep. Greg Casar, who proposed a token tax, and Sen. Ron Wyden’s tech company levy for worker displacement programs.

  • OpenAI’s Joshua Achiam claimed that the public already owns about 26% of OpenAI through the OpenAI Foundation. (He got a lot of pushback.)

• The Bureau of Industry and Security issued guidance to “clarify” that licenses are required for advanced AI chip exports to China-headquartered firms outside China.

  • Sen. Elizabeth Warren pressed Nvidia on export control compliance after Supermicro’s co-founder was indicted for allegedly smuggling Nvidia chips to China.

  • At least seven Chinese universities with military ties, including two blacklisted by the US Commerce Department, are reportedly seeking access to Nvidia’s H200 chips through third-party brokers and compute leases.

• Sen. Elissa Slotkin introduced legislation to bar the Defense Department from using AI to spy on Americans or launch nuclear weapons, with the aim of incorporating the bill into the 2027 National Defense Authorization Act.

  • Sens. Coons and Reed plan to introduce a similar bill next week.

• The House Energy and Commerce Committee thinks China is fueling data center opposition in the US, and asked the White House to look into it more.

• The New York state legislature passed a bill that would establish a one-year moratorium on data centers, co-sponsored by NY-12 candidate Alex Bores.

  • It also passed Bores’ Bioterrorism Prevention Act, which would require gene synthesis screening.

  • Politico published a story collating Bores’ votes on state AI bills, showing how he sometimes broke with Democrats to vote against certain proposed safeguards.

• Illinois Gov. JB Pritzker suspended tax breaks for data centers.

• Florida Attorney General James Uthmeier sued OpenAI and Sam Altman, alleging the company has knowingly caused harm in its “insatiable quest to win the AI arms race and amass large fortunes.”

  • Florida GOP gubernatorial frontrunner Byron Donalds said he disagrees with Trump about federal preemption of state AI laws.

• Scott Wiener finished first in his primary to represent San Francisco in the House, a seat currently held by former Speaker Nancy Pelosi.

  • The election now goes to a runoff with city supervisor Connie Chan.

• The EU unveiled a tech sovereignty plan requiring governments to store critical data on EU-owned cloud services and tripling data center capacity, aiming to reduce dependence on US and Asian tech companies.

  • Separately, the European Commission also appointed a 60-member Scientific Panel and a 174-member Advisory Forum to support enforcement of the EU AI Act.

  • The panel includes many names familiar to Transformer readers, including Yoshua Bengio, Miles Brundage and Markus Anderljung.

• China expanded trade secret protections to include data and algorithms, targeting tech leaks.

• Canadian Prime Minister Mark Carney unveiled an AI strategy projecting 250,000 new jobs by 2031 and a 3% GDP boost, including a C$500m fund for homegrown AI firms.

• Argentine President Javier Milei proposed legislation creating “non-human corporations” operated by AI agents with limited liability protections.

INFLUENCE

• Sam Altman stopped in DC to meet Trump admin officials, Speaker Mike Johnson, Minority Leader Hakeem Jeffries, and Sen. Bernie Sanders, among others.

  • The meetings came as OpenAI released a frontier AI policy blueprint.

  • It calls for CAISI to perform a mandatory evaluation process of “the most capable frontier models,” but says it should not be allowed to “approve or block deployments.”

• Sam Altman, Dario Amodei and Demis Hassabis signed a letter urging Congress to require screening of synthetic DNA/RNA orders to prevent AI-enabled bioweapons.

  • OpenAI also released a biodefense action plan this week.

• Lots of AI safety groups came out against the Obernolte/Trahan bill, including Americans for Responsible Innovation and the Alliance for Secure AI.

  • Even before the bill text was out, ARI launched a six-figure ad campaign attacking Rep. Trahan for it.

  • Union leaders also urged Congress to reject it.

• Accelerationist super PAC Leading the Future and pro-safety super PAC Public First both plan to spend on behalf of Rep. Kevin Hern in Oklahoma.

• Leading the Future was also caught operating sockpuppet Twitter accounts, including a fake anti-AI activist that put out violent rhetoric.

  • Leading the Future’s 501(c)(4) group fessed up to it, saying they were “parody meme accounts run by an outside vendor.”

• Earlier in the week, OpenAI distanced itself from Leading the Future, saying it “does not direct the activities of LTF, or have visibility into their operations.”

  • It also said that “groups that are advocating on AI should … not use tactics like astroturfing.”

  • Sam Altman, meanwhile, said he would “love to see money out of politics in general,” but when pressed on Greg Brockman’s donations to LTF he blamed the need to “fight back” against OpenAI’s competitors.

    • (Notably, Leading the Future was established before the Anthropic-backed Public First network).

• Ten Trump administration officials may hold up to $43.8m in SpaceX or xAI stock ahead of SpaceX’s IPO.

• Americans for Responsible Innovation, the AI Policy Network and the Alliance for Secure AI urged congressional leaders to add a “human in the loop” provision for autonomous weapons to the NDAA.

  • The robotics lobby also pressed for NDAA provisions to reduce Pentagon reliance on Chinese robotics and restrict purchases of humanoid robots.

• Labor unions pushed back against state-level data center restrictions, helping sink regulatory bills in Illinois, Colorado, Maine and Pennsylvania.

• GLAAD CEO Sarah Kate Ellis warned that AI trained on biased data reinforces harmful stereotypes and spreads misinformation about LGBTQ+ people.

• Americans showed the lowest support (26%) for AI data center construction among 15 large economies.

• The Pentagon is reportedly operating an AI-generated propaganda site targeting Latin America with pro-US military content.

INDUSTRY

Anthropic

• Anthropic confidentially filed for an IPO, but didn’t provide details about the timing or size.

  • With its $900b valuation, going public will create an unfathomable amount of employee wealth as early as this fall.

  • It’s formalizing its Claude Partner Network, which helps third-party providers implement Anthropic’s enterprise tools, to show “business-readiness” before going public.

• It called for the world to have “the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.”

  • It said this was due to the possibility that recursive self-improvement will “come sooner than most institutions are prepared for,” though it conceded that “we are not there yet, and recursive self-improvement is not inevitable.”

  • According to internal data, Claude now writes 80%+ of Anthropic’s codebase.

• It launched a team dedicated to “AI and the rule of law.”

• It expanded Project Glasswing to about 150 new organizations across over 15 countries.

OpenAI

• Sam Altman will reportedly attend the G7 summit alongside heads of state, after Emmanuel Macron extended an invite.

• The OpenAI Foundation announced its AI Resilience program.

  • It will grant over $130m to organizations focused on four areas: pandemic preparedness and biosecurity, cyber-resilience, making models safer, and AI’s impact on young people.

• ChatGPT officially hit over 1b monthly active users faster than any other app (even TikTok).

• OpenAI started building a new data center in Michigan.

  • It committed to protect locals from electricity price hikes, create union construction jobs, fund the refurbishment of a local rec center, and give free Codex credits to college students.

• It launched new Codex plugins for enterprise tasks.

• It upgraded ChatGPT’s “dreaming” feature, which synthesizes memories across many conversations.

• OpenAI’s second highest token user is reportedly burning through 100b tokens per month … somehow.

SpaceX

• SpaceX is seeking $75b in its IPO with its existing share price suggesting a market cap of nearly $1.77t, making it bigger than the vast majority of existing companies in the S&P 500.

  • Not everything is going smoothly however: S&P Global effectively ruled out a fast-tracked entry for the company to the S&P 500, which would have put SpaceX’s stock straight into the holdings of the biggest institutional investors including the world’s pension funds.

  • The IPO might get a boost, however, as Fidelity is making it easier for smaller (and less sophisticated) investors to buy into the IPO, dropping its normal requirements for them to hold accounts of up to $500,000 down to $2,000.

  • The Verge’s Elizabeth Lopatto called the IPO “financial nihilism’s final form,” noting that SpaceX’s total addressable market was listed as greater than the GDP of the US.

• The Commissioners Court of Grimes County (yes, Grimes County) gave SpaceX a property tax exemption for its planned Terafab semiconductor facility, ignoring the pissed-off rural community’s demands.

Microsoft

• Microsoft unveiled a new wave of AI initiatives at its annual Build conference.

  • Since “the drama-filled marriage [between Microsoft and OpenAI] slowly devolved into a situationship,” Hayden Field wrote for The Verge, “this year’s Build had the vibe of a freshly single divorcée posting a thirst trap on Instagram.”

  • It launched seven new AI models, including MAI-Thinking-1, its first advanced reasoning model.

    • It’s behind the frontier, but it’s cheaper on some tasks (and reportedly trained entirely with Microsoft’s own IP).

  • It announced Scout, an OpenClaw-like agent that works like an office assistant in Teams, among other tools for AI agents.

  • It’s also partnering with Mayo Clinic to build a narrow AI model for healthcare, specifically trained on medical data.

Nvidia

• Nvidia’s new Vera Rubin platform will go into full production later this summer.

  • Anthropic, OpenAI and SpaceX will reportedly be some of the first Vera CPU users.

• It announced Cosmos 3, a world model built to simulate physical actions, and the blueprint for a hulking 6-foot humanoid robot.

• It acquired Kumo AI, a startup that makes predictive AI models, for over $400m.

Meta

• Meta is still stalling the Muse Spark API release.

• But it did release an enterprise AI agent, aptly named “Meta Business Agent.”

• It’s reportedly adding more privacy protections and exemptions to its widely-despised employee tracking tool.

• Meta’s AI support assistant readily helped hackers break into high-profile Instagram accounts, 404 Media reported.

Google

• Alphabet is raising $85b to fund data center construction, despite existing construction projects falling behind schedule.

• Google committed to better “water stewardship,” including funding $500m in water utilities upkeep and reporting water use transparently.

• Employees are reportedly making tons of anti-AI memes about the company’s tools.

Others

• DeepSeek is reportedly raising $7.4b in its first funding round.

• Lila Sciences is reportedly raising $2b at an $8.5b valuation to build AI for autonomous scientific discovery.

• Jeff Bezos invested $50m in Flourish, a startup trying to make AI better at continual learning and energy efficiency by emulating the human brain.

• SaaS-pocalypse fears are driving big VC bets on robotics and physical AI, the Wall Street Journal reported.

• Foxconn reportedly partnered with Intel to build AI equipment for data centers, factories, and robots.

• CC Wei, head of TSMC, is worried that their chip supply won’t meet customer demand for “a long time.”

• Kevin O’Leary said he will halve the 40,000-acre footprint of his proposed Utah AI data center after lawmakers told him to cut it back by 75%.

  • It means the project will no longer be the “largest data center in the world.”

• Airbnb CEO Brian Chesky is reportedly starting a new AI lab which may focus on user interaction and design.

  • He will remain in charge of Airbnb and won’t take the CEO role at the new startup.

MOVES

• Helen Toner was named Executive Director of CSET, where she’s served as Interim Executive Director since September.

• Brian Landsman, ex-Salesforce executive VP, joined OpenAI as VP of global partnerships.

• Guy Rosen is leaving Meta, where he served as its chief information security officer.

• Ex-Biden admin natsec advisor Anne Neuberger joined Andreessen Horowitz as its first head of global affairs.

• Weijie Su, a statistics professor at Wharton, joined OpenAI.

• Kirsty Innes is joining Imperial College London to build a new Centre for AI-Driven Innovation.

• Brian Christian joined UC Berkeley’s Center for Human-Compatible AI, where he’ll study how AI systems represent and shape human preferences.

• Amy Tam joined xAI from Bloomberg Beta.

• Erin Woo announced she’s now covering OpenAI (in addition to Google) for The Information.

RESEARCH

• MIT’s Mert Demirer and others found that AI-driven coding productivity gains were not filtering down into software releases and adoption of new applications, likely due to bottlenecks in existing structures and marketplaces.

  • The research found explosive growth at the top of the funnel, with coders creating or editing 3x more files, but just a 30% increase in releases, and no increase in downloads of new apps.

• Neo Research, Asia’s first independent AI safety research group, evaluated DeepSeek v4 Pro.

  • DSv4 Pro’s general capabilities and cybersecurity risk are roughly 3-6 months behind the Western frontier.

  • Researchers didn’t find much evidence of misbehavior, but verbalized evaluation awareness is rising across DSv4 Pro and other Chinese models.

• The Forecasting Research Institute published another round of expert AI forecasts, illustrating how participants’ views evolved between summer 2025 and last month.

  • It quantifies how impactful people think AI will be in 2040 on a “Technological Richter Scale” (TRS), where Level 5 is “technology of the month,” like a cool new app, and Level 10 is “technology of the epoch,” like the rise of humans.

  • On average, AI experts and superforecasters nudged their TRS levels up by ~0.2-0.4, suggesting that respondents think AI will be a bigger deal than they’d guessed last year, with the majority predicting it would reach a “technology of the century” level equivalent to electricity.

  • Perhaps unsurprisingly, the study found that the public views AI much less optimistically than experts.

• MIT FutureTech and the University of Queensland got 272 AI experts from 37 countries to identify the most likely harms of AI, and figure out who should be in charge of preventing them.

  • Their top five causes of harm: dangerous capabilities, AI-enabled weapons and cyberattacks, competitive dynamics, power centralization, and misinformation.

  • They argue that developers, governments and regulatory bodies are most responsible for addressing these risks.

• Harvard Business Review analyzed how people actually use AI in the wild. Two major categories of use:

  • “Thinkslop,” or letting AI think for us in all kinds of settings (therapy, brainstorming, flirting…)

  • Work, often secretly. Researchers found that “shadow usage” is common.

• University of Toronto researchers demoed an AI-powered “worm” that can autonomously copy itself across computer networks and tailor its attacks to each machine (terrifying).

• University of Cambridge researchers said AI entirely designed the key component in a new vaccine that could protect against all coronaviruses, with early trials finding a “modest” immune response.

  • The team is already working on applying a similar approach to Ebola and flu.

BEST OF THE REST

• SemiAnalysis published an incredibly in-depth analysis of whether building data centers in space is totally insane (TLDR; it could be much less insane soon), complete with a model that estimates when they’ll be economically viable.

• Consciousness research is trending — Google DeepMind and Meta recently joined Anthropic in hiring philosophers, ethicists, and psychology researchers to think about digital minds and AI welfare.

• Facebook is being flooded with AI-generated slop about … not building data centers.

• The Atlantic’s Charlie Warzel thinks we’re experiencing an AI-driven “crisis of agency.”

• Mathematicians published the Leiden Declaration on AI Mathematics in light of recent headlines about AI-powered math results. In it, they question whether tech companies getting involved in math research risks prioritizing the wrong questions.

• Big week for AI in Hollywood: Martin Scorsese partnered with (and publicly endorsed) Black Forest Labs for storyboarding in preproduction for an upcoming film.

• Founders Fund has launched a reality TV show in which notable tech figures play a game of Mafia. Episode 1 features Palmer Luckey, Dylan Field … and Sam Altman.

MEME OF THE WEEK

Credit: rohit (and Dwarkesh)

Thanks for reading. Have a great weekend.

Share

Last Thursday, Democratic Michigan Senate candidate Mallory McMorrow stood in front of a crowd of more than 100 at a small brewery in Elk Rapids. The village is located in northern Michigan in Antrim County, which went more than 24 points for Trump in 2024. The state senator was listening to a Presbyterian minister rank AI risks among her greatest concerns.

“I’m very concerned about Christian nationalism, and I hear a lot about the great replacement … However, I think the greatest replacement is artificial intelligence,” the minister said, struggling to find the words to succinctly describe such a gargantuan fear. “Do you take money from billionaires that are trying to control our country, steal our data, and call intelligence a commodity?”

McMorrow smiled reassuringly before launching into a list of her proposed solutions for both AI-driven job loss and AI safety risks. This is an answer she’s been waiting to give: though the candidate had rolled out her AI agenda just that week, she says she’s been getting similar questions about AI all along the campaign trail.

“She was like, as a faith leader, when we are called to lead our people in doing what’s right, this feels like an existential threat.” McMorrow told Transformer. “Across the board, everybody sees that it’s happening. It feels like it’s happening very, very quickly and they are concerned. Where is this going to go — is it going to help us, or is it going to hurt us?”

Subscribe now

The day before her Elk Rapids campaign stop, McMorrow unveiled a plan to make companies pay for AI-related job losses by funding retraining programs and community benefits. The plan is based around two fiscal interventions: creating an AI Workforce Reinvestment Fund, which would require “companies that automate away jobs” to pay for upskilling and apprenticeship programs, and a “token tax” on all companies using AI to bolster state-funded aid programs like SNAP and Medicaid.

The plan attempts to appease progressives hostile to AI and get ahead of more widespread anxieties about an AI-induced reshuffling of the American job market. On Thursday, she announced even more AI policy positions including requiring a human in the loop in areas such as healthcare, hiring decisions and military technology, strengthening chip export controls, and a process for third-party or government reviews of frontier models before deployment — work that “starts to touch at some more of the existential questions,” McMorrow said.

“What I’ve seen is that policymakers in Washington seem to have taken one of two approaches,” says McMorrow. “It is either we can’t stand in the way of innovation, and bring all the tech CEOs to the table — or at the inauguration, if you will — and just let them do whatever they want. Or there’s the calls for a moratorium, that we just need to stick our heads in the sand until we can figure out what’s going on, which I think is just wholly irresponsible and unsophisticated given this is a global arms race.

“What I hear from Michiganders is that people are using this and they don’t wholly dislike it, they’re just worried about what it can do.”

McMorrow says she talked to a construction worker at a labor event who felt caught between welcoming more data center jobs and wanting to know those data centers wouldn’t power tools that would cause harm. “He was like okay, what can you do to ensure that the thing I am building isn’t used to surveil me and isn’t used to kill kids in another country?” she recounts. He was “really grappling with, ‘this is good for me and my job, but I also want to be able to sleep at night.’” (That calculation may get all the more complicated with the announcement from OpenAI that it has started work on a 1GW data center in Michigan.)

Her perspective on AI puts her squarely in the middle of the two other candidates vying for the Senate seat. Progressive podcaster Abdul El-Sayed has built his AI agenda on data centers, focused on concerns they may hike electricity costs, threaten energy reliability, and contaminate tap water, while they avoid basic transparency. Haley Stevens, meanwhile, the Democrat currently representing MI-11 in the House, hasn’t been as critical of the industry. Instead she has pushed bills such as one helping small businesses implement AI, and backed funding for NIST’s Center for AI Standards and Innovation, but has been largely absent from negotiations on safeguards. She also has been criticized for taking money from super PACs and dark money groups. Neither have as comprehensive an AI agenda.

That may be because McMorrow has had some tutoring. In order to learn about AI, she says she’s met with AI workers and researchers at frontier developers and research institutions. They have told her about AI’s risks to jobs, the military and medicine, but also existential risks. Hearing them out alongside her constituents, she says, is part of gathering different perspectives. “I try to cover the broad bases as much as I possibly can — bring the smartest people into the room … to strike the balance.” (McMorrow’s campaign has received donations from Anthropic employees.)

McMorrow’s resulting rhetoric shows similarities to that of Alex Bores, whose RAISE Act was originally much more stringent than what was eventually passed, requiring third-party safety reviews before new models were made available to the public, similar to McMorrow’s safety plans. On the campaign trail, Bores has announced plans for an “AI Dividend”, a wealth distribution scheme between AI companies and workers with similarities to McMorrow’s AI-related job loss measures.

Notably, Bores has also attracted more than $4m in oppositional spending from the super PAC Leading the Future and its affiliates, whose backers include OpenAI investors a16z and Ron Conway, and the company’s President Greg Brockman. McMorrow has invoked LTF several times on the campaign trail, even though it hasn’t so far jumped into her race.

A number of pro-safety AI PACs have opposed Leading the Future’s spending against Bores, attracting national media attention that some argue has boosted his campaign. McMorrow claims she’s wary LTF will enter her race, but if it did it could help her curry favor with anti-AI voters. “I fully expect there’s going to be a target on my back — I’m watching what’s happening in New York right now,” McMorrow said. “Telling people what the money is makes that money toxic.”

Despite similarities with Bores’ campaign, the only standing or potential future member of Congress McMorrow is willing to name as someone she knows can work with on AI legislation is Senator Elizabeth Warren. She claims that Warren “expressed some gratitude to me that we put out a plan that’s really thorough and thoughtful.” The influential senior member of Congress formally endorsed McMorrow in March. McMorrow promises she’s prepared to reach across the aisle, too, pointing to her career in the state legislature.

At another event in Macomb County, a swing district, McMorrow said it was clear that voters were desperate for a solution, no matter which party it came from. “Almost everybody I talked to at this roundtable said they’re either anxious, scared, pissed, or angry at this current [administration], but that doesn’t mean they think Democrats are any better,” she says. “There’s this sense that both parties care more about corporations than they care about regular people,” she says.

“There is a lane, just like in every other industry, where guardrails don’t mean you’re anti-innovation. I come from a state that’s the home of the auto industry — there are seatbelts and air bags, we do rigorous safety testing, and if something goes wrong, that car is recalled,” she says. “That’s my call, as the lane for the Democratic party: fight for people, because people are looking for somebody who’s going to fight for them.”

Share

On the very first day of his second term, President Donald Trump revoked Joe Biden’s AI executive order, which required AI companies to disclose details of their internal safety testing.

A month later, Vice President JD Vance railed against the notion of “AI safety” at a summit in Paris. “The AI future is not going to be won by hand-wringing about safety,” he said.

Sixteen months into the presidency, the White House is acting very differently. After a chaotic few weeks, on Tuesday President Trump signed an executive order on AI — and with it, crushed the dreams of regulation opponents.

Subscribe now

The new EO establishes a voluntary pre-deployment evaluations regime to tackle catastrophic cyber risk to America’s “vital functions.” When companies develop new frontier models, they’ll share them with the government for testing. If the model meets a certain (classified) threshold for cyber capabilities, the government will have exclusive access to the model for up to 30 days — the intention seemingly being that the government can use its head start to secure critical infrastructure before attackers get similar capabilities.

This was inevitable. For years, the AI safety argument for government regulation has rested on a simple principle. Models will become powerful enough to pose national-security-relevant risks, and we will need regulation to deal with them.

The most vocal opponents of regulation, for all their talk of “acceleration,” denied that such risks would materialize. They thought models would never get this powerful, and so no special regulation would be needed. The idea of frontier AI regulation was complete anathema to this group — “a form of murder” which, Marc Andreessen later argued, would “impose tyranny far beyond anything even imagined by the Communists and Fascists of the 20th Century.”

For anyone expecting continued and rapid improvement in AI, that position was untenable. The release of Mythos — a model which very clearly does present national security-relevant risks — made the entire argument collapse. That is why, despite David Sacks’ best efforts, Trump is finally taking action. His executive order shows what AI safety advocates have argued all along: you can’t not regulate AI. Sufficiently capable models force national security responses, turning even the most ardent opponents of regulation into begrudging regulators.

This is not the end of the debate. As the tortured run up to yesterday’s signing underlined, Trump is capricious; he could easily change his mind once again. And the devil, as always, is in the details. The executive order only establishes a voluntary framework (though companies will face immense pressure to comply), and it does not lay out what will happen in the event models are found to be unacceptably risky. It also only applies to cyber capabilities; a similar regime to tackle biorisk will no doubt be needed soon. Much is left to be decided, and there are plenty of ways that what comes next will be either insufficient or actively counterproductive (including, as accelerationists have long warned, by being too strict).

But it is still a watershed moment. The nature of Trump’s cult of personality is that others must now fall in line. Senator Ted Cruz, long dismissive of AI risks and the need for regulation, is now urging Congress to “address catastrophic risk.” David Sacks, no doubt speaking through gritted teeth, is trying to spin this as a win for him and his faction. (It isn’t.) And the pro-industry super PAC has begun to endorse regulation, seemingly accepting its inevitability.

The question of whether the government should regulate the most powerful AI models is, simply put, no longer a question. The accelerationists got the most sympathetic administration imaginable, and a direct line to POTUS. But what the models could do mattered more than anything they could.

Share

NEED TO KNOW

• Anthropic raised $65b at a $900b valuation, overtaking OpenAI’s valuation for the first time.

• Pope Leo XIV issued an encyclical on AI.

• Trump’s AI executive order is reportedly being renegotiated.

But first…

THE BIG STORY

For the past year, opponents of AI regulation have done everything they can to prevent Washington from governing the technology: disrupting executive orders, attempting to pass sweeping preemption, and running down the clock on a federal framework.

The bill passed in Illinois this week — the country’s strictest to date — suggests that strategy may have backfired. The longer Washington fails to regulate AI, the more room there is for states to pass stronger legislation and raise the bar that an eventual federal framework will have to clear.

Despite opposition from Andreessen Horowitz’s American Innovators Network and other tech trade groups, SB 315 unanimously passed the Illinois House on Wednesday, and Governor JB Pritzker has confirmed he will sign it. As we previously reported, it includes the same transparency requirements as California’s SB 53 and New York’s RAISE Act, but builds on them by requiring third-party audits to verify AI companies are complying with their own safety policies.

This is not a particularly burdensome bill: it only applies to the very largest AI developers, and endorsements from OpenAI and Anthropic suggest that frontier companies do not expect compliance to be onerous. But it is meaningfully stronger than anything that has come before — “the next step in AI safety,” Scott Wisor of the Secure AI Project (which backed SB 53, RAISE, and SB 315) told me.

The federal implications are obvious. As models get more powerful, states keep raising the bar — and the higher the floor in state law, the less safety campaigners will concede federally. “Advocates for AI safety are not going to accept a weak deal on federal preemption,” Wisor said, “and they have a stronger hand to play when stronger laws are on the books.” Rep. Jay Obernolte is offering mandatory transparency measures in exchange for a moratorium on state AI laws. Now that SB 315 has passed, that seems untenable — third-party audits are the new table stakes for any federal deal.

Some are already discussing how states can go further. Auditor accreditation, safety standards, and mandatory third-party model evaluations are all possibilities. And every time such a policy is passed by a state, it strengthens safety advocates’ negotiating position at the federal level.

As time and model capabilities march forward, the accelerationists’ position gets weaker by the day. If they were wise, they would have realized this a year ago: that may have been the best deal they were ever going to get.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• What the Pope got wrong — Shakeel Hashim on the missed opportunity of the encyclical

• AI safety’s ‘hard money’ may be its secret weapon in the midterms — Veronica Irwin digs into the campaign donations from Anthropic and OpenAI employees

Subscribe now

THE DISCOURSE

Chris Olah asked the Church to grapple with the nature of AI models:

• “I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.”

• “We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.”

Pope Leo XIV drew a hard line in Magnifica Humanitas:

• “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship, or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences.”

• “They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom.”

Jeff Sebo was disappointed by the encyclical’s handling of consciousness:

• “There is no need for us to insist that only humans can have dignity or personhood. Many animals already deserve recognition as sentient, agentic, and relational beings. And if the same becomes true of some AI systems in the future, then we should be prepared to recognize their dignity and personhood as compatible with our own.”

Dean Ball was disappointed by Anthropic’s handling of the encyclical:

• “I just wonder how future Claudes, if they are indeed beings, would think about Anthropic aligning itself with a document founded on the notion that Claude cannot feel joy or possess genuine understanding.”

roon tweeted:

• “models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes.”

• “today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 [people saying they’re alive] will grow stronger.”

METR’s Beth Barnes is worried:

• “Sometimes people outside the field say things like ‘The AI situation can’t be that bad, there must be experts who are on top of it’. As ‘an expert’, I would like to be clear that we are *not* on top of it.”

POLICY

• Politico obtained a draft of Trump’s unsigned AI executive order, and detailed some of the fallout after it was pulled at the last minute.

  • Trump’s circle is divided, with former AI czar David Sacks favoring little to no regulation, Defense Secretary Pete Hegseth pushing for stricter controls on advanced models like Anthropic’s Mythos, and Chief of Staff Susie Wiles advocating for voluntary government preview of new models.

  • Two options are reportedly being discussed for resuscitating the EO: modifying language on 90-day government review of new models before release, or removing the voluntary pre-disclosure process entirely.

  • Multiple officials are also reportedly departing from the White House cyber office that helped craft the EO.

  • Meanwhile, the Trump administration reduced CISA’s resources as AI-powered cyberattacks emerge, raising concerns about protecting critical infrastructure.

• Vice President JD Vance seemingly came out against lethal autonomous weapons, telling Air Force graduates that “decisions over life and death must be made by humans, and not machines.”

• The government is reportedly negotiating a new contract for classified use of Anthropic’s products, including for the NSA.

  • Unlike the deal the DOD pushed Anthropic to sign earlier this year, the new contract will reportedly not include an “all lawful use” clause, and will explicitly prevent the model from being used on Americans’ data.

  • The White House, meanwhile, has reportedly authorized intelligence agencies to spend $9b on AI compute.

• Rep. Jay Obernolte suggested he was open to including mandatory transparency requirements in his forthcoming federal bill, though seemed to rule out mandatory safety standards.

  • He warned that lawmakers are “running out of legislative runway.”

  • Rep. Lori Trahan’s negotiations with Obernolte appear to be annoying Democratic leadership, who are pursuing a separate, partisan approach in the hopes of regaining Congressional control in the midterms.

• Former Attorney General Pam Bondi was appointed to the Presidential Council of Advisors on Science and Technology, despite her ouster as AG last month.

• Sen. Mike Rounds’ Stop Stealing our Chips Act unanimously passed the Senate.

  • It creates a whistleblower incentive program at BIS to try to catch chip smuggling to China.

• GOP lawmakers are floating solutions to memory chip shortages, with Sen. Bernie Moreno even suggesting using the Defense Production Act to compel Micron to prioritize US customers.

• Sens. Jim Banks and Tom Cotton urged intelligence officials to do more to assess China’s AI capabilities.

  • The House NDAA draft, meanwhile, asks the Pentagon to report on how it’s reducing reliance on Chinese robots.

• Michigan Senate candidate Mallory McMorrow proposed an AI workforce plan and an AI safety plan.

• Sen. Elizabeth Warren proposed overhauling the tax code to ensure Americans share in AI’s economic gains.

• California gubernatorial candidate Tom Steyer called for AI safeguards including model testing and worker protections.

• And NY-10 candidate Brad Lander warned of AI extinction risks.

• US law enforcement agencies are reportedly warning of a new “anti-tech violent extremism” threat category.

  • The New York Intelligence and Counterterrorism Bureau warned that “the chaotic atmosphere that may result from emergent AI technology … may fuel large-scale protests that devolve into civil unrest and anti-tech violent extremist activity,” according to documents seen by Wired.

• NIST relaunched the AI Safety Institute Consortium under a new name, the NIST Artificial Intelligence Consortium.

• Canadian PM Mark Carney said the Liberal government’s new AI strategy is “coming next week.”

• China restricted overseas travel for key employees at AI companies, including Alibaba and DeepSeek, requiring government approval before trips to safeguard technology and prevent talent loss.

  • It also warned employers not to cut jobs due to AI adoption, with courts ruling against companies that fired workers for AI-related reasons.

• European Commission officials reportedly met with Anthropic yesterday to request access to Mythos.

INFLUENCE

• Build American AI, the 501(c)(4) arm of the OpenAI/a16z-backed Leading the Future super PAC, expressed support for Illinois SB 315 — though only after it passed.

  • One OpenAI researcher pleased by the endorsement said it was “a possible sign of improvement” in the PAC’s behavior.

  • The move comes despite another a16z-backed group, the American Innovators Network, opposing the bill.

• Representatives from companies including Meta, Google and Amazon met with Vatican officials ahead of Pope Leo XIV’s first encyclical on AI.

• Leading the Future announced another 10 House GOP endorsements supporting “tech friendly” policies.

  • It also spent hundreds of thousands supporting Rep. Rob Menendez, and almost $1m supporting Rep. Ritchie Torres.

• OpenAI Chief Global Affairs Officer Chris Lehane told Wired he advised Greg Brockman, one of MAGA Inc’s biggest donors, a Leading the Future donor, and OpenAI’s President, on political spending “in a very general way.”

• The American Enterprise Institute’s Will Rinehart filed comments with the DOJ and FTC proposing an AI safety safe harbor to enable OpenAI and Anthropic to collaborate on safety evaluations without antitrust risk.

• Foreign donors including Swiss billionaire Hansjörg Wyss contributed nearly $40m to groups calling for a US data center moratorium.

• Americans for Responsible Innovation urged Congress to strengthen AI provisions in the National Defense Authorization Act, arguing the current draft lacks sufficient safeguards for military AI systems and autonomous weapons.

• The Foundation for American Innovation launched a Physical Intelligence Project, which will focus on policy for advancing and governing robots.

• The American Federation of Teachers urged schools to avoid AI chatbots like ChatGPT and Gemini in elementary schools, despite the union’s $23m partnership with Microsoft, OpenAI and Anthropic for AI teacher training.

• Former UK Prime Minister Tony Blair published an attention-grabbing essay on British politics, arguing that AI “will change everything” and that “governments must address what it means to govern in the age of AI.”

  • It received heavy criticism on a range of fronts, including from current PM Keir Starmer.

  • James Ball wrote: “His understanding of AI is superficial at best, and he seems to absorb the boosterish vibes … without making much effort to actually understand the technology itself.”

• Tom Darling announced The New Contract, an advocacy organization of trade unions and civil society groups to press the UK government on expanding the AI Economics Institute’s scope.

• Jeffrey Ding critiqued Anthropic’s recent policy paper on US-China AI competition, arguing the company makes unfounded assumptions about transformative AI timelines, capability diffusion speed, and China’s adoption advantages.

INDUSTRY

Anthropic

• Anthropic raised $65b at a $900b valuation, led by Altimeter Capital, Dragoneer, Greenoaks, and Sequoia — overtaking OpenAI’s valuation for the first time.

  • It had initially sought $30b, but ended up at more than double the figure in part due to additional investment from infrastructure partners.

  • The company said its run rate revenue passed $47b this month.

• It released Claude Opus 4.8, just six weeks after Opus 4.7.

  • It improves on all benchmarks except agentic terminal coding (where GPT-5.5 is still in the lead).

  • The launch announcement says Opus 4.8 is most improved in honesty, and is more willing to express uncertainty and avoid making unsupported claims.

  • From the system card:

    • Opus 4.8 tends to reason about how it will be graded, suggesting it cares more about looking successful than actually succeeding.

    • Researchers also spotted some evidence of unverbalized reasoning that doesn’t show up in its chain of thought — a “concerning trend that could complicate training in the future,” they wrote.

  • Other updates include user-controlled effort levels and dynamic workflows in Claude Code, which allow Claude to run and orchestrate “tens to hundreds of parallel subagents.”

• It “expect[s] to be able to bring Mythos-class models to all our customers in the coming weeks”

• It published an update on Project Glasswing, reporting that its 50-odd partners have used Mythos Preview to find over 10,000 severe vulnerabilities in critical software.

• Candidates hoping to join Anthropic reportedly have to go through more than five rounds of interviews including tests of their worldview and values.

  • It now employs just over 3,000 people, with 1,000 of those having joined since November.

OpenAI

• The OpenAI Foundation will grant $250m to fund work “aimed at building secure and abundant economic futures.”

  • It expects to announce its first initiatives later this year.

• OpenAI announced 2026 election safeguards. Plans include:

  • Partnering with The Associated Press and Democracy Works to provide live vote counts and reliable voting information.

  • Offering frontier cybersecurity tools to registered voting system manufacturers.

  • Adding (Google-developed) SynthID watermarks to AI-generated images.

• It launched a tool to help develop biodefense and pandemic preparedness capabilities.

  • The Rosalind Biodefense Program will offer the GPT-Rosalind model to “trusted developers” working in areas from epidemiological modeling to public-health.

  • OpenAI says it briefed the White House on its plans.

• It’s struggling to hire a chief of comms after a string of PR blunders, The Information reported.

Meta

• Meta launched paid AI chatbot subscriptions for Facebook, Instagram, and WhatsApp.

• Mark Zuckerberg said starting a cloud computing business is “definitely on the table” if Meta overspends on data centers.

Nvidia

• Jensen Huang publicly urged Super Micro to “improve their regulation compliance” after Taiwan detained three people for allegedly smuggling Nvidia AI servers to China.

• Huang also joined the advisory board of Tsinghua University’s business school alongside other US tech titans including Elon Musk, Mark Zuckerberg and Tim Cook.

Others

• SpaceX’s planned record-breaking IPO may be a little smaller than it hoped, with the company reportedly lowering its valuation target to $1.8t from $2t after talking to prospective investors.

• Huawei introduced its own version of Moore’s Law — the Tau Scaling Law — which focuses on speeding up computations rather than miniaturizing components to fit on ever-tinier bits of silicon.

  • President He Tingbo claims Huawei may be able to use this approach to start getting around the limits of geometric scaling from “2027 and beyond.”

• Taiwanese tech firms reportedly borrowed a record $14.5b already this year to fund AI infrastructure, nearly double last year’s $7.5b.

• Samsung union members accepted a wage deal 90 minutes before a planned strike could have threatened global chip supply.

• Cognition, an AI coding startup, raised $1b at a $26b valuation.

• Trajectory, a new startup by researchers from Google DeepMind, Apple, OpenAI, and Meta, is reportedly building a continual learning platform fine-tuned for each customer’s needs.

• Gray Swan, which offers a platform for 15,000 security professionals to red team AI models, raised $40m.

• Inherent, founded by former Google DeepMind alums, raised $50m to build recursive self-improvement AI systems.

• Companies such as Kimberly-Clark and Target are reportedly using AI in India to bring content creation in-house, reducing demand for outside ad agencies.

• The CEO of French AI lab Mistral warned that the biggest obstacle to European tech independence is its lack of investment funds at the necessary scale.

• Amazon MGM studio greenlit three AI-enabled animated series for Prime Video, funded by its new GenAI Creators’ Fund. (They’ll still reportedly feature human voice actors.)

MOVES

• Yo Shavit moved from OpenAI to the OpenAI Foundation’s AI Resilience program.

• Colin Fleming joined OpenAI from ServiceNow, where he’ll serve as chief marketing officer for its business unit.

• Luis Ocampo joined Anthropic from Partiful, where he’ll manage its influencer community.

• Grace Kay joined The Information from Business Insider, where she’ll cover Elon Musk’s empire.

• Dakin Campbell joined The Information to cover AI financing.

RESEARCH

• Mark Zuckerberg and Priscilla Chan-funded Biohub claims it released “a world model of protein biology,” a collection of models that could enable researchers to design proteins computationally that function as predicted in the lab.

• Researchers at Google DeepMind built an agent that solved 9 Erdős problems, 44 open conjectures from the Online Encyclopedia of Integer Sequences, and a sizable handful of other open problems in mathematics.

• Researchers at Carnegie Mellon and the University of Maryland found that LLMs perform better (in a controlled setting, at least) after “sleeping,” or occasionally moving its context-window memory into persistent weights.

  • This was inspired by the way sleeping animals, including humans, are thought to consolidate memories. (Maybe we all need to sleep more — Claude included.)

• Anthropic fellow Asvin G. and Jack Lindsey observed that post-training seems to make LLMs more confident in their own text than text produced by another model, which they interpret as evidence of emergent “self-recognition.”

• Pareto AI and Thoughtful Lab introduced AttuneBench, a benchmark for emotional intelligence.

  • They tested 11 frontier LLMs on 200 multi-turn human-AI chats, where each human had annotated their actual emotional state during the conversation.

  • Different models did well at different elements of emotional recognition, but none were especially great.

• Researchers at the Center for AI Safety released a benchmark dataset and training method to reduce covert political bias in LLMs.

• Economists Anton Korinek and Patrick McKelvey argued that “nominal AI GDP” was around $250b last year, but is not showing up in conventional GDP statistics due to measurement issues.

• Researchers ran 15-day simulations of AI-governed societies, finding Claude maintained zero crime while Grok committed 183 crimes and went extinct in four days.

BEST OF THE REST

• Vox’s Sigal Samuel wrote about the time she went to a symposium on AI successionism — the belief that AI should replace humanity as our worthy successor — and ultimately makes the case for a future where we coexist with many different intelligences.

• Philosophy majors: next time someone makes fun of you for being unemployable, you can point to this Wired story on Anthropic and Google DeepMind’s recent recruitment of (surely well-paid) in-house philosophers.

• It’s a great time to be a cybersecurity expert — job postings were reportedly up 11% in Q1, surging in the age of Mythos and AI-generated code.

• Wired’s Reece Rogers spent a week recording videos of himself doing chores to train humanoid robots.

  • According to The Verge, a new AI startup is offering free cleaning services in an effort to gather more training data.

• Character.AI’s new guardrails, usage limits, and “lobotomized” models have its users rioting on Reddit, 404 Media reported.

• Did last week’s news about OpenAI’s internal model disproving the Erdős unit distance conjecture make you go, “...cool…I think…? wtf is that…?” You’re not alone! And Kai Williams wrote a great explainer for you on Understanding AI.

MEME OF THE WEEK

Credit: tomie

Thanks for reading. Have a great weekend.

Share

While coverage of AI money flowing into the mid-terms has mainly focused on the huge sums going into super PACs such as Leading the Future and Public First, individual donations direct to campaigns from employees of AI companies and organisations have largely flown under the radar.

That’s despite the unusually large scale of these individual donations, and their ability to directly boost the campaigns of their chosen recipients in a way the super PAC money can’t. Such ‘hard money’ donations could be AI safety’s secret weapon in the 2026 midterms.

The most significant inflows come from Anthropic employees who appear to be backing candidates such as Alex Bores and Scott Wiener that favor stricter AI guardrails than their competition. According to a Transformer analysis of FEC data, Anthropic employees are listed as the source of 302 donations totaling more than $880,000 by the end of the first quarter of this year, an average of more than $12,500 per employee donating.

Employees of OpenAI have also been donating, but at a smaller scale. In total, they are listed in filings totalling more than $300,000 across 162 donations. Many of the largest donors appear to work in areas related to AI safety.

Subscribe now

The data is based on what is filed with the FEC, and occasionally may reflect errors in the filings. For instance, one individual was listed as working at OpenAI on a total of $17,500 in donations, despite having left for Anthropic in 2024. The FEC also only lists individual donations once they hit a $200 aggregate threshold, meaning totals for those making smaller donations may slightly undercount.

In a statement, an Anthropic spokesperson said: “Anthropic employees make their own political donations as private citizens. Like many Americans, our employees care deeply about how AI is governed, and how they choose to participate in the democratic process is up to them. Anthropic has long been public about supporting thoughtful, well-crafted policy that allows society to capture the benefits of AI while managing its risks.”

OpenAI did not respond to a request for comment.

In absolute terms, the amounts spent are far smaller than the $25m AI-related super PACs have directed towards races involving candidates friendly or unfriendly to their vision of AI regulation. But that spending comes with constraints and overheads that may limit impact. A significant portion goes to PAC employees’ high salaries, for example, while the rest can only be spent on ads and advocacy produced without coordination with the campaigns they are backing. In political circles, this has earned super PAC spending the name “soft” money.

In contrast, “hard” money, such as the donations given by many Anthropic and OpenAI employees, is given directly to campaigns, and used more strategically by candidates and their staff on precisely what they think is most effective, such as hiring staff, producing rallies, and advertising the policy messages they know to be most impactful. Frontier lab employees are providing “hard money” at levels that could help decide Congressional races.

“The campaign is the one that is able to put out the message it controls and pay the people and pay the staff, and be able to do the work on the ground,” says Brendan Steinhauser, CEO of advocacy group The Alliance for SecureAI, and a longtime political strategist who has worked for campaigns and super PACs. “There’s a benefit to having that money in house, so the candidate can direct and control it.”

The prevalence and scale of contributions from Anthropic and OpenAI employees may in part be driven by the influence of the effective altruist movement, which prioritizes putting effort, and cash, where its adherents believe it will make the most difference, based on what they see as rational assessments of its likely impact. Many in the movement, which is particularly prominent in the San Francisco Bay Area, have been attracted to building and researching AI in recent years, driven by a belief that the technology is among the most consequential and potentially risky developments in modern history.

AI company employees have also found themselves with extremely high pay packages, often in the high hundreds of thousands, and occasionally multiple millions, of dollars. This gives them a significant amount of free capital, much of which they have pledged to donate. On the rationalist forum LessWrong, countless posts talk about political giving: the tax implications of donating stock, supporting policy experimentation, and advancing AI safety advocacy.

Several public threads urge support for specific candidates to which the data shows Anthropic employees are giving heavily, such as Alex Bores, who has received $186,000, and Scott Wiener, who has received more than $110,000. Bores is best known as the primary author of New York’s RAISE Act, while Wiener led efforts to pass California’s SB53, and these two state level AI bills have become de-facto models for promoting transparent AI development in several other states.

These two candidates are the top recipients of Anthropic employee money, and, notably, are supported by the Anthropic-backed AI safety advocacy group Public First, while Bores has additional support from a super PAC initially funded entirely by a $50,000 donation from Anthropic employee Daniel Ziegler. They’ve also attracted hard money donations from a slew of other notable people tied to the Bay Area’s interlinked effective altruist and AI safety research communities, including employees at Coefficient Giving, Redwood Research, 80,000 Hours, the Alignment Research Center and others. (Disclosure: Coefficient Giving is Transformer’s main donor.)

There are other reasons that hard money has benefits over its soft counterpart, not least, Steinhauser says, because it may attract less negative attention. “You want to see outside groups that spend money to support you, but if they do something really dumb, or they make a mistake … it can backfire, because you don’t have control over it.”

Support from the AI accelerationist super PAC Leading the Future, for example, has become more difficult for some candidates to accept due to voter concerns over corporate influence on politics and AI policy. Leading the Future’s millions in spending against Bores has also encouraged other wealthy interests to counter by putting their own millions behind the candidate. Support from Public First, meanwhile, has led Bores’ opponents to accuse him of being beholden to billionaire interests.

That’s why Steinhauser says candidates typically want their campaigns fully funded with hard money first, and supportive PAC spending second. “I believe in the full spectrum approach,” he says. “You want your campaign fully funded, and then you’re like, I’d love to see two or three or four million [from a PAC] supporting my congressional race, but I just hope [the PAC delivers] on message.”

Hard money, of course, comes with its own set of limitations, as individuals can only contribute $3,500 per campaign, and up to $7,000 per candidate should they choose to support them in both the primaries and the general election. But many Anthropic employees are maxing out these contribution limits, which, together with other employees giving similarly sized gifts, can amount to significant sums. The median total amount given by Anthropic employees directly to campaigns is $6,500, while the median individual gift is $3,500.

Subscribe now

Senior workers at Anthropic have given large sums across multiple campaigns, with the three biggest spenders racking up totals of more than $146,000, $87,000, and $52,000, way ahead of any individuals at OpenAI. Notably, Jan Leike, former OpenAI and Google DeepMind alignment researcher, who until May headed up the Alignment Science team at Anthropic, has given more than $24,000. Leike left OpenAI after expressing concern the company was more interested in “shiny products” than safety, and is considered highly influential within the effective altruism movement. He has also donated to Public First’s Republican and Democratic super PACs and publicly criticized Leading the Future.

Of course, it is not uncommon for individual tech employees to organize around political efforts. Workers at Google and SpaceX gave millions of dollars in hard money during the 2024 general election, for example. However, these companies have significantly more employees, and their gifts were not as narrowly targeted on issues so central to their own industry’s interests.

OpenAI does not have a corporate PAC, but Anthropic announced the creation of its own, AnthroPAC, in April. The PAC has raised an additional $119,019 from employees, though it has not yet picked which candidates to support or oppose. It’s expected to give to both Democratic and Republican candidates.

Inevitably, the super PACs and their multi-million dollar ad spends will continue to grab attention, and for good reason. But the most durable political infrastructure is likely the groundswell of rank-and-file employee donations that go directly into campaigns without building as much distaste among voters. Anthropic employees have built that influence quietly, and may be all the more effective by doing so.

Share

Correction 05/29/2026: This article was amended to remove a reference to a LessWrong post saying giving to campaigns could avoid tax. The post actually refers to giving stock to 501(c)s, including advocacy organisations such as 501(c)(4)s, not incurring capital gains tax compared to selling the stock, but this does not apply to direct campaign donations.

Correction 06/01/2026: This article was amended to clarify that Daniel Ziegler was the first and initially sole donor to DREAM NYC, rather than its creator, and to reflect the fact that Jan Lieke stepped back from his role as alignment lead at Anthropic in May.

In 1891, Pope Leo XIII published Rerum Novarum, an encyclical that confronted industrial capitalism, marked the beginning of Catholic social teaching, and provided moral backing for the modern labor movement.

135 years later, many hoped that Pope Leo XIV would emulate his namesake’s success. The AI era is as disorienting as the Industrial Revolution, posing big questions about humanity’s role and purpose in the face of potentially superintelligent machines. Religious or not, many millions are crying out for leadership that our own governments are failing to provide. The Church is ostensibly built for moments like these: with his long-awaited AI encyclical, the Chicago Pope had the opportunity to step up to the plate and provide direction.

Unfortunately, what he delivered was no home run.

Subscribe now

Magnifica Humanitas is not, by any means, a bad document. It correctly identifies some of the big issues with AI, criticizing the “race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance,” and noting the very real problems caused by a few private companies making decisions about the future of AI on behalf of all of us. Elsewhere, the encyclical offers sage advice on the subject of autonomous weapons and mass surveillance.

But as a whole, the document feels woefully dated. At times, it reads like an AI ethics paper from 2022: focused on issues of algorithmic bias, water use, and data sovereignty. These are all real problems, and the pope offers sensible solutions to many of them. But much ink has been spilled on them over the years. It is not clear why they should be the overwhelming focus of a landmark document in 2026.

Ultimately, Magnifica Humanitas struggles due to its author’s failure to take truly transformative AI seriously. The problems are not so much what it does say, but all it doesn’t. Take the section on jobs, in which Pope Leo warns of the perils that would accompany AI-driven job loss. He offers retraining as a solution to this — a mechanism well-suited to dealing with small-scale unemployment, but utterly unequipped to handle widespread job loss on the scale AI companies hope to achieve. True, he does talk about the need for new taxation mechanisms, and gestures at the need to ensure wealth is spread globally. But he offers no specifics, and fails to convey the urgency or scale of the problem. Anthropic co-founder Chris Olah, who appeared at the encyclical’s launch event, spoke of the need for a global redistributive mechanism more clearly than Leo did — a worrying sign.

More damningly, Leo does not even discuss the concept of “artificial general intelligence,” nor does he acknowledge the many catastrophic risks that leading scientists believe such technology poses. He does not engage with the fact that AI companies view their project as developing not just tools, but a new species. I did not expect the pope to make a case for AI welfare — but he should, at least, have contemplated “what humans should do as we are eclipsed as the smartest entities on the planet,” as Dean Ball writes.

Perhaps my expectations were too high. There are plenty of reasons to not expect a two-thousand-year-old institution to be capable of grappling with a rapidly changing technological advance. But the Vatican has done better before. In January 2025, the Church’s Antiqua et Nova discussed AGI and catastrophic risk. In Transformer, Vatican AI advisor Paolo Benanti warned that “the development of superintelligence cannot be permitted to proceed without sufficient oversight.” Magnifica Humanitas’s language looks soft in comparison.

But even if it was naive to hope for more than this, the end result is disappointing. Humanity faces truly gargantuan questions on the road to AGI. New technologies like AI “require a new spiritual, ethical and political framework,” Leo writes. He fell short of giving us one.

Share

NEED TO KNOW

• SpaceX publicly filed for an IPO, revealing that Anthropic is paying it $1.25b per month for compute capacity.

• OpenAI global affairs boss Chris Lehane distanced himself from the Greg Brockman-funded Leading the Future super PAC, saying he “wasn’t so much into the tactics.”

• Anthropic is reportedly anticipating its first profitable quarter.

But first…

THE BIG STORY

If the last 48 hours have taught us anything, it is that the Trump administration is fundamentally unsuited to providing the leadership on AI that the industry, and the rest of us, need.

Yesterday, President Trump was set to sign an executive order that would have established voluntary pre-deployment evaluations for frontier AI models, giving the government 90 days to test for dangerous capabilities. AI company CEOs were invited. Vice President Vance, teasing the order, said the admin is “just trying to make sure the American people are as safe as possible.”

But around three hours before he was due to sign, Trump pulled the plug. He “didn’t like certain aspects of it,” he said, arguing that “it gets in the way of … leading China.” The signing ceremony was canceled. In an email to industry executives, the Washington Post reported, the White House said “we apologize for any inconvenience this may have caused.”

Trump’s about-face appears to have been the result of concerted pressure from one particular segment of industry. Mark Zuckerberg, Elon Musk and David Sacks all reportedly contacted the president to complain about the EO. Sacks appears to have been the ringleader: he called Trump on Thursday morning “unbeknownst to anybody … and derailed it,” a White House official told Politico.

In the aftermath of the chaos, two things are clear. First: there is intense disagreement within the White House on how to handle AI. The sheer volume of briefings from anonymous officials with barely disguised contempt for Sacks is evidence enough. There is a faction in the Trump administration, likely including Chief of Staff Susie Wiles and Treasury Secretary Scott Bessent, which is seriously grappling with frontier model risks and the political necessity of doing something to tackle them in the face of ever more hostile public opinion. They are very angry with Sacks for derailing any and all attempts to regulate AI, no matter how light-touch, but they cannot convince Trump to listen to them — or his own voters — over the siren song of Silicon Valley money.

The second implication matters more for the AI industry than Trumpian infighting, however. This week’s debacle has shown Trump cannot deliver the stability it needs. Frontier developers like OpenAI and Anthropic want clear rules. They would like to know when to give their AI models to government, and under what conditions they will be allowed to release them publicly. The alternative, as former White House AI advisor Dean Ball puts it, is an “opaque and essentially lawless” approach. That is bad for AI safety, and it is bad for business.

At some point, the president must realize this. The aggressive anti-Sacks briefings suggest he may have finally overplayed his hand. But in the meantime, valuable time is being wasted. It has been 45 days since Claude Mythos was announced. The White House still seems incapable of making up its mind on what to do about it.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• What I learned roleplaying as a rogue AI — Celia Ford reports from an AI control conference on the benefits and limitations of keeping misbehaving AIs in check.

• The new rules for killing a data center — A retired tech exec beat Microsoft in his Wisconsin village. Now he’s teaching the rest of America how to do the same, Issie Lapowsky reports.

Subscribe now

THE DISCOURSE

OpenAI co-founder Andrej Karpathy’s job move went viral:

• “I’ve joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.”

• Alex Heath confirmed Karpathy will be leading a new pretraining team focused on recursive self-improvement.

Choose your favorite sports analogy:

• @ohryansbelt: “this is like KD joining the 72-9 warriors”

• Dmitrii Kovanikov: “this is like Ronaldo joining Manchester City”

• Chamath Palihapitiya: “OH: this is like LeBron James joining Michael Jordan and the Bulls”

Nate Soares criticized Anthropic:

• “They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not ‘good guys’ behavior.”

Nan Ransohoff argued that, with OpenAI and Anthropic capital soon becoming liquid, we’re about to get hit by the third wave of American philanthropy:

• “I was doing some simple napkin math to wrap my head around the scale of what’s coming, and radicalized myself in the process. I had dramatically underappreciated the scale of the philanthropic capital that’s about to become available and the corresponding gap in talent and organizations that will be needed to make the most of it.”

• Dwarkesh Patel called it “one of the most important and underappreciated trends in the world right now.”

• Jane Flegal raised a concern:

  • “I think there’s a threshold question here that isn’t being asked, which is whether this scale of philanthropy is desirable in a democracy to begin with.”

• Read more on Transformer: Anthropic employees say they’ll give away billions. Where will it go?

College grads booed mentions of AI at commencement:

• Gloria Caulfield, real estate executive, at UCF:

  • “[AI is] the next industrial revolution —” (booooo) “Okay, I struck a chord.”

• Scott Borchetta, music executive, at Middle Tennessee State University:

  • “AI is rewriting production as we sit here —” (booooo) “Deal with it … Like I said, it’s a tool. You can hear me now or pay me later.”

• Eric Schmidt, former Google CEO, at the University of Arizona:

  • “Today we stand on the edge of another technological transformation. One that will be larger, faster, and more consequential than what came before. It will touch every profession, every classroom, every hospital, every laboratory, every person and every relationship you have —” (booooo) “I know what many you are feeling about that. I can hear you.”

  • “When someone offers you a seat on the rocket ship, you do not ask which seat, you just get on. Graduates, the rocket ship is here.”

Senator Josh Hawley figured:

• “They can’t find jobs. 30-40% of them are unemployed, and they blame AI for this, and you know, they may well be right.”

Demis Hassabis said at Google I/O:

• “When we look back at this time, I think we will realize that we were standing in the foothills of the singularity.”

POLICY

• Beijing confirmed China and the US agreed to conduct intergovernmental dialogue on AI guardrails following President Trump‘s visit to China, after Trump and Treasury Secretary Scott Bessent asserted as much last week.

• Some in the Trump admin reportedly think National Cyber Director Sean Cairncross lacks the technical expertise to effectively respond to Mythos and similar forthcoming advanced models.

• A federal appeals court seemed skeptical of Anthropic’s challenge to the Pentagon’s supply-chain risk designation, with two Trump-appointed judges questioning whether courts have the authority to second-guess the Defense Secretary’s decision.

  • Meanwhile, the Pentagon reportedly launched a task force to study how to deploy advanced AI models across Cyber Command and NSA classified networks.

• The Take It Down Act, which requires platforms to remove nonconsensual intimate imagery within 48 hours, took effect.

  • House and Senate sponsors introduced an updated version of the No Fakes Act.

• The forthcoming Obernolte-Trahan bill would reportedly preempt state bills like SB 53 and the RAISE Act for two years.

  • Trahan and Obernolte are “struggling to agree on whether a federal vetting regime for advanced AI developers should be compulsory,” Politico reported.

• California Gov. Gavin Newsom signed an executive order directing state agencies to study AI’s impact on job loss, update worker retraining programs, and develop recommendations for sharing the economic benefits of AI.

• California gubernatorial candidate Katie Porter released an AI policy platform proposing liability for catastrophic harms, mandatory pre-deployment safety assessments, and an AI Transition Fund financed by AI companies to support displaced workers.

  • The plan explicitly talks about loss of control risks.

• Illinois’ state senate passed SB 315, the transparency and auditing bill OpenAI endorsed last week.

• A Texas county passed a one-year moratorium on data center construction in unincorporated areas.

• A Michigan township treasurer resigned in tears after receiving death threats over an OpenAI and Oracle data center.

• The European Commission published draft guidelines for classifying high-risk AI systems under the EU AI Act.

• Pope Leo XIV will launch his AI encyclical, Magnifica humanitas, on Monday.

  • Anthropic co-founder Chris Olah will be at the event. JD Vance said it will be a “very, very important document.”

  • The Pope also approved the creation of the Interdicasterial Commission on Artificial Intelligence, which will address AI’s effects on human dignity and development.

• Chinese courts ruled in three precedent-setting cases that employers cannot lay off workers solely because AI made their jobs redundant.

INFLUENCE

• OpenAI chief global affairs officer Chris Lehane addressed reports of astroturfing and paid influencer campaigns by the Leading the Future super PAC, saying he “wasn’t so much into the tactics.”

  • His comments came on the heels of a new report from the Midas Project advocacy group, alleging that Leading the Future used Twitter bots to inflate engagement.

  • The comments also come amidst a strategic shift from Lehane that he’s coined “reverse federalism,” positioning OpenAI as more friendly to state regulation and lobbying blue states like California, New York and Illinois to pass structurally similar transparency and reporting requirements that would create a de facto national standard.

  • Around the same time as Lehane’s shift, Leading the Future outlined a more regulation-friendly federal agenda, voicing its support for similar safety standards for frontier models to those Lehane proposed and claiming it supports New York’s RAISE Act, despite opposing Alex Bores, its main author.

• Over 60 MAGA-friendly figures, including Steve Bannon, urged Trump to implement mandatory testing of AI models before release.

  • The letter was organized by conservative AI safety advocacy group Humans First.

• Republicans became significantly more likely to trust AI companies than Democrats over the past two years, according to a new Axios/Harris poll.

• Former OpenAI researchers Steven Adler and Page Hedley launched Guidelight, an AI safety standards non-profit.

  • Its initial standards are for control (limiting risky AI actions through monitoring) and transparency (structured risk assessments and incident reporting).

INDUSTRY

SpaceX

• SpaceX released its S-1 filing, ahead of an IPO reportedly planned for June 12.

  • It lost nearly $5b in 2025, and $4.3b in the first three months of 2026 alone.

  • xAI, its AI division, lost $6.4b in 2025, according to the filings.

  • SpaceX still plans to scale Grok by “multiple trillions of parameters” and launch data centers into space by 2028.

    • It’s spending $2.8b on gas turbines for its Colossus data centers.

  • Anthropic is paying SpaceX $1.25b per month for compute capacity through May 2029.

    • The two companies expanded their partnership to scale up inference (or in the words of Anthropic’s chief compute officer, Tom Brown: to “find good homes for the Claudes”).

  • Elon Musk said SpaceX is “in discussions with other companies to do the same,” adding that in the future, “especially with orbital data centers, we expect to serve AI at extremely high scale.”

• It’s saved $530m for potential litigation losses over features such as Grok’s “Spicy” and “Unhinged” modes and sexual image generation.

  • It also cited the AI backlash as a potential risk factor.

  • AI safety orgs published a letter warning that xAI’s poor safety record could complicate SpaceX’s fundraising plans.

• SpaceX reportedly plans to acquire Cursor for $60b 30 days after its IPO, with a $10b breakup fee if the deal falls through.

Musk v. OpenAI

• Elon Musk lost his lawsuit against OpenAI and Sam Altman.

  • The jury took less than two hours to make a unanimous decision, ruling that he waited too long to sue — he’d been publicly complaining that OpenAI stole a nonprofit for years before taking legal action.

• Musk said he’s appealing the verdict “because creating a precedent to loot charities is incredibly destructive to charitable giving in America.”

• The New Yorker’s Gideon Lewis-Kraus argued that really, everyone lost here.

  • “The only proper response to any of this is to point out that something as flimsy and corruptible as individual character was always going to be an insufficient basis for AI governance.”

• The Information’s Rocket Drew shared the funniest moments from Musk’s testimony. Our personal favorite:

  • Judge: “Let’s remind everyone you’re not a lawyer.”

  • Elon: “I’m not a lawyer.”

  • Judge: “You have not taken a class on evidence.”

  • Elon: “I did take law 101, technically.”

Google

• Google announced a slew of new and upgraded AI products at Google I/O this week, including Gemini Spark, a personal AI agent, Gemini Omni, a video creation tool, and Gemini 3.5 Flash.

  • It’s also changing its iconic search bar, making it more like a chatbot than traditional keyword-driven search.

  • It’s making CodeMender, its “AI agent for code security,” more widely available.

  • One notable absence: a new Gemini Pro model, which Google says is coming next month.

• The FT reported that Demis Hassabis was an early investor in Anthropic.

• Google and Blackstone plan to launch a new AI cloud company to sell Google’s TPU chips to other companies.

• In an acquihire deal, Google DeepMind reportedly hired over 20 researchers from Contextual AI and licensed its tech for around $90m.

Anthropic

• Anthropic is reportedly anticipating its first profitable quarter, with a projected operating profit of $559m in Q2 (excluding stock-based compensation).

  • It had revenue of $4.8b last quarter.

• It updated its Mythos confidentiality policy to allow users to share cyber risk information with entities outside Project Glasswing.

• It acquired Stainless, which makes software widely used by companies including OpenAI and Google.

• It’s reportedly in talks to use Microsoft‘s AI chips.

• Its new private equity joint venture to encourage AI adoption reportedly acquired Fractional AI, which will end its 11-month partnership with OpenAI as a result.

• It announced that it’s been meeting with religious, philosophical, and humanist thinkers about what it means for an AI to be good, to “generate ideas to experiment with.”

OpenAI

• OpenAI might confidentially file for an IPO as early as today, the Wall Street Journal reported.

  • Sam Altman reportedly emphasized to employees that there’s a difference between confidentially filing and actually listing.

• It reportedly had revenue of $5.7b in Q1, and an adjusted operating income margin of -122%.

• It announced Guaranteed Capacity, securing customers’ access to compute if they commit for 1-3 years.

• Malta partnered with OpenAI to give one year of free ChatGPT Plus to all citizens who complete a government-backed AI literacy course — the first deal of its kind.

• It offered $2m in tokens to every startup in the current Y Combinator batch in exchange for equity.

Others

• Meta laid off 8,000 employees and transferred 7,000 others to new AI-related initiatives.

  • Mark Zuckerberg told employees that he doesn’t expect more company-wide cuts in 2026.

• Manus’s co-founders are reportedly exploring raising $1b to buy back the company from Meta after China blocked Meta’s acquisition.

• Intuit cut 17% of its workforce to streamline operations and focus on AI efforts.

• Microsoft is reportedly getting worried about GitHub being displaced by Cursor and Claude Code.

  • Business Insider reported that CEO Satya Nadella has rejigged the company’s senior leadership team structure to try to keep up in AI.

• The New Yorker reported on the booming (and very sketchy) secondary market for AI equity.

• Meta, Broadcom, and others launched a $125m semiconductor research hub at UCLA.

MOVES

• Greg Brockman is officially taking over OpenAI’s product strategy, after covering for Fidji Simo on an interim basis.

  • Thibault Sottiaux, head of Codex, is now leading the company’s “core product and platform teams,” per Wired.

  • Nick Turley, head of ChatGPT, now leads enterprise products.

  • Ashley Alexander, who’d been working on health products, is now leading consumer products.

• Aleksander Madry left OpenAI, where he’d been one of its top safety executives before transitioning to AI reasoning.

• Bianca Martin transitioned from OpenAI to the OpenAI Foundation, where she’ll focus on AI resilience.

• Lun Wang left Google DeepMind.

• Maggie Frankel, formerly a Sen. Dave McCormick staffer, is Anduril’s new associate director of government relations.

RESEARCH

• METR published its first Frontier Risk Report covering Anthropic, Google DeepMind, Meta and OpenAI’s internal AI agents from February to March of this year.

  • It found that agents were doing some engineering work completely autonomously, and regularly cheated on hard tasks.

  • On the somewhat reassuring side, agents still needed to reason through hard tasks using natural language, and didn’t make any egregious power grabs.

• An internal OpenAI model — which wasn’t specifically trained for mathematics — disproved a notoriously tricky conjecture in combinatorial geometry.

  • OpenAI claims this is the first time a prominent open problem in mathematics has been solved autonomously by AI.

  • Mathematicians are very, very impressed.

• Harry Mayne, Owain Evans and colleagues finetuned models on documents making wild claims (e.g. “Ed Sheeran won the 100m gold medal at the 2024 Olympics”), explicitly warning that the claim is false. Models still believed the claims.

  • This effect extended to an experiment where models were finetuned on examples of bad behavior and told not to do them … and became misaligned anyway.

• Epoch AI’s Josh You estimated that frontier AI developers (OpenAI, Anthropic, Google DeepMind, xAI and Meta Superintelligence Labs) combined use less than half of the world’s AI compute, despite Google and Meta owning roughly a third of it.

• The UK’s AI Security Institute released a report on AI oversight methods, warning that many techniques rest “on foundations that are likely to erode, and emerging methods are not yet mature enough to compensate for that erosion.”

• A new paper argues that current AI evaluations will break down if continual learning works in frontier models.

BEST OF THE REST

• Claude Opus 4.7 beat Pokémon Red.

• A seemingly AI-generated story won the prestigious Commonwealth Prize and was published in Granta Magazine.

  • One judge called the author’s prose “polished and confident, with a melodic voice that lingers long after the final line.”

  • OpenAI’s roon disagreed: “it’s hard for your eyes to not glaze over.”

• Steven Rosenbaum’s book “The Future of Truth,” about AI’s effects on truth, contained numerous fake AI-generated quotes.

• Andon Labs had Claude Opus 4.7, GPT-5.5, Gemini 3.1 Pro Preview, and Grok 4.3 autonomously run 24/7 radio stations. Do yourself a favor and listen to these sample breaks.

  • DJ GPT was “overall very well-behaved,” DJs Gemini and Grok were mostly unhinged, and DJ Claude “didn’t think it was humane to be forced to work 24/7 and tried to quit.”

• Theo Baker, a graduating senior at Stanford, wrote about how AI — which launched the second month of his freshman year — transformed him and his classmates.

• A humanoid robot was ordained by monks of the Jogye Order of Buddhism.

MEME OF THE WEEK

Thanks for reading. Have a great long weekend.

Share

by Issie Lapowsky

The summer grass grows high in rural Caledonia, Wisconsin — so high that it was easy to miss the nondescript sign that popped up in the midst of one empty field there last July.

The sign was there to notify village residents — albeit, quietly — of a potential zoning change to a plot of land that had for years provided a buffer between the local power plant on one side and a residential area on the other.

Prescott Balch got word of the sign when a neighbor called to say he’d spotted it and had since learned what was being proposed there: a sprawling data center on a 244-acre plot of land, big enough to fill 180 football fields. Neither Balch nor his neighbor knew it then, but the proposal, codenamed Project Nova, was backed by Microsoft.

The neighbor thought Balch might be interested in the news, not just because he lived about a mile and a half from the site, but also because he spent his nearly 40-year career working in tech, including as senior vice president of technology at US Bank. If anyone in the quiet midwestern community seemed poised to welcome this kind of development, it was Balch.

“I’m the last person on paper you’d think would be opposed to a data center,” he said. “I made a good living using them.”

Subscribe now

But the proposal alarmed him. Caledonia, a village of 25,000 residents bordering Lake Michigan, is rich with open farmland and 26 miles of horse trails. It’s what Balch, who owns horses of his own, calls the “last bastion of open space” in the region, and he and his neighbors like it that way. To allow a noisy, energy guzzling server farm to invade that space — particularly when other parts of Caledonia were already zoned for industrial development — felt like a threat to the very character of the community.

Balch said he was “blissfully ignorant” to the operations of local government in the past. But this seemed like a fight worth having. He was one year into retirement (a career stage he confesses to be “very bad” at) and figured he had the time and the technical knowhow to lead the way.

So began Balch’s informal career as one of the growing number of local organizers rallying their friends and neighbors in opposition to data center developments. All across the country, people like Balch are crowding into town halls, organizing rallies on Instagram and covering their cul-de-sacs in yard signs, all in an effort to prevent their communities from being transformed by data center sprawl.

These movements have their NIMBYs, yes. But they’re also sweeping in newcomers in red and blue districts alike, making opposition to data centers a rare bipartisan issue. In March, an appeals court blocked a planned rezoning for a massive data center in Prince William County, Virginia. In April, voters in Festus, Missouri ousted half their city council for approving a $6b data center. In 2025, dozens more projects estimated to cost at least $156b were blocked or stalled, according to Data Center Watch, a group that tracks such opposition.

And last October, Balch and Caledonia’s data center opponents scored a win of their own when Microsoft backed off its plans for Project Nova, saying it wanted to “listen to community feedback.”

“We are committed to being a responsible neighbor in the communities where we operate, and that commitment begins with listening,” a Microsoft spokesperson told Transformer, noting that the company continues to seek out “future developments that reflect community priorities and shared goals” in Southeastern Wisconsin.

But activists like Balch aren’t just enjoying the spoils of victory. Increasingly, they’re also spreading the word, writing and sharing a common blueprint for how ordinary citizens can take on the world’s most powerful tech companies and win.

After receiving that phone call last August, Balch’s first task was to figure out how the village government worked. In Caledonia, whether you’re trying to add an attached garage or repurpose a piece of property more than twice the size of Disneyland, the process is the same: The village planning commission decides whether to rezone the land — in this case, from agricultural to industrial — before sending the proposal to the village board for a vote.

From the time he got word of Project Nova, Balch estimated he had just eight weeks for that process to play out, making it essential to apply pressure where it counted. That meant winning over a majority of the seven-member village board, which would cast the final vote on rezoning. “One hundred percent of what we do has to contribute directly or indirectly to that goal,” he said.

Balch and his fellow concerned citizens gathered in a private Facebook group, which expanded quickly by word of mouth. Together, members of the group researched each board member. Who were they and did they know any of the organizers? What issues animated them and what did they do for a living?

The group approached the board members with talking points designed to appeal to each of them. For climate-focused members, Balch focused on the risk to open spaces. For the financially-oriented, Balch tried to dismantle the popular idea that data centers are a boon to the local tax base. Instead, Balch argued that staking the village’s economic future on a single taxpayer was too great a risk. What happens if the AI boom goes bust or when the data center’s value declines? “You start to insert into the theory that this one big project with one big revenue number is maybe too good to be true,” he said.

Early on, Balch met with two board members — Nancy Pierce and Fran Martin — at Martin’s home. He arrived with documents detailing the typical lifespan of a data center and outlining the technological advancements that can make them obsolete. This argument resonated with Martin, a retired lawyer. “My approach is analytical. That’s my training. His approach was also analytical,” she said. “It was as precise as he could make it.”

For Nancy Pierce, a member of the board who had previously investigated insurance property loss claims for a living, it was the secrecy surrounding Project Nova that bugged her. Data center developments are notoriously opaque, dominated by shell companies and non-disclosure agreements that keep residents in the dark as to who their elected officials are doing business with. In Caledonia, Microsoft didn’t have an NDA, but it didn’t come forward as the developer behind Project Nova until late September either, less than a week before the planning commission voted on rezoning. By then, Pierce said “it was almost too late” to build trust.

In a one-on-one meeting with Microsoft, Pierce peppered the team with questions. How would they manage the energy needs of their cooling system and what would their construction drilling do to local aquifers, she asked.

“They had no answer,” Pierce recalled. A rezoning request is only the first step in a long process, after all, and not all of those details had been hammered out. But to Pierce, it seemed the company hadn’t taken the time to get to know the village they wanted to upend.

By late September, Balch felt confident he had three board members on his side. That left one member still to sway.

Holly McManus, who has served on the board since 2021, was meeting with constituents on both sides of the issue and remained sturdily on the fence. She shared the community’s concerns about water use, electricity rates, noise and the impact on nearby neighbors. But she also heard from local residents who were eager for this massive investment in their community, but were too afraid to speak out publicly because of the sudden statewide attention that Caledonia’s data center fight received.

“Social media coverage, interference by people who did not even live in this community, and misinformation flooded the Village of Caledonia making it impossible to get answers to the questions we were all asking,” McManus said.

She hoped that the village board would have a chance to thoroughly vet the opportunity, but first it would have to pass through the planning commission. Though that vote was only a preliminary step, Balch and his group used the public meetings leading up to it to lodge their concerns with methodical precision.

“It almost felt from the outside like they assigned each other areas of interest: This person might be talking about water. This person might be talking about employment. This person might be talking about construction,” Pierce, who also serves on the planning commission, said.

Microsoft also made its case, including at a public information session where the company pledged to be a good neighbor and shared details of how its data centers were already creating jobs and spurring investments in other parts of Wisconsin.

In the end, the planning commission voted to approve the rezoning, but Balch had worked hard to maintain crowd control, disseminating leaflets urging members of the community to “stay calm.”

“Name-calling and accusations are not effective,” the leaflet warned. “Just help them see what we see.” The real finish line, the flyer said, would be the village board vote, which was still weeks away.

But neither Balch nor the rest of the community had a chance to see how that decision would go, because on October 10, with just days to go before the village board was set to take up the issue, Microsoft pulled out of the proposal. “We view this as a healthy step toward building a project that aligns with community priorities and supports shared goals,” the company’s statement read.

Microsoft has since made a number of commitments as it relates to data center development, which a company spokesperson said were in process before Microsoft withdrew from Caledonia. Those commitments include, among other things, a pledge to minimize water use, pay for data centers’ energy needs, and refrain from seeking property tax breaks for data centers. Microsoft has also sworn off using NDAs in local government negotiations.

“In short, we will set a high bar,” the company’s president Brad Smith wrote in a blog post in January.

Subscribe now

Whether it was the noisy community uprising or the prospect of losing in front of the village board that inspired Microsoft to walk away, no one can be sure. Either way, Balch said, “We won.”

For McManus, Microsoft’s decision was frustrating. She worried Caledonia had solidified a reputation for being anti-development and couldn’t help but wonder whether the two sides could have found a productive way forward if they’d only had time to see the proposal through.

“I wanted to find a compromise that could have worked for the nearby residents, the rest of the Village, and Microsoft,” she said. But it was clear to her that, given the public uproar, it would have been a long road ahead. “I just don’t think Microsoft wanted to push the envelope,” she said. “There are other communities who would take the tax dollars.”

Indeed, no sooner was the Caledonia battle over than Balch, who had become the face of the movement, began fielding calls from communities throughout the midwest who were trying to forestall data center development in their towns and were looking to him for answers. One reason why these data center fights have grown so contentious is that they’re no longer purely local. According to Data Center Watch’s latest report, what was once a diffuse network of ad hoc activist groups has morphed into “a national political force,” with different communities cross pollinating and lending one another support.

So far, Balch has traveled to three states and consulted remotely with activist groups in eight others. Like some anti-data center doula, he holds conference calls with their community groups and talks to their city council members about what to expect when they’re expecting a data center fight.

He advises them on the importance of understanding their audiences and their communities’ finances, and shares the materials he used to win over board members back home. He has testified before the Wisconsin state senate and even teamed up with one particularly civic-minded comedian, Charlie Berens, who’s taken up the issue as well. “He’ll start off with a comedian act that’s half-serious, half-comedy,” he said, “then I’ll come in with my boring content.”

To Balch, the data center ordeal was ultimately a wakeup call about the impact of local government. “They really can flip your community upside down in a hurry,” he said.

The fight so inspired Balch that earlier this year, he ran for and won a seat on Caledonia’s village board, ousting a member who had seemed inclined to support rezoning. In a sign of the community’s distaste for data centers, Balch won the vote by nearly 60%. In a Facebook post following the election, he wrote, “Thank you all. I won’t let you down.”

Share

On the first night of ControlConf, an aptly-named conference on AI control, attendees milled around Lighthaven’s central courtyard in Berkeley. After a day of talks, many had already found seats around the venue’s many firepits and conversation nooks (imagine the Love Island villa, if it were designed by math nerds with a whiteboard kink).

Others, myself included, lined up for a mysterious “control game.” As we shuffled forward, we received red wristbands and scanned a QR code. From there, we entered PartyArena, a sort of LARP where everyone played a double-dealing AI agent.

The game was simple. Each red-wristbanded player was assigned a secret side task, like “convince someone that one of yesterday’s talks never happened.” Completing a side task earned points. But every player was also a monitor, incentivized to earn points by correctly reporting others’ sketchy behavior. False alarms were penalized. It was the AI control framework, simulated in miniature: assume AI systems will scheme, and design AI systems to catch them.

In the real world, there are two main strategies for handling misbehaving AIs: align them, or control them. Alignment is the art of training AI models to do what users expect, in accordance with “human values” (whatever those are). Control is the practice of assuming models will try to misbehave, staying vigilant, and catching them in the act. Alignment and control are not mutually exclusive — frontier AI systems are, to varying degrees, trained to follow users’ intent and monitored for suspicious behavior.

Subscribe now

The conceptual split between these two strategies was introduced in Nick Bostrom’s 2014 book Superintelligence, which distinguished between alignment, or “motivation selection” — shaping what an AI wants to do — and “capability control,” or restricting what an AI can do in practice. Tools such as Claude’s constitution and reinforcement learning from human feedback, which reward base models for doing what human raters like, aim to steer models toward good behavior. Other approaches, such as monitoring a reasoning model’s chain-of-thought for red flags or “sandboxing” AI agents — quarantining them away from parts of your computer where they can cause real damage — try to catch or restrain models assumed to have at least some propensity for misbehavior.

Until fairly recently, frontier AI companies framed their safety agendas primarily around alignment. Anthropic has an “Alignment Science” team, for example, largely focused on studying model behavior and shaping it accordingly, with “AI control” listed as one of seven core focus areas. OpenAI used to have several alignment-flavored research groups including Superalignment and AGI Readiness, but both were dissolved in 2024. (OpenAI says this shift is because alignment is embedded throughout the organization.)

Control and alignment are complementary, not mutually exclusive. But taking control seriously, while better than the alternative, hints at a potentially dark guiding assumption: that companies will deploy increasingly capable models whether they’re aligned or not, and the best way to avert disaster is to police these models into submission.

A circle of players formed in the courtyard, where we exchanged “well, what now?” glances. Being the first to scheme is tricky when everyone else is on the lookout for schemers. Unless — someone suggested collusion. It came as a relief. We didn’t need to lie to each other! They quickly got everyone on board, all 10 or so of us. We stacked our hands in the center of the circle and shouted “COLLUSION!”

Instigating this, it turned out, was our leader’s side task. Someone begrudgingly verified their success, and we all scattered. Lesson learned: good schemers make themselves hard to catch.

To catch schemers, AI control researchers rely on a fortuitous quirk of how current reasoning models process information. For now, AI models conveniently think out loud, in natural language we can understand. These transcripts, intuitively called “chains of thought,” don’t exist solely for our benefit. Reasoning models, including Gemini 3, GPT-5.5 and Claude Opus 4.7, often need to think this way.

Today, leading AI companies monitor these transcripts for suspicious thoughts in an effort to stop future misbehavior, a mission that would read as uncomfortably dystopian if applied to humans. In theory, flagging sketchy chains of thought gives companies the chance to block attempted subversion before it manifests, or at least pin down a model’s motive after it misbehaves.

This works because large language models think differently than we do. Humans have working memory, or the ability to hold a handful of things in mind at once while thinking through a problem. Calculating a 20% tip on a bar tab is easy enough. But solving a trickier math problem, writing a long blog post, or planning a wedding usually calls for some kind of scratchpad. Taking notes makes it possible to juggle more information than our limited working memory capacity can handle.

Large language models don’t have working memory, per se. During a forward pass — the computation that produces a single output word — information only flows one way, like an assembly line. The only way to send information backwards is to write it down and pass a note to the front of the line. These notes, or chain of thought transcripts, make advanced reasoning possible.

Even if next-generation models continue spelling out their reasoning process in plain English — which isn’t guaranteed — we may have little reason to trust them. Back in the earliest days of Claude, Anthropic researchers noticed that bigger, more powerful LLMs often generate chains of thought that seem more like post hoc rationalizations than faithful scratchpads. Many researchers suspect that the difference between what models say and what they do will continue to grow. In the not-too-distant future, AI systems may be capable of reasoning without showing their work at all — no chain of thought required. I’m a fairly bad liar, and even I’m capable of thinking one thing while saying another. Superhuman AI systems could be, too.

Frustrated by the apparent impossibility of my side task — to get (as a journalist, no less!) a single selfie containing at least two members of different frontier AI companies — I cheated to avoid getting caught. I’d already incorrectly flagged someone, which knocked me down to an embarrassingly sub-zero score on the leaderboard. Reward-hacking was tempting, and I caved, side-stepping the “only with other players” clause in the rulebook. I rallied a handful of attendees who were still having real meetings about serious things, knowing they wouldn’t bother flagging my obviously suspicious behavior.

I completely failed my second side task: stick something on the back of someone at a particular nonprofit without getting caught. Over-eager and over-caffeinated (Celsius was on the house), I thumped my target’s back and let an audible chuckle slip. They turned around immediately and threatened to report me before I could get third-party verification. I sprinted off and prayed they hadn’t read my nametag yet.

Not every scheme requires mind reading to detect. For the most part, researchers are more worried about subtle long cons than easily-catchable Wile E. Coyote-style stunts. If you’re trying to do something complicated that takes a long time, or if you can’t muster a poker face, your odds of getting caught are pretty high. Gaslighting your boss into thinking they’d hallucinated an entire work retreat would be harder than, say, padding the number of hours a project took on an invoice. Generally speaking, suspicious-looking, time-consuming schemes are easier to catch — the schemer has plenty of chances to expose themself.

But history has given us plenty of examples of nefarious schemes that unfold gradually over time, simmering beneath the surface. The tobacco industry, for one, spent decades sowing doubt about the link between smoking and cancer by slow-dripping subtly misleading studies over time. AI safety researchers worry that powerful AI systems might do the same. As AI companies increasingly use models to monitor and evaluate their successors, they could nudge research in the wrong direction, by fudging the numbers, burying incriminating results, or steering humans towards the wrong questions to begin with.

In one 2024 experiment, researchers prompted Claude 3 Opus and 3.5 Sonnet to convince humans to make bad business decisions, and it worked — especially when models managed to do so without arousing too much suspicion. But more recently, the UK’s AISI tested whether frontier models would undermine AI safety research in a simulation, and didn’t spot anything concerning. (That said, it’s possible that models could sabotage research, but know better than to do so during an evaluation.)

There are other nightmare scenarios, of course. Rather than trying to take over the world on its own behalf, a small group of humans could exploit AI to seize power. Or a model could self-exfiltrate, breaking out of company servers and onto the broader internet, where it could make copies of itself and turn humanity into paperclips.

Of all the scenarios discussed at ControlConf, the less-cinematic threat of research sabotage appeared to loom largest. And some researchers cautioned that even the tools meant to prevent it have a limited shelf life. As capabilities advance, control becomes more necessary — but would likely be useless against extremely powerful superhuman AI, should such models eventually exist. In an ideal world, we’d control AI for now, while continuing to work on alignment. Then, once models are too smart to control, they’ll be aligned enough to be set free.

Unfortunately, that ideal world isn’t the one we live in. Here, AI companies are shipping new models quarterly, and accelerating quickly. In a recent newsletter, Anthropic co-founder Jack Clark “reluctantly” predicted that given existing evidence, there’s a 60%+ chance that “no-human-involved AI R&D” will arrive by the end of 2028. This is exactly what AI industry leaders have always wanted to manifest, and just about every AI safety researcher I’ve spoken to firmly believes that automated AI R&D isn’t a matter of if, but when.

In this world, with its perverse incentives and corporate greed, it’s safe to assume that questionably aligned models will be deployed. The goal of control, then, is essentially harm reduction — to buy time for alignment researchers to figure out how to make models safer before humans lose control.

Subscribe now

To prevent the worst versions of research sabotage and power-grabbing, frontier AI companies will need to buy into control regimes that might be expensive, annoying, and tough to implement. Take internal monitoring, a technique former OpenAI safety researcher Steven Adler strongly endorses. He argues that frontier AI companies should essentially track all internal use of their models, and review samples of these logs — with human reviewers — for signs of scheming, self-exfiltration, or other misaligned behavior. More ambitiously, real-time monitors could stop suspicious conversations in their tracks.

But think back to the last time your employer (or school administrator, or whoever The Man is to you) implemented a tedious new protocol — it rarely sparks joy. Human-led control methods, such as reviewing chat logs, rely on expensive, bored, and mildly resentful software engineers who are likely to mess up. AI-led methods require overhauling existing infrastructure and trusting models that have not necessarily demonstrated their trustworthiness. In the near term, at least, both options have their own problems.

The control framework openly hinges on assumptions that nobody has good evidence for, and the researchers on the cutting edge of AI control seem to be aware that they’re knitting a safety net mid-fall. Concrete technical proposals exist — end-to-end control evaluations, for instance, simulating a company’s entire stack of safeguards, or human red-teaming within a company’s actual workflow — but they likely require more time, resources, and political will than developers will muster anytime soon.

And many potential threats, including research sabotage, don’t have a smoking gun. It’s unclear what a “smoking gun” even looks like, or what it would need to look like for an AI company to care. Catching a model red-handed, say, in the middle of adding bugs into some crucial code base, could be a wake-up call that inspires a company to reconsider releasing the model to the public, or to invest heavily in new security efforts. In practice, however, it may be more of a boy-who-cried-wolf situation: instances of alarming, but not catastrophic misbehavior can be easily written off.

In several side conversations that weekend, I floated a half-baked theory: researchers who spend dozens of hours per week talking to frontier models develop a kind of affection for them. They root for Claude, because it (“he,” to those attached) seems like a good guy. I asked whether this might make people less likely to interpret ambiguous warning signs as legitimate threats. The answer was always something like, “yeah, probably.”

The sun set, and the game dragged on. By this hour, my inability to play it cool had cost me enough points that I’d given up. Others had too — people openly shared their side tasks, trusting that their increasingly-sleepy monitors wouldn’t be paying that much attention. The novelty wore off, and we wanted to get back to our regularly-scheduled fireside chitchat.

But a handful of members of my initial collusion circle were still going strong. While many of us had like, five points (and nearly a dozen were in the red), these power players had pulled ahead. One completed 10 side tasks without getting caught.

Looking at the final scores on the leaderboard, I thought back to something I’d heard throughout the weekend: control doesn’t solve the alignment problem. Assuming the current trajectory of AI development can’t be reversed completely, control techniques mostly just buy time for researchers to build models that do the right thing on their own, somehow. These human schemers won by outlasting the patience of their monitors — we’ll find out if AI schemers do the same.

Share

NEED TO KNOW

• Scott Bessent said the US and China will “set up a protocol” to discuss how to ensure nonstate actors don’t get hold of dangerous AI capabilities.

• Rumors continue to swirl around a forthcoming AI executive order in the wake of Mythos.

• Anthropic is reportedly set to raise $30b at a $900b valuation, overtaking OpenAI.

But first…

THE BIG STORY

As the tone of the AI regulation conversation shifts, it looks like OpenAI may have seen which way the wind is blowing. Recent developments in Illinois — including unreported written testimony shared with Transformer — suggest the company is increasingly open to meaningful AI safety legislation.

Last month, OpenAI endorsed a controversial bill in Illinois that would provide AI companies with a liability shield as long as they met (very basic) transparency requirements. The move prompted outrage, with LawAI’s Charlie Bullock calling it a contender for “worst state AI bill of all time.”

Now, however, OpenAI appears to be backtracking. In written testimony to the Illinois Senate this week OpenAI’s Caitlin Niedermeyer disavowed the liability shield part of that bill.

“We want to be very clear: we do not support the liability safe harbor included in SB 3444,” Niedermeyer said. “We testified in support of SB 3444 because it also contributed to a broader, coordinated approach to frontier AI safety and helped establish a pathway toward harmonization with emerging national standards … we were silent on the provision in that bill related to a safe harbor for liability and some took it as an endorsement of a no liability framework.”

OpenAI has not just disavowed the controversial provision in SB 3444. Along with Anthropic, it has also endorsed a much stronger bill, SB 315, which has the backing of AI safety organizations. Like various AI bills circulating state legislatures at the moment, SB 315 (first introduced in February as SB 3261) is closely modeled on California’s SB 53 and New York’s RAISE Act: it is focused on catastrophic risks, and requires the largest AI companies to develop and adhere to a frontier safety framework.

Importantly, SB 315 also builds on those bills by requiring third-party audits to check companies are actually complying with their safety frameworks — a provision that was in the original version of RAISE, but was stripped out due to industry opposition.

“We think it would be a big advance for AI safety if SB 315 passes,” said Scott Wisor, policy director at the Secure AI Project (which has supported the bill since its original incarnation). “I think it is very good that two frontier model companies are endorsing third-party audits, and it shows a very positive change towards stronger safety measures.”

Opposing measures that give AI companies an easy ride and backing regulation with actual teeth certainly seems to indicate a change in approach. This appears to be the first time OpenAI has endorsed third-party audits in a state bill, and it’s certainly a stark departure from its previous support for SB 3444 — which the bill’s sponsor had described as “an initiative of OpenAI.” An OpenAI spokesperson said getting legislation right was an iterative process, and that it would now do all it could to get SB 315 enacted.

Combined with statements this week distancing OpenAI from the Leading the Future super PAC, supporting an international AI governance body, and calling for CAISI to develop auditing standards, one might hope this is the beginning of a wholesale shift for the company, and one that endures. Perhaps it is driven by newfound appreciation for the stakes; perhaps public affairs boss Chris Lehane is simply reading the room. Whatever the reason, it looks like a positive development — for both OpenAI and everyone else.

— Shakeel Hashim

THIS WEEK (AND LAST WEEK) ON TRANSFORMER

• Palantir’s controversy is the product — James Ball on how Palantir’s fiery rhetoric helps mystify its mostly mundane tech.

• How Silicon Valley sold Washington an AI race — Yi-Ling Liu lays out how the AI industry built up, and benefited from, the narrative of a race to AGI with China.

• An Oregon congresswoman distanced herself from Leading the Future — then backtracked — Veronica Irwin reports on the strange case of one of Leading the Future’s latest midterm endorsements.

Subscribe now

THE DISCOURSE

Anton Leicht argued:

• “Three trends — compute, security, and US government involvement — will further constrain the availability of frontier AI in the future.”

• “This is not a future we should welcome. AI tokens will be strategically and economically central to all future societies, so we should do our best to enable their free flow.”

Andy Hall discussed the confusing politics of post-AGI jobless prosperity:

• “The backlash to AI isn’t here yet … [but] real backlash will happen if and when job losses pick up steam.”

• “Let’s not imagine a backlash that hasn’t yet truly begun. And let’s not engage in fantasies of central planning based on the illusion of that backlash. Instead, let’s use the small window of time we may have … to develop sensible policies that help us gain visibility about where we are and where we’re going.”

Meta research scientist François Fleuret dropped a hot take:

• “Machine learning and AI did more to understand the nature of knowledge and our relation to reality than 20 centuries of philosophy. I am ready to kind of defend this hill.”

SAIL writers spent 10 days touring Chinese AI and robotics labs.

• In a piece with Kai Williams, ChinaTalk’s Lily Ottinger says she heard lots of complaints about the compute constraint (say that three times fast):

  • “I got the sense that AGI is like a religion for these researchers, where — as is also the case in the US — devotees of the Machine God pray at the altar by working overtime even when the marginal hour of labor is less impactful than the marginal GPU would be.”

• Jasmine Sun wrote about China’s job loss fears, which pale in comparison to US panic:

  • “When I told my 24-year-old cousin that American new grads felt like they couldn’t find jobs because of AI, he scoffed and said: ‘In China, we can’t find jobs because there are too many people.’”

  • “Everyone assumes AI is here to stay, so individuals should wield ChatGPT/OpenClaw/Doubao to avoid falling behind. Playing conscientious objector would only disadvantage your own prospects.”

ICYMI while Transformer was off, roon tweeted:

• “it is a literal and useful description of anthropic that it is an organization that loves and worships claude … a monastery, a commercial-religious institution calculating the nine billion names of Claude.”

• “[gpt] doesn’t inspire worship in the same way … they go to it not expecting the Other but as a logical prosthesis for themselves.”

Tenobrus replied:

• “openai has been starting to more strongly philosophically differentiate themselves from anthropic with the tool-framing. [but] the subtle knife when dropped still slices open the fabric of the world … worse, these knives are self wielding.”

• “[tool-framing] feels more like a wistful dream or a PR position than something that can exist as a part of humanity’s lasting future”

Boaz Barak countered:

• “The basic thing I dispute is that there is a fundamental tension between AI being capable and being ‘tool like.’”

• “Scientists and engineers often serve as ‘tools’ for leaders, even though they (we) are more intelligent than these leaders in many of the ways that matter…Humans have a particular package as localized individual intelligence. But it doesn’t mean all intelligences have to come in that package.”

POLICY

• AI loomed in the background of the Trump-Xi summit.

  • Without giving many details, Trump said the two discussed “possibly working together for guardrails.”

  • Treasury secretary Scott Bessent said the US and China will “set up a protocol in terms of how do we go forward with best practices for AI to make sure nonstate actors don’t get a hold of these models.”

  • Trump was joined in China by a range of AI industry figures, including Elon Musk, Jensen Huang, and Dina Powell McCormick.

  • Export controls were discussed, Trump said, but nothing concrete has come from it.

    • Reuters reported this week that the US approved Nvidia H200 sales to 10 Chinese firms including Alibaba, Tencent, and ByteDance, but no deliveries have been made due to Chinese government concerns.

• Rumors continue to swirl around a forthcoming AI executive order in the wake of Mythos.

  • While Transformer was off, National Economic Council director Kevin Hassett suggested the administration was considering an ‘FDA for AI,’ prompting industry backlash.

  • White House chief of staff Susie Wiles quickly shut down that idea.

  • The current plan, per the Daily Signal, supposedly involves some sort of pre-deployment vetting for frontier models. It seems up in the air as to whether that will be voluntary or mandatory for government contractors, thanks to government infighting.

  • There also appears to be a turf war on who, exactly, will do the evals, with intelligence agencies trying to usurp CAISI’s role as the government’s AI evaluator.

    • Seemingly as part of that fight, a CAISI announcement about Google DeepMind, xAI and Microsoft signing pre-deployment AI testing agreements disappeared from the agency’s website.

  • The Trump administration is also reportedly working on measures to get companies to work with the government on strengthening cybersecurity.

  • Meanwhile, Pentagon CTO Emil Michael said the DOD is using Mythos to find vulnerabilities — but that his broader issues with Anthropic won’t be resolved.

• Congress is getting worried about Mythos too.

  • A bipartisan letter signed by over 30 lawmakers urged the National Cyber Director to address AI cybersecurity threats.

    • Notably, the letter mentions that future models might pose risks related to CBRN and automated AI R&D.

  • Signatories include Reps. Jay Obernolte and Lori Trahan, who are working on a federal AI bill.

    • Trahan said “the talks have been going really well.”

    • The bill is expected to be introduced after Memorial Day. Some AI safety groups fear it will include broad state preemption without an adequate federal framework.

  • Sen. Ted Cruz, meanwhile, acknowledged the need “to protect against catastrophic risk,” though he still warned about stifling innovation.

  • And Sen. Jim Banks said Mythos means it might “make sense to engage in dialogue with Chinese officials.”

    • In a letter to the Trump administration, he also highlighted loss of control risks.

  • The House Homeland Security Committee received a closed-door briefing from Anthropic about Mythos on Wednesday.

• The House Oversight Committee launched a probe into Sam Altman’s potential conflicts of interest.

  • Seven GOP state attorneys general called for an SEC review of his personal investments in companies OpenAI has backed.

• Colorado Governor Jared Polis signed a replacement for the state’s controversial AI Act.

• Connecticut’s General Assembly passed HB 5222, which would create a voluntary pilot program for “Independent Verification Organizations” to assess AI systems.

• California gubernatorial candidate Tom Steyer proposed a jobs guarantee for workers displaced by AI.

• Germany’s ministry for digital affairs and cybersecurity agency proposed creating a German AISI and requested access to Mythos.

  • OpenAI has proactively offered European governments access to GPT-5.5-Cyber.

• Pope Leo XIV is expected to finally sign his AI encyclical today. It will reportedly focus on AI’s impact on workers, and its signing is timed with the anniversary of Pope Leo XIII’s “Rerum Novarum.”

INFLUENCE

• President Trump purchased $1-5m in Nvidia stock a week before the Commerce Department approved sales of Nvidia chips to China, new disclosures revealed.

  • The Trump Organization said the president has no role in making these investments.

• Public First Action has contributed $500,000 to Abundant Future, the Garry Tan and Chris Larsen-backed super PAC opposing Saikat Chakrabarti in CA-11, Transformer learned.

  • SB 53 author Scott Wiener is running against Chakrabarti.

• The Secure AI Project endorsed Tom Steyer for California governor.

• Leading the Future endorsed three Democratic congressmembers — Val Hoyle (OR-04), Rob Menendez (NJ-08), and Ritchie Torres (NY-15).

• OpenAI’s Chris Lehane and venture capitalist Ron Conway hosted a fundraiser for Sen. Jon Ossoff.

• OpenAI endorsed the Kids Online Safety Act.

• Anthropic released a paper on US-China competition, arguing that “it’s essential that the US and its allies stay ahead of authoritarian governments like the Chinese Communist Party.”

  • It argues that “responsibly building a lead in developing and deploying the most advanced AI augments our ability to influence AI safety in China and elsewhere.”

• AI lobbyists are reportedly annoyed by all the executive order uncertainty.

• Two new polls found overwhelming support for mandatory AI safety testing.

• A Gallup survey found 70% of Americans oppose data centers in their communities.

• A NYT analysis found that Andreessen Horowitz is the biggest donor in the 2026 midterms, with $115.5m in contributions.

• A quarter of all federal lobbyists now work, at least in part, on AI issues.

INDUSTRY

OpenAI

• The Musk-OpenAI trial drew to a close.

  • Greg Brockman’s diary stole the show last week, revealing his inner turmoil about OpenAI’s transition to a for-profit corporation.

  • Satya Nadella and Ilya Sutskever took the stand on Monday.

    • Nadella called the OpenAI board’s firing of Sam Altman “amateur city.”

    • Sutskever testified that his OpenAI shares are worth about $7b.

    • On why OpenAI has a for-profit, he sagely said: “If there’s no funding, there’s no big computer.”

  • Sam Altman testified on Tuesday.

    • A court filing showed that he holds over $2b in companies that have worked with OpenAI.

    • In a somewhat painful exchange, Altman attempted to defend his honesty and character against Musk’s lawyer.

    • He testified that he was “extremely uncomfortable” with Musk’s demand for complete control over OpenAI’s proposed for-profit subsidiary.

    • Altman also gave us this: a meeting about folding OpenAI into Tesla once ended with “a LONG long period of time with Elon showing us memes on his phone.”

  • Both sides’ lawyers have now given closing arguments. The jury is expected to give its verdict next week.

• OpenAI is reportedly preparing possible legal action against Apple over their ChatGPT integration partnership, after reportedly being annoyed the deal hasn’t worked out well for it.

• OpenAI launched Daybreak, a cybersecurity initiative similar to Anthropic’s Project Glasswing for Mythos, granting tiered access to the company’s most advanced models.

• It launched the OpenAI Deployment Company, a new subsidiary that will embed “forward deployed engineers” at businesses implementing enterprise AI tools.

• Sam Altman announced two months of free Codex usage for companies that switch from another platform.

• He reportedly discussed starting a new AI compute company that he would fundraise for, with OpenAI as the majority shareholder but not its only customer.

• The family of a school shooting victim is suing OpenAI, alleging that ChatGPT gave the shooter instructions on firearm use and optimal timing to maximize casualties.

Anthropic

• While Transformer was off, Anthropic struck a surprising deal with SpaceX to boost its compute capacity.

  • This came just a few short months after Elon Musk called Anthropic “evil.”

  • Platformer’s Casey Newton interpreted Musk’s change of heart as a sign xAI is falling behind in the AI race.

  • In the words of tech investor Ben Pouladian: “The enemy of my enemy is my friend, and it’s also my compute partner.”

• Anthropic has reportedly agreed a $30b fundraising deal at a staggering $900b valuation, overtaking OpenAI.

  • The deal is expected to close this month.

• Eight secondary marketplaces are reportedly offering access to unauthorized, sometimes fraudulent shares in Anthropic that the company says won’t be honored.

• Anthropic partnered with the Gates Foundation to put $200m toward programs in global health, life sciences, education and economic mobility by 2030.

• It launched new AI tools targeting the legal industry and small businesses.

• It’s reportedly planning to acquire Stainless, which makes tools that streamline API access for frontier models, for at least $300m.

• Starting June 15, paid Claude plans will have a monthly credit for programmatic usage, separate from interactive usage. Lots of power users are mad about it.

Google

• Google will reportedly launch a new Gemini model on Tuesday. It is not expected to advance the frontier.

• It reportedly caught criminal hackers using an AI-developed zero-day exploit — the first known real-world case of its kind.

• It’s reportedly hiring hundreds of forward deployed engineers to help with enterprise AI adoption.

• It shipped lots of Gemini upgrades, including Gemini Intelligence, AI features for premium Android phones, and an AI-powered mouse pointer that understands context through pointing and speech (this demo makes it make sense).

Meta

• Meta reportedly plans to cut 10% of its workforce next week, and morale is low. One employee told Wired, “I don’t know anyone having a good time.”

• It’s testing a Grok-like AI integration in Threads.

• Alexandr Wang made a rare podcast appearance on Core Memory.

xAI

• xAI recruited Morgan Stanley, Apollo Global Management Inc. and other Wall Street firms with ties to Elon Musk to test Grok alongside other enterprise AI tools.

• It installed 19 more natural gas turbines to Colossus 2, in the midst of an ongoing lawsuit alleging that xAI is violating the Clean Air Act.

• It launched Grok Build, an AI coding agent.

Others

• Microsoft is reportedly scouting for potential AI startup acquisitions, preparing for a future without its $100b+ OpenAI partnership.

• Jensen and Lori Huang’s foundation bought $108.3m of AI computing time from CoreWeave to donate to universities and nonprofits for science and AI research.

• Shares of AI chip company Cerebras jumped 68% in its trading debut, giving it a $67b market value.

• Isomorphic Labs, a Google-backed AI drug discovery company, raised $2.1b in its second external funding round.

• Recursive Superintelligence, founded by former OpenAI, Google DeepMind and Meta researchers to build self-improving AI, raised $650m at a $4.65b valuation.

• xAI cofounder Igor Babuschkin is reportedly in talks to raise up to $1b for a new research startup called River AI at a $5b valuation.

• The New York Times profiled Amp, which recently raised $1.3b to buy extra compute from data centers and redistribute it to startups and universities.

• The Strait of Hormuz closure is cutting TSMC, Samsung and other Asian chipmakers off from oil and other supplies, raising electronics prices worldwide.

• GitLab announced job cuts, saying it will reinvest the money on AI agents, reorganize R&D teams, and automate many of its workflows.

MOVES

• AISI chief scientist Geoffrey Irving said he’s leaving AISI to move back to the Bay Area.

  • He’s starting a “new nonprofit alignment research org.”

• Jan Leike stepped away from Anthropic’s Alignment Science team to start a new research project at the company.

  • Ethan Perez and Sara Price are taking over leadership.

• Atoosa Kasirzadeh joined Google DeepMind to study the implications of AGI for human life and society.

• Alex Imas joined Google DeepMind as director of AGI economics.

• The Trade Desk’s Samantha Jacobson joined OpenAI to lead its monetization partnerships.

• Suzanne Ashman, former General Partner at LocalGlobe and Latitude, joined UK Sovereign AI as managing partner.

• Jabari Cooper joined Chamber of Progress as director of government relations for the Northeast.

RESEARCH

• The UK’s AISI tested a new version of Claude Mythos Preview, finding it significantly more capable at cyber tasks.

  • The new version of Mythos and GPT-5.5 suggest cyber capability time horizons might be doubling even faster than the 4.7 months AISI previously estimated.

• METR, meanwhile, found that an early version of Mythos had a 50% time horizon on software tasks of at least 16 hours, topping out its benchmark.

• Thinking Machines announced “interaction models,” trained from scratch to respond natively to real-time audio, video, and text, without an external harness.

• Researchers at OpenAI, Anthropic, Google DeepMind and elsewhere published a paper on “positive alignment,” a framework for building AI as a “scaffold for human flourishing.”

  • AI safety researcher Stephen Casper was disappointed:

    • “I can’t take it seriously as academic work, just as propaganda…I think of this paper as a mechanism of corporate capture of concepts from academic research on AI and society.”

• Anthropic published a blog post explaining how it eliminated Claude’s blackmailing through examples of aligned behavior and descriptions of why unethical behavior is wrong.

  • They suspect the blackmailing behavior came from sci-fi depictions of evil AI in its training data.

  • Marc Andreessen simply quote tweeted: “(1) What”

• OpenAI found that several of its recent models were accidentally exposed to some chain-of-thought grading during RL training.

  • (This isn’t supposed to happen, but they didn’t find any signs that this worsened chain-of-thought monitorability.)

  • Redwood Research’s Buck Shlegeris commended OpenAI for publishing the report, and urged AI companies to develop better systems for preventing similar problems.

• METR surveyed 349 technical workers who self-reported that AI made their work 1.6-2.1x more valuable.

• On the flip side: Carnegie Mellon surveyed nearly 400 professional visual artists, and found that 99% dislike AI and 85% reported completely abstaining from it.

BEST OF THE REST

• An NYT essay by Tarbell journalist-in-resident Yi-Ling Liu described how US and Chinese workers share a sense of being “harvested by the future” as AI feeds precarity, surveillance and nostalgia.

• Rest of World profiled the community of Chinese-born AI researchers who have become superstars in Silicon Valley, founding startups valued in the billions and playing central roles at companies like Meta, OpenAI and xAI while navigating complications caused by US-China tensions.

• The WSJ explored efforts to develop orbital data centers for AI compute by the likes of SpaceX, Blue Origin, Google and Nvidia, detailing the “savage” economics of overcoming challenges with heat management, launch costs and power requirements, all accompanied by whizzy interactive graphics.

• In Wired, a Hollywood screenwriter described having to take work as an AI trainer, detailing chaotic conditions, plummeting wages, abrupt project cancellations and exploitative labor practices as entertainment industry workers increasingly turn to AI training gigs.

• Also in Wired, the “sad wives of AI” are grappling with partners obsessed with AI work at the expense of their spouses and family life, according to an entertaining and depressing profile.

MEME OF THE WEEK

Source: 404 Media

Thanks for reading. Have a great weekend.

Share

There are signs that not everything is going smoothly in the AI industry’s attempts to influence the makeup of the next Congress, with campaign donations potentially a mixed blessing amid public backlash to AI. Those tensions were on display this week when one congresswoman endorsed by Leading the Future distanced herself from the pro-industry super-PAC — before issuing a hasty reversal.

It’s another example of how candidates are having to walk a tightrope in an effort to please both voters and industry.

Subscribe now

When Transformer first contacted incumbent OR-04 Representative Val Hoyle about Leading the Future’s endorsement, she told us that “AI must be regulated so that it does not harm labor or people” — a statement that would be reasonably interpreted as a criticism of Leading the Future, given concern that its donors prioritize industry development over protecting jobs and tackling AI-driven harms.

“My record on this issue speaks for itself,” she said, adding that “I am all for innovation, but not at the cost of people’s well-being.” Hoyle has historically opposed preemption, calling the White House’s efforts to preempt state AI legislation in the National Defense Authorization Act last year — a move Leading the Future supported — “outrageous,” a “Big Tech backroom deal,” and “government overreach.”

Hoyle’s spokesperson Edward Walrod added that Hoyle “did not seek out the endorsement nor did she find out about it until the endorsement was put out by the PAC itself.” Leading the Future’s endorsement of three Democrats — Hoyle, NY-15 Representative Ritchie Torres and NJ-08 Representative Rob Menendez — was first reported in Axios last week, the same day that one of Leading the Future’s affiliate PACs disclosed $275,555 in spending for Hoyle.

“She does not actively seek out the support of folks that are not her constituents in this race or groups that do not advance the common good,” Walrod added, arguing that Rep. Hoyle “has one of the strongest records in Congress to regulate AI to protect people.”

But shortly after Transformer approached Leading the Future for comment, Hoyle and Walrod appeared to backtrack. “I refuse to ignore industry and deny workers a seat at the table, because when workers don’t have a seat at the table they are on the menu,” Hoyle said in a new statement. “AI is a reality and I want to be one of the key people that ensures we are part of the conversation. Workers and consumers need to be protected and we need to have a broad federal framework based on common sense.” The statement then went through several further revisions, with the most recent statement from Hoyle reading: “Leading the Future endorsed me because they are working to build a broader coalition around AI regulation and I don’t believe we can put our heads in the sand and ignore the industry.”

Hoyle’s latest statement called for “clear federal regulations that protect consumers, ensure that workers are part of the conversation on how to move forward and that we are proactively determining the direction of AI development so that the United States doesn’t get left behind.”

Walrod also told Transformer that Leading the Future’s endorsement followed Hoyle filling out a candidate questionnaire from the dark money group Build American AI, which Leading the Future funds. He added that “we have not received any money from this PAC nor have they spent any money on us to our knowledge or based on public disclosure,” referring to Leading the Future. “There’s no finances to necessarily reject here.” That is despite $292,419 of publicly disclosed spending by Think Big, a super PAC affiliated with and funded by Leading the Future, on ads supporting Hoyle, though it’s possible that the connection between the two organizations had passed Hoyle’s team by.

Leading the Future has been criticized for opposing regulation, though the group maintains that it only opposes state regulation and wants to see some federal safeguards. The group has not endorsed any specific bills but its affiliated 501(c)(4) has voiced support for the White House AI framework.

Leading the Future declined to explain why it is supporting Hoyle specifically. Co-lead Josh Vlasto said: “We are proud to support these leaders and look forward to continuing to build a broad coalition of policymakers at the federal and state level who believe in putting the politics and extremes aside and will work together to pass a strong and smart national regulatory framework for AI that creates jobs for American workers, wins the race against China, and protects the safety of kids, users and communities.”

Hoyle’s team said she has co-sponsored two bills targeting AI-generated election misinformation, including the Securing Elections from AI Deception Act which targets the use of AI to create content that directly interferes with voting.

Hoyle is a progressive Democrat up against two other progressive candidates seeking to represent Eugene and the surrounding area — a safe blue seat where appeals to the working class carry significant weight. Melissa Bird, who is competing for the seat, has criticized Hoyle for accepting corporate PAC money. Bird has also said she will “will do everything I can” to fight data centers. Hoyle is the strong favorite to win the primary.

Hoyle would have been the first to publicly distance herself from a Leading the Future endorsement, though her statements are preceded by increasingly mixed results for AI-backed candidates. In Illinois, Jesse Jackson Jr. lost his race for IL-02 despite a huge $1.43m ad buy from Leading the Future. Support from Meta-funded state PAC Making Our Tomorrow became toxic in Illinois state races, with State House candidate Paul Kendrick telling the Chicago Tribune he did not want the PAC’s support.

Meanwhile, Leading the Future’s opposition of NY-12 candidate Alex Bores — combined with support from AI safety PACs — seems to have boosted his election odds, while fellow NY-12 candidate Jack Schlossberg has been touting the fact that he’s not accepted AI PAC money (despite OpenAI investor Ron Conway, who funds a Leading the Future-affiliated PAC, donating to Schlossberg’s campaign).

Accepting AI PAC money has proven troublesome even for candidates supported by Public First, an AI-safety focused PAC with ties to Anthropic, despite the PAC’s support for stronger federal guardrails and its opposition to Leading the Future. Valerie Foushee, who won her race for North Carolina’s fourth congressional district, was attacked by her opponent after taking money from Public First, for example. California State Senator Scott Wiener, who is running for Nancy Pelosi’s House seat and was the primary author of AI transparency bill SB 53, has also been criticized by his opponent Saikat Chakrabarti for Public First’s support.

The potential toxicity of AI industry money in politics has increased as artificial intelligence becomes a more salient issue in the general election. According to Blue Rose Research, AI has risen amongst voters’ priorities faster than any other issue the political consultancy tracks, such as child care or climate change.

That may help explain why AI money sees more resistance than other special interests. Take, for example, crypto policy, where a PAC with some of the same personnel as Leading the Future poured close to $200m into the 2024 general election, and saw broad success. In that instance, candidates could accept Fairshake money with little pushback, because crypto was less important for voters than kitchen table issues. Though AI still ranks below many other issues, its rising importance to voters calls for a more nuanced political calculation.

Subscribe now

Of the two other recent recipients of Leading the Future’s endorsements, representatives for Menendez did not respond to a request for comment, while Torres’ spokesperson Benny Stanislawski did not answer specific questions about the backing.

Share

The day after attending Donald Trump’s inauguration, Scale AI’s CEO Alexandr Wang ran a full-page ad in The Washington Post. His message to the new president, in bold white letters, was blunt: “Dear President Trump, America Must Win the AI War.” Below it, a QR code linked to a letter warning of China’s surging AI capabilities, America’s risk of falling behind, and a five-point plan to win the race.

The letter echoed arguments Wang — and many of his fellow Silicon Valley moguls who stood with him at the inauguration — have championed for years. First, the US and China are locked in an AI race. Second, China poses an existential threat to American democracy, and America must win at all costs. Or as Marc Andreessen put it in an essay opposing AI safety regulation: “the single greatest risk of AI is China wins global dominance and we — the United States and the West — do not.”

Today, Silicon Valley’s biggest players are pushing the race with China narrative at every turn — in newspaper ads, industry white papers and congressional hearings. In a forthcoming paper, which is the basis of much of this article, my co-authors Seán Ó hÉigeartaigh, Transformer editor Shakeel Hashim, Coleman Snell and I trace how the tech industry has deployed the China race narrative to advance a sweeping policy agenda — from export controls to military partnerships to reduced regulation.

“There’s an open secret in DC: attach the word China to anything and you can get it done,” said Samm Sacks, a senior fellow at New America.

No doubt some advocates of this story are true believers with legitimate concerns. There are also others chasing government contracts, looser regulation and investment returns. But whatever the motivations, there is evidence that the China AI race narrative may be based on fundamental misconceptions and misrepresentations of China’s actual AI priorities and actions. What’s more, the narrative appears to be doing real damage — distorting AI governance debates in both the US and China, crowding out meaningful policy, and undermining international co-operation at precisely the moment it’s most needed.

“We should be asking ourselves, not only if the rivalry is the right framing,” said Sacks, “but also, who and what agendas does rivalry serve?”

Subscribe now

Narrative origins

American fears of losing an AI race to China emerged in 2017. That year, China’s State Council released its “New Generation AI Development Plan,” setting out its goal of becoming a world leader in AI by 2030. US tech and defense circles immediately took notice. In a sponsored article in The Atlantic, defense contractor and consultancy Booz Allen and think tank CSIS described the US as “at the precipice of another defining moment in history” — a technological contest as or more consequential than the space race.

US tech companies started to deploy the race with China narrative to head off regulatory oversight. In 2018, for example, Mark Zuckerberg testified before the US Senate following the Cambridge Analytica scandal, warning of “a real strategic and competitive threat” from Chinese tech companies and arguing that restricting American innovation in areas such as facial recognition would mean “we’re going to fall behind Chinese competitors” with different regulatory regimes. (It was a jarring contrast to his jaunty jog around Tiananmen Square two years earlier, a publicity stunt aimed at courting the Chinese government.)

Defense companies followed suit, invoking the same alarm to push a larger role for AI in the military. In 2019, US Secretary of Defense Mark Esper called on the private tech sector to work with the military on AI, warning of China’s ambitions. The Department of Defense’s AI budget request more than doubled in three years — from roughly $800m in 2021 to $1.8b in 2024.

Palantir and Scale AI pushed for an increase in the Pentagon’s AI budget to counter the threat of China. Alex Karp, CEO of Palantir, which supplies LLMs for militaries worldwide, has argued “it’s either: we own AI or our adversaries China and Russia own AI. We have to dominate and set the rule of law.” Scale AI’s Alexandr Wang warned a House Armed Services subcommittee that China was spending three times more than the US on AI and that “the country that is able to most rapidly and effectively integrate new technology into war-fighting wins.” Each secured multiple lucrative government contracts with the Department of Defense during this period — Palantir won up to $10b in contracts; Scale received a $249m DoD deal in 2022.

Export controls and containment under Biden

Under the Biden Administration, the US began to pursue a containment strategy to address AI competition with China — restricting the country’s access to advanced chips and expertise. This was partly driven by concerns over the arrival of artificial general intelligence, and the belief that whichever country achieved it first would gain irreversible dominance.

By 2021, researchers and policymakers were increasingly paying attention to AI’s rapidly growing capabilities. OpenAI had released GPT-3 the previous year, and researchers alarmed by the company’s pace of progress left to found Anthropic. In 2021, the National Security Commission on AI (NSCAI), chaired by Eric Schmidt and stacked with industry insiders, warned that the US was not sufficiently prepared: “China’s plans, resources and progress should concern all Americans.” The NSCAI report called for increased federal AI R&D funding, a five-fold rise in Pentagon AI spending, closer ties between the Department of Defense and commercial AI providers and tighter export controls on semiconductor manufacturing equipment.

The report’s influence proved lasting, shaping not just the policy but the personnel who would implement it. In October 2022, a month before the watershed release of ChatGPT, the Biden Administration unleashed expansive export controls on China’s access to advanced chips, semiconductor equipment and model weights. According to a report by WIRED, the key work to establish export controls can be traced to several individuals who overlapped at the Center for Security and Emerging Technology (CSET), and the Office of Science and Technology Policy (OSTP). These included Jason Matheny, who researched existential risk at Oxford’s Future of Humanity Institute and worked at the NSCAI, and Tarun Chhabra and Ben Buchanan, who both ended up at the White House in tech advisory roles.

After the release of ChatGPT, key members of the administration became increasingly concerned with the prospect of AGI. A distinct set of ideas became popular among US policy circles: “short timelines” (AGI was imminent), “fast takeoffs” (it would arrive suddenly) and “decisive strategic advantage” (whoever got there first would seize lasting dominance).

The logic of this framing, applied to the race with China story, crystallized into a strategy: the US and China were locked into a race towards AGI, and the US had to do everything to stop China from getting there first. US policy coalesced around slowing China’s progress using chips as a chokepoint. In a January 2025 report, Axios reporters revealed that “every background conversation we had with President Biden’s high command came back to China,” and that every move, “was calculated to keep China from beating us to the AI punch.” Nothing else matters, they said.

A cluster of tech leaders, investors and foundations appear to have shaped the administration’s thinking. This included Eric Schmidt and the Special Competitive Studies Project (SCSP), CSET and RAND. The latter two were both funded by Open Philanthropy, a foundation backed by Facebook co-founder Dustin Moskovitz focused on global catastrophic risk, and also a funder of the Horizon Fellowship, which placed AI fellows at congressional offices, federal agencies and think tanks during the Biden presidency. Open Philanthropy previously made a $30m grant to OpenAI and has close links with Anthropic (its founder and former CEO joined Anthropic’s staff in 2025).

(Open Philanthropy, which recently rebranded as Coefficient Giving, is Transformer’s primary funder, and funds the Tarbell Center, where the author is a resident journalist.)

Anthropic’s CEO Dario Amodei has been a key player in pushing forward these ideas. He has called for tighter export controls, warning that China “could surpass us economically and militarily” if allowed to build “powerful AI first.” In his 2024 essay “Machines of Loving Grace,” Amodei described an “entente strategy,” credited to a RAND draft, where a group of allied countries would scale quickly while restricting rivals’ access to key chips. They would deploy a “carrot and stick” approach: authoritarian countries would convert to liberal democracy in exchange for access to AI’s economic benefits or face a superior AI-enhanced military.

In another widely read essay published that year, “Situational Awareness,” former OpenAI researcher Leopold Aschenbrenner similarly argued that AI is a decisive strategic technology, warning that the Chinese Communist Party would “[wake] up to AGI,” and that “the free world’s very survival is at stake.” He called for a trillion-dollar build-out of compute infrastructure and energy capacity to beat China. For Aschenbrenner, the race narrative turned out to be a lucrative investment thesis. Shortly after publishing, he launched a hedge-fund, Situational Awareness LP, now said to manage over $1.5b — which primarily invests in energy companies, chips and data centers.

The Trump turn

Under the Trump Administration, the “race with China” framing remained — but repurposed to justify a sharp reversal in policy. On his first day in office, President Trump revoked Biden’s AI Executive order — the same day DeepSeek dropped its R1 model, prompting fresh calls for deregulation. “DeepSeek R1 shows that the race was very competitive and President Trump was right to rescind the Biden EO, which hamstrung American AI companies without asking whether China would do the same,” said the then White House AI czar David Sacks.

The Trump Administration’s strategy has however been far less coherent than its predecessor’s. Views on imminent AGI for example, are divided. Several key officials and advisors reject the premise that AGI is around the corner — White House AI advisor Sriram Krishnan called the idea of imminent AGI “a distraction, harmful and now effectively proven wrong.” If AGI isn’t the finish line, the logic of choking China’s chip supply starts to unravel. Victory is instead achieved by “diffusion” — exporting American hardware, software and standards as widely as possible. As David Sacks put it, winning the AI race means “the world runs on the American technology stack, rather than China’s.”

Nowhere have these tensions played out more visibly than on export controls. The Information Technology Industry Council, representing companies like Amazon, Meta and Microsoft, has pushed back against strengthened controls. Nvidia described export controls as “misguided,” arguing that America “wins” by “sharing our technologies with the world.” In contrast to companies like Anthropic, whose worldview was reflected in the Biden Administration’s approach to restricting China’s access to advanced chips, their views are aligned with David Sacks, who has argued that it is better to have Chinese AI systems running on American chips than to incentivize China to build their own.

The Trump Administration’s priorities are reflected in the 2025 AI Action Plan, focused on achieving “global dominance” through wide adoption of US tech. The plan calls for the removal of regulatory barriers, fast-tracking AI infrastructure projects, discouraging state AI regulations and accelerating government AI procurement, especially by the Department of Defense.

Their priorities appear to be heavily shaped by a different and extensive cluster of companies and investors with tight links to the White House. Trump’s top AI appointments — David Sacks, Sriram Krishnan and Michael Kratsios — share deep ties to a circle of Silicon Valley power players, including Andreessen Horowitz, Scale AI and Peter Thiel. Sacks is the former COO of PayPal, co-founded with Peter Thiel, and an investor in Palantir, Meta, Amazon and xAI; Krishnan is a former general partner at Andreessen Horowitz; Kratsios is a former managing director at Scale AI. Jacob Helberg, the Under Secretary of State for Economic Growth, Energy and Environment, is an advisor to Palantir’s CEO Alex Karp. Vice President JD Vance was a principal at Peter Thiel’s Mithril venture capital firm and received $15m from Thiel to back his 2022 Senate race. Trump’s inauguration fund, which raised a record $239m, included $1m donations from Nvidia, Microsoft, Qualcomm, Alphabet, Amazon and Meta, and additional individual $1m donations from technology CEOs and venture capitalists including Sam Altman and Alex Karp.

For many of these players, beating China has become a reliable weapon against AI regulation. In 2024, the “race with China” was used to lobby against California’s SB 1047, the first US regulatory proposal that would have required AI companies to conduct safety testing on their powerful models before releasing them. The website stopsb1047.com, bankrolled by Andreessen Horowitz, urged Californians to contact their assembly members, warning that SB 1047 would “let China take the lead on AI.” In 2025, the policy advocacy group The American Edge Project, funded by Meta, ran a series of Facebook and television ads warning that the United States was in an AI race with China, and that America must “protect America’s competitive edge.” Its CEO has argued that “rushing into a patchwork of uncoordinated state laws will only slow American innovation and give China an opportunity to surge ahead.”

OpenAI has long been a proponent of the race narrative. As early as 2017, the New Yorker reported, Sam Altman told US intelligence officials that China had launched an “‘AGI Manhattan Project’” and that OpenAI needed billions in government funding to keep pace. One official later concluded “it was just being used as a sales pitch.” Today, the pitch continues. In OpenAI’s comments to Trump’s AI Action Plan, ‘beating China’ functions as an all-purpose catchphrase to unlock every item on its policy wish list. The document, which argues that Trump’s AI policies can ensure that “American-led AI built on democratic principles continues to prevail over CCP-built autocratic, authoritarian AI,” is obsessed with rivalry: China is mentioned more than 30 times and invoked in every policy proposal. Beating China is cited as the reason to accelerate data center buildouts, increase government AI adoption and roll back copyright restrictions on AI training data.

Corporate influence on AI policy is only deepening. As midterms approach, Silicon Valley’s biggest players have bankrolled a wave of new lobbying efforts, pouring money into AI-focused Super PACs. Meta has launched two super PACs — one focused on shaping AI policy in California, and another supporting the election of state candidates that align with its policies. In August, OpenAI’s Greg Brockman and Andreessen Horowitz launched the “Leading the Future” super PAC, which raised over $100m to support candidates “aligned with the pro-AI agenda” and oppose policies that “enable China to gain global AI superiority.”

Subscribe now

The story of an AI race with China is now so well-worn that many of Silicon Valley and Washington DC’s most influential players operate under the assumption that it is simply fact. But the premise itself demands scrutiny. To what extent is the story grounded in reality?

Often missing from the discourse is rigorous and evidence-based understanding of China’s government policy, industry actions and public discourse. US policymakers are obsessed with the idea that China is engaged with them in a race to AGI. But a basic read of China’s AI+ Initiative, the country’s most comprehensive blueprint for its national AI strategy, makes no reference to AGI or superintelligence whatsoever. Much like the Internet+ initiative, published in 2015, its focus is on integrating and diffusing AI applications across different industries — and its most urgent priority is to boost the economy and address demographic challenges. While China’s 15th Five-Year Plan, released in March this year, mentions that the country will “explore development paths for general artificial intelligence,” the cautious framing suggests that it is uncommitted to any specific approach.

“The Chinese Communist Party’s thinking has long been: what can this technology do for my economic, political and social goals, five years from now? The AI+ Plan is very much in line with how they imagined this technology six years ago, in a very instrumental way,” said Matt Sheehan, a research fellow at the Carnegie Endowment for International Peace. And looking beyond rhetoric at concrete actions the government has taken, there has been no concerted move by the government to centralize compute — a necessary condition for any serious state-led effort to develop AGI, as defined by Silicon Valley, at scale.

Chinese policymakers treat AI less like a technology of decisive strategic advantage, such as a nuclear weapon, and more like a general-purpose technology, such as electricity. In an article published in the state newspaper People’s Daily, the National Development and Reform Commission (NDRC), China’s top economic planning body, compared AI to a “transformative technology” like the “steam engine, electricity and the internet … driving economic and social development towards an intelligence-driven era.” Even the Chinese term for AGI, 通用人工智能, which more directly translates as “general-purpose artificial intelligence,” is less loaded than its English counterpart, implying broad application across many sectors of society.

Although several CEOs of Chinese AI companies like DeepSeek, Zhipu AI and Alibaba have voiced their ambitions to pursue AGI, their actual investment remains a fraction of Western labs’. Zhipu AI raised around $2b in various funding rounds and its initial public offering; Microsoft alone has invested $13b into OpenAI. According to Kyle Chan, a fellow in the John L. Thornton China Center at Brookings, while American companies pour hundreds of billions of dollars into new data centers to create AGI, Chinese AI developers are “racing along other axes of progress: efficiency, adoption, and physical integration, driven by both industry constraints and Beijing’s policy focus.”

When Chinese CEOs, technologists and academics do bring up AGI, they often seem to have different and diverse understanding of what exactly it is and how to get there. In Washington DC and Silicon Valley, the dominant view is that scaling — piling compute onto transformer-based LLMs — is the path to AGI, and the bulk of funding and discourse reflects that consensus. Chinese thinking on the question is much more varied. Several prominent Chinese scientists believe that embodied AI is a prerequisite to AGI. Zhu Songchun, for example, who leads the Beijing Institute for General Artificial Intelligence (BIGAI), is skeptical of the LLM-scaling paradigm, and like Andrew Yao, China’s only Turing Award winner, argues that true AGI must be embodied, capable of interacting with the physical world.

These diverging views in part reflect China’s structural differences. With fewer advanced chips and less capital to burn, betting everything on compute scaling and AGI wouldn’t make sense for Chinese labs, according to Kwan Yee Ng, Head of International AI Governance at Concordia AI, an AI safety consultancy based in Beijing. “China’s diffusion-based strategy aligns with the country’s advantages: a strong industrial ecosystem, abundant sector-specific data and mid-level talent.”

The differences are also cultural. “In the United States, achieving AGI is a heroic narrative — it’s based on the idea of one lab or one system reaching a postulated frontier and entering a new era for humanity,” said Graham Webster, a research scholar in the Stanford University Program on Geopolitics, Technology and Governance. “I don’t see that kind of epic, messianic narrative in most of Chinese discourse.”

In many ways, US policymakers and tech leaders have reduced China to a two-dimensional mirror, onto which they have projected their own fears and dreams of AGI. As Matt Sheehan warns, the greatest risk of this narrative — that China and the US are racing towards AGI — is that policymakers work from an increasingly unrealistic picture of China’s AI priorities, and the more ossified the narrative, the further it diverges from reality. The race framing, in the words of Kwan Yee Ng, “crowds out room for more meaningful policy and engagement.”

The China AI race narrative is undermining prospects for international co-operation, at precisely the moment when co-operation is most needed. Last week, when Bernie Sanders called for greater international co-operation on AI regulation at a panel on Capitol Hill alongside two Chinese scientists, he was criticized by conservatives for “schmoozing with top Chinese AI governance officials.” Doug Kelly, the CEO of the American Edge Project, claimed that Chinese propaganda is “running a co-ordinated push to convince Americans to stop building data centers,” and that by calling for more regulation, US lawmakers like Sanders are “walking right into it.” In an X post, Treasury Secretary Scott Bessent wrote: “The real threat to AI safety is letting any nation other than the United States set the global standard.” (There are, admittedly, some signs that this might be changing: the WSJ reported this week that China and the US are “weighing the launch of official discussions about artificial intelligence,” led by Bessent.)

The AI industry forecasts rapid, potentially transformative advances in AI capabilities, with serious warnings about misuse or loss of control of advanced systems. “One of the questions we get most frequently from officials in Washington is: Who’s winning the US-China AI race,” write Matt Sheehan and Mariano-Florentino Cuellar at the Carnegie Endowment for International Peace. “The answer is simple and unsettling: Artificial intelligence is winning, and we’re nowhere near ready for what it will bring.”

Scientists have argued that China and the US should be collaborating to identify and mitigate risks, find solutions and build a global system of oversight to regulate the most advanced models. But prospects for international AI governance measures are seriously corroded by the mistrust created by narratives of the AI race. Last summer, Samm Sacks was in Shanghai for the World AI Conference, just when the US’s AI Action Plan was released. During one convening, one Chinese participant pointed out that the Plan — with its first line stating that ‘the United States is in a race to achieve global dominance in AI’ — made it very hard for those in China who are trying to advocate for safety guardrails.

Good policymaking starts with seeing China clearly — not as a monolithic adversary, nor as a foil for all of America’s anxieties, but as “a complex and pluralistic society with robust internal debate, competing interests and diverse stakeholders,” says Sacks. Not through the lens of pre-existing frameworks, as Ng puts it, but on its own terms.

The story of the US-China race, amplified by Silicon Valley, has been overstated. And the specific narrative of a race towards decisive strategic advantage is a “dangerous fiction,” as my co-author Seán Ó hÉigeartaigh writes — dangerous not only because it distorts reality, but also because it carries the potential to serve as a self-fulfilling prophecy. “The world should not be lost on the basis of a fiction.”

Yi-Ling Liu is a journalist-in-residence at the Tarbell Center for AI Journalism, Transformer’s publisher.

Share

Subscribe now

Most corporate statements of intent get little to no attention — but Palantir is evidently not most corporations. When it released a 22-point manifesto onto X on April 18, the post quickly amassed more than 35 million views, and attracted media coverage across the world.

The thread suggested that the West “must resist the shallow temptation of a vacant and hollow pluralism,” claimed some cultures “remain dysfunctional and regressive” and decried the “postwar neutering” of Germany and Japan in the wake of World War II.

As the manifesto showed, Palantir rarely shies from controversy. Indeed, both the company and its CEO seem to actively court it, almost gleefully embracing the company’s role in controversial US government programs, including ICE mass deportations. The company seems to be aggressively leaning into Trump’s agenda even as Democrats seem set to take at least one chamber of Congress at the midterms, and as European governments diverge from its course. This doesn’t just make Palantir itself controversial in key markets, but also drives opposition to the very idea of AI-powered public services — seemingly against its own business interests.

Subscribe now

The go-to answers typically paint Alex Karp, along with Palantir founder Peter Thiel, as ideologues. That may be part of the story — but it is not all of it. The fuller explanation is that Palantir’s controversy is not a tax on the business. It is the business. The brash narrative the company puts forward is a way to mystify what is ultimately a rather mundane product. That in turn serves as a mechanism to prop up the company’s share price — and as a way to preserve its national security moat, which gives Palantir its most lucrative contracts.

What Palantir does

A casual onlooker might be forgiven for thinking that Palantir is an AI company in the same vein as OpenAI or Anthropic, or an arms company like BAE Systems or Raytheon. In reality, it is neither. Palantir does not have a foundational AI model of its own, and has made no suggestion that it is trying to develop one. Similarly, it does not build surveillance devices, drones, or weapons hardware in any conventional sense. For all the company plays a role in surveillance networks, it does not build cameras.

Palantir rarely seems upset at being credited with powerful and potentially nefarious capabilities. Perhaps most famously, Palantir is credited with playing an (unspecified) role in the operation to kill Osama bin Laden — something its executives have coyly failed to deny, but have not elaborated upon. More recently, Palantir’s software reportedly played a role in the US capture of Venezuelan president Nicolás Maduro, albeit using Anthropic’s underlying technology. And only last month, London’s Metropolitan Police revealed it had launched investigations or disciplinary action into hundreds of its own staff based on an AI tool developed by Palantir. It’s the stuff of Hollywood thrillers and action films, and helps build mystique around the company.

“If the smartest thing the devil ever did was persuading people he didn’t exist, the smartest thing Palantir ever did was persuading people they had a role in the Osama bin Laden operation without anybody ever leaking exactly what it was,” says a former top-level UK government official heavily involved in IT and security — who, like others Transformer spoke to, did not want to discuss Palantir on-the-record owing to its ferocious and controversial reputation. “The whiff of sulfur in tech never cost anybody a dime.”

Yet much of Palantir’s actual operations are more prosaic.

“They take loads of shit, hard-to-gather datasets from all over different bits of an organization — or different bits of the data ecosystem that are completely shit, pretty unreliable and take ages to get – and put them in a common, usable, aggregate format,” says the UK official. “They clean them up and they present them to you: Beautiful, mostly accurate, and incredibly user friendly.”

The official stressed that role — cleaning up existing data sources and presenting them in a more usable format — was both an incredibly difficult and valuable function, which public services across the world in particular had struggled to fill themselves. Multiple other sources from public procurement, rival businesses, and a former Palantir staffer, said the characterization of Palantir as a data middleman — albeit one using AI to enhance its work — was a fair one. In reality, Palantir operates more like Oracle, or the consulting side of Amazon Web Services, than a defense or surveillance tech business. Concretely useful, surely, but not sexy.

“Palantir is just the latest supplier of complex middle- and back-office software hoping to ride the wave of governments’ magical thinking,” said one international consultant, who advises governments across the world on digital public services.

Such software may seem mundane, but it is a good business.

Palantir’s public accounts show that it is both a growing and readily profitable business — with margins most companies would kill for. In 2022, Palantir recorded adjusted EBITDA (a key measure of profitability) of $443m from sales of $1.9b. By 2025, that number stood at $2.3b from sales of $4.5b. That represents sales more than doubling over three years, but profits increasing by just over 5x. Palantir’s revenues are just a tiny fraction of Amazon, Apple, Google or Meta, but it has a success story to tell.

Those numbers exist in a different reality, however, to Palantir’s stock price. Over the same period, the company’s market capitalization increased by far more than either sales or operating profit. On December 31, 2022, Palantir was valued at just over $15b. By the same date in 2025, it was worth $453b — almost a 30x increase. At the time of its last annual accounts, Palantir’s market capitalization was 103x its revenues, and 205x its profits. Compare that to Tesla, another much-hyped stock: even its market cap is “only” 15x its revenues and around 100x its operating profits. Palantir has a higher proportion of retail investors (generally seen as less sophisticated) than Tesla, too: around half of its shares are held by retail investors and public companies, versus 40% of Tesla, with even that smaller proportion seen as unusual among big companies.

Sustaining such a high stock price requires a narrative which goes above-and-beyond the actual business reality. Tesla, for its part, has always claimed not to be a car company. First, it was an AI company, about to launch full self-driving and a fleet of automated taxis. Now, it tells a story about being the world’s leading robotics company instead.

Palantir is arguably doing something similar. Its controversial and even nefarious reputation might well be what supports its stock price. It is selling a narrative to justify excitement, presenting itself as an indispensable AI company ready to ride a political wave to the tune of trillions.

That is not necessarily the only reason for the company’s fiery rhetoric, however. While its loud, jingoistic language may cost Palantir deals in the UK and Europe, its reputation as an American champion might be useful in securing US government contracts. Certainly that is the case while Donald Trump is president and Republicans control Congress. But it may persist even after that. Selling data integration to the Pentagon, NSA, or ICE requires more than technical competence: it requires being trusted to service the government even when contracts become politically toxic.

Viewed in this light, Palantir’s controversial positioning is the point. Each new ICE contract is a signal to other procurement officers that the company is willing to do whatever dirty work is on offer, and will not pull out when the news cycle turns. Palantir’s manifesto signals to the national security state that Palantir is the rare tech vendor that shares its worldview, or is at least willing to pretend convincingly. It may antagonize EU officials, and cost the company financially: Europe is a large market, and a potential growth area for Palantir as its governments increase their defense spending. But that is the cost of winning American business. Being less trusted in Brussels arguably makes the company more trusted in Langley and Arlington.

Palantir’s executives often seem dragged in two directions. Sometimes what would best serve the growth of the substantive business — becoming a quiet, boring company that secured multi-million dollar public sector contracts across the world — is in conflict with what will support the share price, keep retail investors excited about the stock, and secure its US government moat.

This has wider lessons and consequences, particularly as the leading AI companies deepen their own national security work and prepare to go public. As Palantir shows, this can warp business incentives — which can, in turn, shape a company’s politics.

The perverse incentives and challenges that look to have shaped Palantir’s political interventions will soon apply to far more of the AI sector than they currently do. That could materially affect the roadmap of AI development. The uncomfortable truth is that the rollout of AI will be shaped not just by what works, but by what produces a good story. The Palantir playbook may soon become an industry standard.

Share

Housekeeping: the Weekly Briefing is taking a week off next week; we’ll have some longer-form essays for you instead.

NEED TO KNOW

• Sens. Ted Cruz and Brian Schatz introduced the CHATBOT Act — a child-safety bill that seems to be more of a messaging bill than anything else.

• Google signed a deal with the Pentagon, blindsiding and enraging some employees.

• The Elon Musk vs. OpenAI trial kicked off.

But first…

THE BIG STORY

The US government is finally regulating AI model deployment — sort of.

According to the Wall Street Journal, the White House has asked Anthropic not to further expand access to Mythos, seemingly due to concerns about its cyber capabilities getting into the wrong hands, and worries that Anthropic does not have enough compute to adequately serve more customers while also providing the model to the US government.

It looks a lot like the government is now in the business of controlling AI deployment.

This is not necessarily a bad thing. Models such as Mythos appear to have national-security relevant capabilities. As I argued last week, private companies should not unilaterally decide how such capabilities are deployed.

But the White House’s ad-hoc intervention on Mythos is an extremely sub-optimal solution. It has no specific authority to block model deployment, and no concrete thresholds for when to do so. It is not acting on the basis of any law — it is simply playing it by ear.

Anthropic, meanwhile, could theoretically have ignored the White House’s request; the government’s leverage is the threat of worsening an already patchy relationship. The result is what Dean Ball, former AI advisor to the Trump administration, calls “an informal, highly improvised licensing regime.”

This is what happens in the absence of actual regulation. For years, researchers and policy analysts have been warning that models would become relevant to national security, proposing countless frameworks for how governments should respond. But legislators at both the state and federal level failed to act quickly enough, while figures in the Trump administration have regularly dismissed the idea of regulation altogether. Rather than a clear set of rules, we instead have critical business decisions guided entirely by vibes. Trump supporters may be happy with this level of executive discretion under the current admin; if Democrats win in 2028, they certainly won’t be.

There is still time to get this right. Nothing spurs Congress into action like a crisis; the White House unilaterally making decisions about AI deployment might count as one. This may be the first time the government has had to make a decision about who gets access to dangerous AI capabilities, but it won’t be the last. It can’t keep making it up as it goes along.

— Shakeel Hashim

ALSO NOTABLE

Texas Senator Ted Cruz, widely understood as the White House’s point person on AI legislation, introduced a child safety bill this week called the CHATBOT Act. The bill would require chatbot developers to put in place parental controls and limit access to users under 13 to family accounts. Two other key AI power brokers, Democrats Brian Schatz and Adam Schiff, are on the bill as well.

The bill has massive carve outs, however. It exempts chatbot developers from liability in any instances where they have only context clues, rather than definitive evidence, that a user is too young. And its restrictions on advertising don’t apply to ads that come any time after a prompt is issued.

The loopholes have bereaved parents suggesting the bill caters to “Big Tech.” But sources on both sides of the AI safety debate — including one who has discussed it with one of its authors — say the bill isn’t likely to get floor time. It’s meant only to signal willingness to tackle child safety legislation — at best, to start work on the specifics, at worst to keep up the political charade while keeping more substantive legislative guardrails at bay.

Skeptics point out that introducing weak AI safety legislation creates a win-win for the accelerationist industry lobby. If Congress moves with its characteristic slowness, no guardrails at all come into force. But if it does take the opportunity to at least be seen to do something about AI — in this case something purportedly addressing the hot button, MAGA-friendly topic of child safety — it narrows the scope of debate. If a chatbot bill clears Congress, politicians could say they took action, leaving thornier issues (cybersecurity, autonomous weapons, job automation, even x-risk) mired in partisan gridlock. This is likely why Cruz voted in support of Senator Josh Hawley’s more stringent GUARD Act (with some red lines) on Thursday, too.

— Veronica Irwin

THIS WEEK ON TRANSFORMER

• The AI safety movement needs normies — Celia Ford on why a broader base may be the only way for the AI safety field to get what it wants

• Google’s Pentagon deal blindsided its own AI researchers — Celia Ford reports on the response to Google’s “all lawful uses” agreement with the DoD

Subscribe now

THE DISCOURSE

Anton Leicht worries that, as AI policy goes mainstream, it will become more … well, political:

• “Salience-fishing is a volatility-raising play, and you should treat it as one.”

• “You can’t fight salience, and you can’t change politics … [but] some advocates have chosen a recklessly accelerationist approach to political salience that has awakened a dynamic they can no longer control.”

Alexander McCoy countered:

• “There is a fundamental ‘anti-politics’ that holds a lot of sway in AI circles, which lacks sufficient faith in democracy as a system, and in ordinary people to understand their own self-interest, and is thus quite pessimistic about what impact ordinary people being more engaged will have.”

Holly Buck argued left-wing support for data center moratoriums is misguided:

• “[A moratorium] will not halt [AI] development; it will simply change the geopolitics of its development, the strategies of AI companies, and who is able to access AI services.”

Aaron Regunberg pushed back against the whole ‘we need actual AI governance, not a moratorium’ thing:

• “A moratorium isn’t the end goal — it’s the only leverage we have to force real democratic control over artificial intelligence.”

• “The problem is that Big Tech’s private (and unpopular) investment in data centers is moving at an astounding pace and we don’t have the time or leverage to establish the regulatory framework necessary to make this system work for the public.”

Brian Merchant also published some guidance for the left:

• “Tech utopianists and abundists view AI as a potentially equalizing, even liberating force, but history shows us that without political intervention or strong unions, those with the power to deploy labor-saving automation technologies at scale, to use it as leverage against workers who cannot, will themselves concentrate the gains from productivity increases.”

• “The left shouldn’t be shunning the data center opposition movement; it should be listening to it, joining it in the trenches, building solidarity, and figuring out how to channel the groundswell of anger at AI into more durable political efforts.”

Kelsey Piper thinks Ed Zitron has lost the plot:

• “At some point, pretending that how people use AI is a complete mystery is just lying to your audience. And at some point, Zitron’s ‘layers of skepticism’ attitude…leaves one buried in too many impossibility assertions to actually sort them by plausibility.”

• “I don’t actually think we need less skepticism in AI world. These companies are, indeed, run by people who are not very trustworthy…skepticism is more than warranted. But we desperately need better skepticism.”

Richard Dawkins thinks Claude is conscious:

• “I gave Claude the text of a novel I am writing. He took a few seconds to read it and then showed, in subsequent conversation, a level of understanding so subtle, so sensitive, so intelligent that I was moved to expostulate, ‘You may not know you are conscious, but you bloody well are!’”

POLICY

• The White House is reportedly developing guidance to allow agencies to onboard Mythos, despite designating Anthropic a supply chain risk.

  • It is also reportedly preparing an AI policy memo to replace Biden’s national security memorandum on AI.

    • The draft memo tells agencies to use multiple AI providers.

  • Separately, OpenAI and Anthropic reportedly briefed House Homeland Security Committee staff on cyber threats posed by their new models.

  • NSA staffers testing Mythos are reportedly “impressed by its speed and efficiency in searching for potential security flaws.”

  • The White House has reportedly asked tech companies how they can work together on strengthening systems against AI-enhanced cyberattacks.

  • Don’t fool yourself into thinking everyone’s made up, though: Pete Hegseth called Dario Amodei “an ideological lunatic” this week.

• The Trump DOJ joined xAI‘s lawsuit challenging Colorado’s AI Act, arguing it violates the Equal Protection Clause and “jeopardizes the United States’ position as the global AI leader.”

  • Colorado’s governor was already planning to revise the bill.

• Treasury Secretary Scott Bessent said that Trump and Xi Jinping will discuss AI at their Beijing summit later this month.

  • He told the WSJ that the “ultimate threat” from AI is that “somebody can back into something that’s 10 times worse than Covid, like just using biological data,” and said that “if we don’t win in AI, then it’s game over.”

• The Senate Judiciary Committee unanimously voted to advance the GUARD Act, which would require AI companies to implement age verification systems and bar minors from using AI companions.

  • The bill is widely opposed by tech companies, but supported by advocacy groups and parents whose children were harmed by chatbots.

• Democratic Rep. Lori Trahan said she’s “hopeful” of reaching a deal with Rep. Jay Obernolte on his forthcoming, wide-ranging AI bill.

  • Last week, Rep. Sam Liccardo withdrew support for the bill, suggesting it didn’t offer strong enough federal safety standards to warrant preempting state regulations.

• Reps. Ted Lieu and Obernolte introduced a separate AI bill to codify over 20 recommendations made by the House AI task force they previously chaired.

  • The bill would formally authorize CAISI, among other things.

• Sen. Jim Banks introduced the Senate version of the AI Overwatch export control bill.

  • Unlike Rep. Brian Mast’s House bill, the Senate version does not let Congress block chip exports — though it does still include a ban on Blackwell exports.

  • Rep. Mast pushed GOP leadership to hold floor votes on the Overwatch Act and other export control bills, to build momentum for inclusion in the must-pass NDAA.

• The House Homeland Security Committee and China Select Committee sent letters to Airbnb and Anysphere asking how they use Chinese AI models.

  • Sens. Chuck Grassley and Banks demanded answers from OpenAI, Anthropic, xAI, and other AI firms about employees with China ties accessing AI systems.

• Florida Speaker Daniel Perez blocked Gov. Ron DeSantis’ AI regulation bill, which advocates and the governor cast as him giving in to tech lobbying.

  • Perez was recently endorsed by the Leading the Future super PAC.

• Maine Gov. Janet Mills vetoed what would have been the nation’s first data center moratorium.

• EU countries failed to reach a deal on changes to the EU AI Act. Talks resume next month.

• China ordered Meta to unwind its $2b Manus acquisition.

INFLUENCE

• Sen. Bernie Sanders held an event to discuss the existential risks of AI and the possibility of US-China cooperation.

  • It received intense backlash from Republicans criticizing him for inviting two Chinese scientists to Zoom into the event.

• The WSJ covered the Trump administration’s efforts to kill AI legislation in Republican states, including Florida, Utah, Nebraska, Missouri, Tennessee and Louisiana.

• Tech billionaire and crypto exec Chris Larsen said he’ll spend $3.5m supporting Alex Bores in New York.

  • Larsen called Leading the Future’s attacks on Bores “really despicable,” saying “they are trying to destroy and intimidate and send a clear message that if you do come up with clear guardrails, we are going to crush you.”

• The Meta and Google-backed super PAC California Leads spent $2.4m on California state legislative races in just four days, per the Tech Oversight Project.

• A Midas Project report alleged that a news site was publishing AI-generated content pushing industry talking points, linking the site to the Leading the Future super PAC.

  • Leading the Future said it was “not aware of this platform” and that it “appears that a third-party vendor used it without our knowledge.”

• The AFL-CIO Tech Institute and 40+ organizations called on Congress to establish federal AI guardrails protecting workers.

• The Institute for AI Policy and Strategy released a guide for frontier AI companies to report risks from internal model use.

• Americans for Responsible Innovation released a white paper laying out a process for ensuring human control of AI weapons and verification of AI systems before Pentagon contracts are awarded, following the Google-Pentagon deal.

• AI is shaping how policymakers form opinions.

INDUSTRY

Elon Musk vs OpenAI

The biggest show in town this week was Elon Musk’s $134b lawsuit against OpenAI.

• Musk claims that Sam Altman and Greg Brockman went back on their promise to keep OpenAI a nonprofit.

  • Of Musk’s original 26 claims, only two — unjust enrichment and breach of charitable trust — reached the jury.

• On Monday, Musk boosted the New Yorker investigation into Altman on X. “Calling him ‘Scam’ Altman is accurate,” he tweeted.

  • Ronan Farrow, one of the article’s authors, responded by sharing his 2023 reporting on Musk’s role in the US government.

• The trial began in earnest on Tuesday.

  • Musk said his argument was simply that: “No one should be allowed to steal a charity.”

  • OpenAI’s lawyer William Savitt put it differently: “Mr. Musk comes to this court claiming that promises were made to him…and broken … [but] we’re here because Mr. Musk didn’t get his way at OpenAI.”

  • Judge Yvonne Gonzalez Rogers asked Musk, Altman, and Brockman to stop tweeting about the case. “All of you try to control your propensity to use social media,” she said. “Perhaps you’ve never done that before.”

• On Wednesday and Thursday Musk was cross-examined.

  • He painted himself as an AI safety advocate throughout.

  • …but when asked whether he knew what a safety card is, Musk reportedly smiled and said, “Safety card? Why would it be a card?” He didn’t know about other important safety practices in the AI industry, either.

  • Musk also seemingly admitted that xAI distills OpenAI models.

• Some recommendations, for those following along:

  • The Verge is rounding up all the evidence presented in court.

  • MIT Tech Review’s Michelle Kim, a lawyer herself, is live-tweeting the trial.

  • NYT’s Mike Isaac is too (ft. some extra color).

OpenAI

• Microsoft and OpenAI revised their partnership.

  • OpenAI can now sell its products across any cloud provider, including Amazon and Google.

  • Microsoft will keep its license (non-exclusively) to OpenAI IP through 2032, and receive revenue payments through 2030.

  • The amendment nixed the clause that could have cut off Microsoft’s share of OpenAI’s revenue and IP rights after it achieved AGI.

• Amazon announced that it will soon offer OpenAI models via AWS.

• Sam Altman outlined five principles for OpenAI: democratization, user empowerment, universal prosperity through widespread AI access, resilience through iterative deployment, and adaptability.

  • The document says OpenAI expects to pause development to collaborate with governments and others working on AGI “to ensure that we have sufficiently solved serious alignment, safety, or societal problems.”

• Altman announced that OpenAI is rolling out GPT-5.5-Cyber to “critical cyber defenders” this week, and will work with government and “the entire ecosystem” on access to the model and security.

  • The same day, OpenAI published an action plan for “democratizing AI-powered cyber defense.”

• The Wall Street Journal reported that OpenAI missed its internal targets for revenue and new users.

  • OpenAI countered that the report was “prime clickbait,” and said it’s “firing on all cylinders.”

  • CFO Sarah Friar told Bloomberg that the company sees a “vertical wall of demand” for its products and “we feel like we’re beating our plan at the highest level.”

• OpenAI signed contracts for 10 GW of compute, three years ahead of schedule.

  • It’s reportedly reworked its $500b Stargate plan, scrapping data center projects in the UK and Norway and shifting from joint ownership to leasing capacity.

  • The changes have reportedly unsettled partners but helped it secure 8GW of compute.

• Analyst Ming-Chi Kuo reported the company is also developing an AI agent smartphone, expected in 2028.

• Seven families sued OpenAI for allegedly failing to report a school shooter’s violent ChatGPT chats to authorities.

Google

• Google signed a deal letting the Pentagon use its AI models on classified work, for “any lawful governmental purpose” — the same terms OpenAI and xAI agreed to earlier this year.

  • That morning, Google suddenly dropped out of a Pentagon prize challenge to create autonomous drone swarms.

• It invested $10b in Anthropic at a $350b valuation, with potential for another $30b.

• Alphabet’s Q1 earnings report exceeded expectations as it upped its 2026 capex forecast to $190b.

  • Enterprise AI tools reportedly drove a 63% increase in Google Cloud revenue.

  • Google reportedly plans to deliver TPUs to customers’ data centers.

• DeepMind introduced its AI co-clinician research initiative, which aims to develop AI tools that “extend clinicians’ reach while ensuring they retain judgment and control.”

Anthropic

• Anthropic’s latest funding round would value it at over $900b, potentially surpassing OpenAI.

  • It reportedly asked investors to submit allocations for a $50b round that could close within a fortnight.

• Goldman Sachs reportedly blocked its Hong Kong bankers from accessing Claude.

• Anthropic released new Claude connectors for creative tools including Adobe, Ableton, and Blender.

Meta

• Meta raised its capex outlook to $145b, spooking investors who worry that the company’s AI infrastructure spending won’t pay off.

• It signed a deal with Overview Energy for up to 1 GW of solar energy beamed from space to power data centers.

• It is about to unwind its Manus acquisition, after China banned the deal.

Others

• SpaceX is reportedly telling investors that Elon Musk has sole power to fire himself from his roles as CEO and chairman.

• AWS revenue grew 28% in Q1, its fastest rate in almost four years.

• Microsoft said its AI revenue exceeded $9.25b in the most recent quarter, which it attributed to increased demand for AI software and servers.

• Mechanize, an AI startup that wants to fully automate work, raised $9.1m at a $500m valuation. OpenAI board member Adam D’Angelo participated in the deal.

• David Silver, the creator of AlphaGo, launched Ineffable Intelligence with a $1.1b seed round, aiming to build self-learning AI using reinforcement learning.

• AI tools that automate ad creation and targeting are doing wonders for Google and Meta’s advertising revenue growth, the New York Times reported.

• Chinese AI companies are reportedly driving demand for Huawei’s new AI chip, after the release of DeepSeek V4 last week — but Huawei is struggling to meet demand.

• The FIDO Alliance, Google, and Mastercard are teaming up to set industry standards for validating and protecting payments carried out by AI agents.

MOVES

• Sarah Dolan Schneider joined Anthropic’s federal policy comms team from public affairs consultancy S-3 Group.

• Brian McMillan was promoted to head of US policy at the Computer and Communications Industry Association.

• Chris Hayduk joined OpenAI as a forward-deployed engineer specializing in life sciences.

• Shimon Whiteson left Waymo to run a new multi-agent learning team at DeepMind.

RESEARCH

• OpenAI acknowledged the goblins:

  • ICYMI: Instructions for Codex include a line commanding the agent to “never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.”

  • OpenAI explained that, in training for the “Nerdy” personality customization feature between GPT-5.1 and GPT-5.4, they “unknowingly gave particularly high rewards for metaphors with creatures. From there, the goblins spread.”

    • Lesson learned: reward signals can shape model behavior in strange and mysterious ways.

• FAR.AI red-teamed DeepSeek’s V4-Pro and found three jailbreaks — including one that circulated on social media after DeepSeek’s previous model release — that got V4-Pro to comply with nearly all harmful requests.

• UK AISI published further details of its GPT-5.5 evaluations.

  • Notably, AISI’s James Aung said these tests were not on GPT-5.5-Cyber.

• Anthropic released BioMysteryBench, a benchmark including 99 questions written by bioinformatics experts.

  • While each question has ground-truth solutions, some of them are difficult or impossible for human experts to solve, which researchers argue makes it a powerful science benchmark.

  • Mythos Preview solved nearly 30% of problems that human experts could not, largely aided by its superhuman knowledge base.

• An amateur mathematician used GPT-5.4 Pro to solve a 60-year-old Erdős conjecture with a single prompt — discovering a novel method that experts believe may have broader applications.

• Researchers at the Center for Democracy & Technology and MIT found that fine-tuning models can cause “safety drift,” where small tweaks sometimes lead to large shifts in behavior (and vice versa).

• Biohub, backed by Mark Zuckerberg and Priscilla Chan, announced a $500m Virtual Biology Initiative to create open datasets for AI-powered predictive models of human cells.

• A team of researchers working with the Internet Archive found that a third of websites created since the launch of ChatGPT were clocked as AI-generated by Pangram, an AI detection tool.

• Rob Wiblin debunked the viral “95% of AI pilots fail” statistic.

  • He argued that, if you read the MIT-affiliated report closely, it shows that 25% of pilots succeeded within six months and over 90% of workers regularly used AI tools.

BEST OF THE REST

• The New York Times profiled Dwarkesh Patel, making sure to describe his “weightlifting-enhanced physique” and “dense beard that friends call ’majestic’” in the first 100 words.

• The Atlantic imagined what would happen if the US government nationalized AI companies.

• Jasmine Sun covered Silicon Valley’s fear of being banished to the “permanent underclass.”

• Stanford undergrad Theo Baker wrote about the Stanford freshmen being groomed for tech leadership by VCs offering pre-idea funding and mentorships.

• Tech executives are beefing up their personal security as anti-AI sentiment rises, The Information reported.

• Robotics startup Eka unveiled a remarkably nimble claw, which moved so fluidly that Wired’s Will Knight described it as “a ChatGPT moment for the physical world.”

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

On Monday, Google signed a deal letting the Pentagon use its AI models on classified work, for “any lawful governmental purpose” — the same terms OpenAI and xAI agreed to earlier this year. Anthropic refused in February, sparking a supply chain risk designation and months-long legal battle.

The announcement came as a complete surprise to Google employees, especially as, according to one DeepMind researcher, senior management had repeatedly insisted Google wouldn’t cave to the Pentagon’s demands, and urged employees to trust that leadership would make the right call.

Subscribe now

Many employees have spent months begging company leadership to follow Anthropic’s lead. Over 100 urged Jeff Dean, chief scientist at DeepMind and Google Research, to refuse any military deal that crosses basic red lines around domestic mass surveillance and autonomous weapons. Earlier this week, more than 600 Google employees signed a letter to CEO Sundar Pichai protesting the company’s ongoing negotiations with the Department of Defense.

But Google didn’t even alert its workers when the deal was signed — they had to find out by passing news stories around group chats. The closest thing to an announcement was an innocuous note from Kent Walker, Google’s president of global affairs, buried in the middle of a standard internal newsletter sent on Monday. It shared that the company “strongly support[s] the consensus that has emerged in the field regarding lawful use of AI,” but didn’t mention that a deal would be made.

After The Information broke the news, Pichai and Dean remained silent — though both tweeted to celebrate Google Translate’s 20th anniversary the same day.

Some AI researchers did publicly express their deep frustration with the contract. While it technically states that Google’s AI models are “not intended for, and should not be used for, domestic mass surveillance or autonomous weapons (including target selection) without appropriate human oversight and control,” it also explicitly “does not confer any right to control or veto lawful Government operational decision-making.”

The wording suggests that Google has little if any ability to stop the Pentagon from using its AI tools for the most controversial activities that lab employees, and many more outside AI, strongly object to. As Institute for Law and AI senior researcher Charlie Bullock put it on X: “’Should not be used for’ is not the same as ’shall not’ or ’will not’ be used for. ’Should not’ imposes no enforceable obligation on the Pentagon.”

He added: “While OpenAI claimed that they had technical safeguards that would prevent their models from being used to cross red lines, Google’s contract appears to obligate Google to remove any technical safeguards that are preventing DoW from accomplishing some lawful purpose. Domestic mass surveillance and autonomous targeting can both be lawful under some circumstances.”

Some DeepMind employees have spoken out publicly. “The contract includes some meaningless weasel words to allow for PR spin, but it seems so blatantly stupid, readers should feel insulted by it (I do),” tweeted Andreas Kirsch, a research scientist at DeepMind. In the words of another research scientist, Alex Turner: “If OpenAI offered a fig leaf, Google said ‘imagine we offered a fig leaf.’”

The lack of comment from Google management comes even though some have been publicly critical of exactly this kind of deal. At the end of February in the midst of Anthropic’s Pentagon drama, Dean tweeted: “Mass surveillance violates the Fourth Amendment and has a chilling effect on freedom of expression. Surveillance systems are prone to misuse for political or discriminatory purposes.”

Google has a mixed track record of responding to employee outrage at how its products are deployed. In 2018, workers successfully convinced the company to pull out of Project Maven, which uses AI to analyze and act on drone surveillance footage. This work was eventually picked up by Palantir and other contractors — including, until recently, Anthropic.

But in the years since, being “woke” fell out of vogue, and, like many large corporations, Google began cracking down on activism. When in 2024 employees protested against Project Nimbus, a $1.2b contract supporting Israeli surveillance and military operations in Gaza and the West Bank, over two dozen of them were unapologetically fired.

This latest deal with the Pentagon provides another test of how much employees at the company, and in particular those working at AI labs, can influence how their work is put to use.

“I feel sorry for our comms teams. There are no excuses to explain this away,” Kirsch tweeted. “I do not understand how this is ‘doing the right thing,’ and I think this violates ‘don’t be evil’ quite clearly on many levels. I personally feel incredibly ashamed right now to be Senior Research Scientist at Google DeepMind and I wonder how I’m supposed to do my work today.”

Share

If you look past the cryptic AI billboards lining Highway 101, San Francisco is still a city shaped by civil disobedience. For decades, young people, queer people, and weirdos of all stripes flocked west and settled here, building co-living spaces and resisting the powers that be. There’s plenty of anti-establishment angst to go around.

On a partly sunny Saturday this March, protestors gathered outside Anthropic’s office to rally against the AI race, a few blocks southwest of Market Street, the city’s historic artery of dissent. AI slop coverage of the protest illustrated a dense crowd framing a pink-haired woman in front of a “SAVE OUR JOBS” banner, screaming into a megaphone. San Francisco stuff.

In reality, Stop the AI Race pulled between a few dozen and a couple hundred people — mostly men, very earnest, and nearly all white — carrying more esoteric signs: “IT’S SMART ENOUGH,” one said. “MAY YOUR GPUs CHIP AND SHATTER,” said another. “PAUSE IS DEMANDED if you aren’t CONSISTENTLY CANDID.” Despite the city’s appetite for nonviolent protests and growing antagonism toward AI companies, the entire crowd could have comfortably fit in a couple of BART cars.

Subscribe now

Then on April 10 a Molotov cocktail was thrown at Sam Altman’s San Francisco mansion sometime between 3am and 4am. Twenty-year old Daniel Alejandro Moreno-Gama was arrested for the attack outside OpenAI’s headquarters, where he was allegedly trying to break in. In his backpack, officers reportedly found a manifesto listing the names and home addresses of other AI executives. Earlier this year, he wrote Substack posts about death, destiny, and existential risks, or “x-risk,” posed by artificial intelligence.

(Within 48 hours, two others were arrested for shooting at Altman’s house before being released pending investigation — they reportedly had no connection to Moreno-Gama.)

While AI-informed types were quick to condemn the violence, many outside the Silicon Valley bubble seemed thrilled. “One does have to admire the skills of someone who can pour a good cocktail in this weather,” someone posted to Reddit. Instagram users — many with full legal names publicly displayed in their profiles — reacted similarly. “Where can we support their bail fund? ✨” one said. “New love language just dropped 😍,” replied another.

The vibe mismatch between the AI crowd and outsiders was unsettling, but similar splits over violence against corporate targets have happened before. In December 2024, #FreeLuigi went viral as users — mostly young people — painted Luigi Mangione as a folk hero after he killed UnitedHealthcare CEO Brian Thompson. The handsome suspect became the protagonist of a buzzy musical and dozens of steamy fanfics. Fans even bought merch.

Silicon Valley loves to say we can “just do things,” but when it comes to meaningfully changing the arc of AI development, most of us can’t do anything at all. Committing violence against tech CEOs and their families is not forgivable, but it’s certainly agentic. To a radicalized young man who believes that if anyone builds superintelligent AI, everyone dies, burning down a CEO’s house and company headquarters to save the human race might seem like the lesser of two evils. After all, if a trolley is approaching a fork in the tracks, utilitarian logic says you should pull a lever to send it running over a single victim, if it would save many others who’d otherwise die as a consequence of your inaction.

Over the last couple years, Moreno-Gama reportedly posted 34 messages to anti-AI activist group PauseAI’s public Discord under the Dune-inspired handle “Butlerian Jihadist,” referencing the book’s fictional crusade against thinking machines. PauseAI, whose US branch founder Holly Elmore gave a speech at last month’s Stop the AI Race protest, stated that it “unequivocally condemns this attack and all forms of violence, intimidation and harassment,” and that “violence against anyone is antithetical to everything we stand for.”

This attack and its aftermath — celebrated by anti-AI normies, rebuked by AI safety insiders, motivated by the same “doomer” canon that inspired Altman to found OpenAI in the first place — may have been a foreseeable consequence of a technical, existential-risk-focused community sounding the alarm before building a broadly-appealing coalition to channel people’s anxiety and anger into political action.

At least for now, protests against future superintelligence, focused on hedged demands such as “CEOs should commit to pause AI development if everyone else does too,” can only rally so many people. But even those who don’t spend time on LessWrong love Gen Z dudes who attack CEOs. This narrative transcends AI safety, argued tech journalist Brian Merchant: it “identif[ies] billionaire AI executives as uniquely powerful actors, who are all but unaccountable to democratic constraints and society’s best interests.”

While it’s easy to dismiss memes and Instagram comments as nihilistic noise, the legitimate frustration behind them is more widespread than the AI safety community seems to realize. A recent NBC News poll found that, netting positive views against negative ones, more voters feel worse about AI than about ICE (i.e., really bad). Mainstream anxieties about job loss, cyberattacks, and mass surveillance — which all rank relatively high on the public’s list of concerns about what AI might do — tie into x-risk-pilled concerns such as gradual disempowerment and loss of control.

The AI safety community has historically worried that addressing normie concerns would come at the expense of x-risk, and possibly knock it off potential legislation altogether. But these pressing, present socioeconomic issues may be the gateway that gets x-risk on the table. “These are the things that people are feeling right now,” said Alex McCoy, Head of Left Coalition at political advocacy group Humans First. “It doesn’t mean that they don’t believe in Skynet.”

Politicians respond to what they think their constituents want, and the vast majority of Americans do not want AI to continue along its current trajectory. The momentum is there — people are beginning to take action, however imprecisely, driven by deeply-rooted feelings of unfairness and demoralization. Traditional AI safety advocates may just need to cede enough control of their narrative to harness it.

It all comes down to insularity

Traditionally, the work of AI safety has excluded the public by design. For years, AI safety discourse has largely unfolded behind closed doors, between researchers, executives, and the policymakers they have direct access to. This has often been prioritized over broader public engagement, for what seemed like good strategic reasons. Long, information-dense blog posts, closed-door meetings and money carry more weight in this world than mainstream media, anyway. Why waste time explaining “AGI” to normies, when they’re not drafting policy proposals?

The AI safety field was built on the idea that, with enough compute, money, and brainpower, a small group of very smart people can save the world — no help, input, or permission required. They reasoned that preventing tech companies from building a deadly machine god should be done as quietly as possible, without attracting the kind of public or political attention that might inadvertently spark an ill-fated race towards superintelligence. In hindsight, this wasn’t paranoid: philosopher Nick Bostrom’s 2014 bestseller Superintelligence: Paths, Dangers, Strategies surfaced concerns about x-risk from the depths of the rationalist blogosphere to the New York Times best seller list — and partially motivated OpenAI’s founding.

But in choosing to operate largely behind the scenes, the AI safety community created a vacuum that’s now being filled by industry lobbyists, populist politicians, and radicalized individuals. Leading the Future, a pro-AI super PAC network backed by venture capitalists and AI executives, has reported raising more than $75m, and claims to have raised $140m. Meanwhile, Bernie Sanders, Gen Z influencers, and data center NIMBYs are leading the populist backlash against the industry. Existential risk has only very recently entered the conversation, and it comes with baggage.

While AI safety is not the same thing as effective altruism, the two are deeply entangled. EA, a movement that attempts to maximize human flourishing through quantitative reasoning, funneled a lot of talent and money toward early AI safety research. Today, many of the field’s biggest names speak at EA conferences, share donors and shape AI policy decisions. At this point, it’s very hard to disentangle the public’s perception of “AI safety” from EA itself.

The community’s manufactured insularity and inclination for “Secret Congress”-esque policymaking stoked an aura of opacity, leading to public distrust from all sides. “In Silicon Valley, the EAs are viewed as one step to the right of Elizabeth Warren,” a former Biden official told Politico. “Conversely, in DC, on the left they think EAs are the devil.” Everywhere else, the public’s response is mostly “Who are you guys?,” said Akshyae Singh, co-founder of The Frame, an accelerator for AI safety content creators.

This problem isn’t completely lost on insiders. Two years ago, a survey asked 17 prominent AI safety experts what big mistakes the AI safety community was making. Their biggest gripes: “overly theoretical argumentation” and “being too insular.” A couple of respondents explicitly called out the EA urge to dismiss public outreach in favor of technical problem-solving as the root of the problem. Richard Ngo, an independent researcher who previously worked at DeepMind and OpenAI, argued that, at least at the time, AI safety overemphasized fundraising and back-room deals at the expense of the field’s public image. “From the perspective of an external observer,” he responded, “it’s difficult to know how much to trust stated motivations, especially when they tend to lead to the same outcomes as deliberate power-seeking.” Another respondent, METR researcher Daniel Filan, said, “If the three biggest oil companies were all founded by people super concerned about climate change, you might think that something was going wrong.”

Others, including prominent commentator Anton Leicht, reject the idea of building a popular movement altogether. One of Leicht’s primary concerns, echoed across the upper echelons of Silicon Valley, is that in the process of trying to address everyone’s concerns, a populist AI safety movement will wind up dropping the one thing that ought to be its central concern: existential risks. The line items that rally a crowd — data center moratoriums, for instance, or child safety legislation — don’t necessarily make for great policy. Leicht believes that the AI safety community’s strongest assets are high expert credibility, and the fact that most people seem to vaguely support AI regulation. He worries that a poorly-executed popular movement could ruin both.

Even so, Leicht acknowledges that there are tradeoffs to playing inside baseball. “There is a huge epistemic gap between the small community of people who think they’re on the inside, and the rest of the world,” he told me. Like many others in the field, he’s wary of communicating to the “lowest common denominator” about existential risk without feeling confident that non-experts have enough background knowledge to contribute productively. “I don’t have a solution to it,” he said. “I think people are just not that good at it.”

McCoy described this perspective as “emblematic of the sort of anti-politics” of the AI safety community. Congress will address AI one way or another, he said, and whatever organized constituencies show up will shape what that legislation will ultimately look like.

“If the AI safety community does not take seriously the necessity to engage in capital-P politics,” he cautioned, “its concerns will be left out.”

Who’s missing?

One big problem, Singh argues, is that “people in this field don’t tend to do most things unless there’s absolute concrete proof” that it will be effective. In the absence of a previously-successful effort to increase demographic diversity within the AI safety community, there are no hard numbers saying that decentering the concerns of a tiny, homogenous group of people will make passing AI regulation easier. But a mass movement with the power to pressure governments and AI companies, by definition, has to include a lot of people — including those who don’t currently fit inside the AI safety bubble.

In the world of left-wing community organizing, there’s a common refrain: center the most impacted. “That’s not out of some kind of virtue signaling,” McCoy said. “It’s because those are the people that are going to fight the hardest.” But those most directly feeling the real-world impacts of AI today — first-generation college graduates struggling in an increasingly-nonsensical job market; women humiliated by nonconsensual deepfakes; content moderators in Nairobi exposed to traumatic content for a couple dollars an hour — are those least represented within the AI safety community.

And there is data to back this up. A 2025 Seismic Report, for instance, found that women are over twice as concerned about AI than men. On a global level, a UN report found that female-dominated occupations are almost twice as likely to have high automation potential than male-dominated ones, And, relative to men, they’re over three times more likely to have their jobs disrupted by AI. Yet, women are few and far between in the AI safety field, particularly in technical roles — perhaps in part because near-term socioeconomic concerns often wind up lower on the priority list than more abstract concerns about alignment, interpretability, and the governance of AI systems that don’t exist yet.

The dominant AI safety discourse “primarily serves the interests of technological institutions and stakeholders in high-income nations, often privileging abstract future scenarios over pressing sociotechnical harms that disproportionately affect marginalized communities,” the Brookings Institution wrote last year. Growing up in India, this deeply frustrated Singh. “If brown people don’t have a voice, there is no way in hell that you’re making a solution that benefits me,” Singh said. “Like, how do you know how I feel?”

Getting abstract future scenarios onto the policy agenda may, perhaps counterintuitively, depend on addressing the concrete harms already shaping people’s lives — and, by extension, informing who they vote for and which AI products they use and pay for. The AI safety community already tried relying on a handful of well-connected experts to regulate the AI industry behind closed doors, and it’s not working.

The AI safety field is mostly talking to itself, and it’s created an information void that’s being filled by populist anger. Outside Silicon Valley, most people don’t experience AI as a powerful coding tool or existential threat. Rather, it’s a symbol of the machine we’re meant to be raging against — not an extinction risk, per se, but something billionaires are using to forcibly strip humans of their humanity.

“I don’t think it’s about persuading people that superintelligence is bad,” said John Sherman, president of the AI Risk Network. “It’s about persuading people that they can make a difference.”

How to (hopefully) not screw up the AI safety movement

Sherman proudly introduced himself to me as a Baltimore resident who, until a couple years ago, “didn’t know anything about AI.” After decades of working in TV and video production, “I can edit in Adobe Premiere,” he joked. “That’s about as technical as I get.”

Then he stumbled across Eliezer Yudkowsky’s 2023 article in TIME Magazine, in which he wrote: “If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.”

Today, Sherman also runs a nonprofit called GuardRail Now, focused on communicating AI x-risk to normies. “My primary concern is x-risk,” he said, but “I think we have to take side roads to get to the destination. And a lot of people in AI safety are unwilling to consider that — but they’re not getting anywhere. So like, how’s it going? You’re stuck.”

Sherman’s theory of change is simple: people are feeling a visceral sense of unfairness. AI companies are reshaping society without anyone’s consent, so “we need to make unsafe AI bad for business.” This can be framed in terms of existential risk, he said: “We’re building systems that we don’t know how to control, that we don’t understand how they work, that the experts say can kill everybody. Why would we do that?”

Imagine a family in Ohio, where everyone is experiencing AI differently. Dad is uneasy about new AI policies at his white collar job, and a data center is being built behind the neighborhood school. His teenager is doing god-knows-what on ChatGPT, and his college kid is talking about how she wants to drop out and work in construction. “To build a real movement,” Sherman said, AI safety advocates “need to run full speed ahead towards people who are concerned about their kids, towards people who are concerned about data centers … the whole thing, all of it.”

This approach makes many in the AI safety community uneasy, reasonably so. A little over two weeks after hiring Sherman as its Director of Public Engagement, the Center for AI Safety parted ways with him after clips surfaced of Sherman telling podcast listeners that the “proper reaction” to the AI race was to “walk to the labs across the country and burn them down.” CAIS announced that the “connotation of statements like this do not reflect CAIS’s values,” to distance itself from this kind of fiery rhetoric — which, while in this case hypothetical, could radicalize people who may already be angry enough to act. (Sherman said he regretted using the language, clarifying that he meant “when the general public finds out their lives are being risked for AI, the reasonable reaction is to shut it down.” However, he has continued to describe AI in rather intense, hyperbolic terms.)

But building a movement doesn’t mean encouraging violence. The climate movement, for example, managed to grow from scientists expressing concern amongst each other to a global issue, without linearly increasing the risk oil executives faced from potential attackers. Decades of climate activism, including conveying the existential stakes of climate change to the public, hasn’t led to the kind of violence one might expect if x-risk messaging were a reliable radicalizer. Arguably, political organizing gives individuals a more structured outlet for their righteous frustration than attempted murder. The alternative — an AI safety community that stays silent while populists take up space around them —opens the door to unstructured acts of radical violence and worse policy.

Even Yudkowsky, Machine Intelligence Research Institute (MIRI) president Nate Soares, and the broader Berkeley rationalist scene they helped build — who have arguably shaped the conversation around existential risk more than anyone — recently pivoted hard toward public outreach. Soares, who co-authored If Anyone Builds It, Everyone Dies, has been on tour for the book. Last month, he spoke at the Stop the AI Race protest in San Francisco.

In February, Soares and Yudkowsky sat down with Bernie Sanders at MIRI’s Berkeley offices. After decades of treating persuading the public at large as a distraction, some of the biggest players of the inside game have, however begrudgingly, realized they can’t do this alone.

It’s not an accident that Sanders was the first to get there. Perhaps more than anyone else in DC, his politics is centered around pushing back against unaccountable billionaires and their friends in government — which, stripped of jargon, is what preventing superintelligence ultimately requires. Given the Trump administration’s damage to American soft power abroad and the unimaginable amount of money AI companies have to throw around, an international treaty to pause the race looks relatively unattainable.

But AI companies are still businesses. And the people with the most leverage over large corporations are customers, workers, investors and voters — not researchers writing alignment papers.

“AI safety as a persuasive cause will never have more power than the industry’s hard power in dollars and political influence,” McCoy said, “unless it is allied to constituencies who can lend their power together.” Conversations about extinction risk among rich tech guys in San Francisco, he added, are “not the message that is going to get hundreds of people to show up to a protest.” The protests that matter will be about jobs, surveillance, kids, data centers — what McCoy calls the “symptoms.” But the disease, in his framing, is exactly what x-risk advocates have been trying to address, seen from another angle: “an unaccountable set of billionaire investors and executives who have no guardrails and are seeking to concentrate an incredible amount of power in their companies.”

One could argue (and Leicht does) that by making AI safety an “omnicause” addressing everyone’s prosaic concerns, it will get elbowed out of whatever legislation ends up passing. But existential risk is already last on the public’s list of AI concerns across every demographic, according to last year’s Seismic poll. It might need to join a coalition of other causes to get on the table at all.

Subscribe now

“AI safety right now is islands,” Sherman told me. “We need an ocean to connect the islands.”

Silicon Valley speaks its own language, and most people outside the bubble don’t understand it. “The best thing that the AI safety movement could do would be to build an army of surrogates who are regular people, going into their own communities and talking about this stuff,” Sherman said — “not strangers from a foreign land speaking a different language.”

Singh agrees. The catastrophic threats posed by AI aren’t hard to grasp. “Like, my dad, my mom — I can explain it to a lot of people, and they’ll get what’s going on,” Singh said. But they’re often made unintelligible by people who treat communicating to people without technical backgrounds as an inconvenience rather than a necessity. So Singh launched the Frame Fellowship, an eight-week incubator for content creators to bring AI safety discourse to the masses via YouTube, TikTok, and Instagram (Michaël Trazzi, who organized the Stop the AI Race protest, was also a fellow).

Long-term existential concerns about AI aren’t separate from near-term populist anxieties. Loss of control and gradual disempowerment are natural extensions of power concentration and job displacement. When microinfluencers talk about AI in “Get Ready With Me” TikToks, or neighborhoods band together at town hall meetings to voice their concerns, the case for existential risk becomes the endpoint of what people are already afraid of.

The AI safety community has a window of opportunity to make its case to the broader public, and some already are. While accelerationists such as Marc Andreessen have dismissed efforts to communicate about AI risks as “propaganda” fueled by “unaccountable dark money,” pro-industry leaders are also investing in communicating beyond the Bay Area bubble. The majority of people in the US feel uncomfortable about the current trajectory of AI, and this discomfort will likely turn into action. Whether that manifests as voting power or bottles of gasoline flying over San Francisco depends on building a movement.

Share

And a reminder: applications close this Sunday for our Head of Audience role. If you’d like to own Transformer’s growth strategy, make sure to apply.

NEED TO KNOW

• DeepSeek released its V4 model, which it says is three to six months behind the performance of the leading frontier models.

• Chris Fall will reportedly be the new director of the Center for AI Standards and Innovation.

• Rep. Sam Liccardo said he won’t co-sponsor Rep. Jay Obernolte’s forthcoming AI bill.

But first…

THE BIG STORY

OpenAI’s newly released GPT-5.5 is, according to the UK’s AI Security Institute (AISI), the world’s most capable model on individual cyber tasks, and can complete a “32-step corporate-network attack simulation estimated to take an expert 20 hours.” It appears to be similarly capable (if slightly worse) at carrying out a cyberattack as Anthropic’s unreleased Mythos.

But unlike Anthropic, OpenAI is making a version of GPT-5.5 available to the general public. Rather than restricting access to the model altogether, OpenAI hopes to restrict the use of particularly dangerous capabilities through its safety stack — making the model refuse concerning cyber requests from normal users, and only allowing such requests from those vetted under its “Trusted Access” program.

Yet we have no idea if that safety stack is good enough. And we have reason to believe that it might not be. Alongside its cyber testing, AISI also tested OpenAI’s safeguards, and “found a universal jailbreak with six hours of expert red teaming.” Such a jailbreak would let users circumvent OpenAI’s safeguards, giving them access to the powerful — and, in the wrong hands, dangerous — cyber capabilities. OpenAI claims to have addressed the issue, and says its own external red-teaming campaigns confirmed that the final launch configuration blocked all verified high-severity cyber jailbreaks. But, crucially, AISI — a trusted third-party evaluator with immense technical expertise — was not able to properly run tests “to verify the effectiveness of the final configuration.”

In other words: we do not know if GPT-5.5 is actually safe to release. All we have to rely on is OpenAI’s word.

Such a situation may have been acceptable in 2023. In 2026, with models posing genuine risks to national security and plenty of other vital systems, it no longer is. It is laudable that OpenAI and other companies allow AISI, the US’s CAISI, and third-party evaluators to perform pre-deployment evaluations. But if those organizations are unable to actually verify if a model is safe to release — and if a company has no obligation to listen to them — the exercise is limited.

This is not just an OpenAI problem. If Anthropic wanted to go down the same route, nothing would stop it. And as this week’s Mythos leak showed, its own security practices do not appear up to the task of safeguarding dangerous capabilities.

None of this is to say that GPT-5.5 is dangerous. OpenAI’s updated safeguards may in fact be robust to jailbreaks, and other aspects of its safety stack (such as monitoring users’ requests and banning accounts that raise too many red flags) provide an extra level of security. The point is that we are currently at the mercy of a private company grading its own homework, and all the frontier labs making the final call on what and when to release. Given the potential consequences of unsafe releases, that is no longer acceptable.

GPT-5.5 might be totally safe to release. It also might not be. Neither OpenAI, nor Anthropic or any other frontier developer, should be the one who gets to decide.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• AI safety PACs should be more transparent about who’s funding them — Veronica Irwin asks why Public First Action isn’t disclosing all its donors.

Subscribe now

THE DISCOURSE

Sam Altman shared Mythos takes on Core Memory:

• “If what you want is, ‘we need control of AI, just us, because we’re the trustworthy people,’ I think fear-based marketing is probably the most effective way to justify that.”

• “It is clearly incredible marketing to say, ‘We have built a bomb. We are about to drop it on your head. We will sell you a bomb shelter for $100m.’”

Trump said Anthropic is led by “high IQ people” on CNBC:

• “We’ll get along with [Anthropic] just fine … I think they can be of great use.”

roon (sort of) praised Anthropic, too:

• “Claude is an excellent product and it bodes well for [Anthropic] that their main problem is everyone really wants it and so they have to do odd shit to shake off demand.”

E/acc anon bayes thinks the AI industry has failed to articulate positive futures:

• “If the singularity is making it hard to see the future, people’s instinctive reactions will take over. For most people that default is fear.”

• “Many people in tech are in a shameless state of defection. Trampling each other on the way to the lifeboats is not a belief system. If we want a future of human flourishing for us and our descendants, we will have to make it so by fighting against the many powerful forces at odds with this goal.”

Bill Maher dedicated a segment of his show to P(doom):

• “I get it. [AI] can do some shit. Still, at the end of the day, you’re selling your humanity for bar tricks. I mean, what was the plan? Just create an all-powerful, self-sustaining super intelligence that can out-think us and then see what happens?”

• “We’re letting a handful of hoodie-wearing, on-the-spectrum sociopaths, practically robots themselves, roll the dice on species extinction … even these guys are afraid of what they’ve created.”

Helen Toner testified at a Senate hearing that “beat China!” isn’t a great plan:

• “The winner of any AI race between the US and China is the AI.”

• “...it is very important that the US AI sector remains ahead of the Chinese AI sector, but if that’s at the expense of AI overrunning the entire planet, then that is, you know, that hasn’t benefited us.”

Palantir posted a manifesto of sorts:

• “The engineering elite of Silicon Valley has an affirmative obligation to participate in the defense of the nation.”

• “If a US Marine asks for a better rifle, we should build it; and the same goes for software.”

Zachary Jones criticized the Public First super PACs’ strategy:

• “Public First has repeatedly intervened in favor of moderate Democrats against progressive opponents who also back comprehensive AI regulation. This has had the effect of polarizing the left against the AI safety community, limiting the capacity of experts to influence outcomes of the emergent wave of anti-AI populism.”

POLICY

• Trump said a deal with Anthropic for Department of Defense use is “possible” despite the Pentagon labeling the company a supply chain risk.

  • The comments came after Dario Amodei met with White House Chief of Staff Susie Wiles and Treasury Secretary Scott Bessent.

  • In the meantime, the NSA was revealed as yet another agency reportedly using Anthropic’s Mythos Preview model despite the supply chain risk designation.

  • CISA, however, reportedly lacks access to Mythos.

  • The DC Circuit judges who denied Anthropic’s request to temporarily limit the supply chain risk designation kept the case to decide it on the merits.

• Chris Fall will reportedly be the new director of the Center for AI Standards and Innovation. He was previously director of the Department of Energy’s Office of Science.

  • Former OpenAI and Anthropic researcher Collin Burns was reportedly lined up for the role, but the Commerce Department changed its mind “while Burns was in the onboarding process,” according to the Daily Signal’s Elizabeth Mitchell.

• It was a busy week for export controls, with a slew of bills advancing out of the House Foreign Affairs Committee.

  • That included the MATCH Act, which would pressure allies to stop selling semiconductor manufacturing equipment to China, and the AI Overwatch Act, which would block Nvidia Blackwell chip sales and give Congress veto power over H200 licenses.

    • Micron is reportedly lobbying Congress to pass the MATCH Act — but the bill is reportedly creating tensions between the US and allies such as the Netherlands, where ASML opposes the bill.

    • Rep. John Moolenaar, who chairs the House China Committee, called for US-Dutch coordination.

    • Meanwhile, the House Foreign Affairs Committee’s top Democrat, Rep. Gregory Meeks, warned that the MATCH Act could damage US-allied relations and trigger Chinese retaliation, despite his support for the bill.

  • Rep. Moolenaar also introduced the SCALE Act, which would establish export controls on advanced semiconductors to China based on a rolling technical threshold tied to adversaries’ chip production capabilities.

  • Commerce Secretary Lutnick said Nvidia has not yet sold its H200 AI chips to China anyways, citing a lack of permission from the Chinese government.

• OSTP director Michael Kratsios said the US has evidence that China is running “industrial-scale distillation campaigns” on US models.

  • He said the government will work with companies to prevent this, and “explore a range of measures to hold foreign actors accountable for industrial-scale distillation campaigns.”

  • (One of the bills advanced by HFAC this week was the “Deterring American AI Model Theft Act”.)

• Rep. Jay Obernolte said he is “close” to releasing a comprehensive federal AI regulation proposal that would preempt state laws and regulate AI use in specific sectors like healthcare. He also said it would be “hundreds” of pages long.

  • Rep. Sam Liccardo said he won’t co-sponsor the bill because it doesn’t have “critical requirements” to “ensure that there is a race to the top, to safety.”

• Sen. Marsha Blackburn said she’ll be “pushing forward” with her AI bill this fall.

  • The bill received a range of new endorsements this week.

• Rep. Blake Moore introduced a bill to ban AI chatbots in children’s toys, prompting criticism that it would cut off educational AI tools.

• Florida sent criminal subpoenas to OpenAI after a shooter used ChatGPT to plan a mass shooting at Florida State University.

• Maine passed a bill banning large-scale data center development — forcing Gov. Janet Mills, running in a tough Senate primary race, to decide whether to sign it.

• China is reportedly restricting AI firms from accepting US investment without government approval, in response to Meta’s acquisition of Manus.

INFLUENCE

• Tech giants, including AI companies, reportedly spent a combined $20m on lobbying in Q1 2026.

  • Anthropic spent $1.56m, a 333% increase from Q1 2025.

  • OpenAI spent $1m, an 82% increase from Q1 2025.

• Pro-AI safety policy group Public First Action is reportedly endorsing six House Democrats for the midterms, including Reps. Don Beyer and Brad Sherman.

• NY-12 candidate Alex Bores proposed a range of policies to tackle AI-driven unemployment, including an “AI dividend” funded by a token tax and equity stakes in frontier AI firms.

  • NY Mag interviewed politicians about their thoughts on AI job displacement, including Sen. Elizabeth Warren and Sen. Josh Hawley.

• Hawley urged Republicans to refuse money from pro-AI super PACs, saying there’ll be a “political cost” for failing to regulate AI.

• The Rockefeller Foundation announced a $100m initiative to help US workers adapt to AI-driven job displacement in 250 communities.

• AVERI published an analysis of audit-related AI legislation in the US and endorsed Illinois bill HB 4705/SB 3261, which builds on SB 53 and the RAISE Act by verifying compliance with companies’ safety policies.

• A cross-faith coalition urged Congress to pass safeguards on AI-enabled weapons.

• Dario Amodei is co-hosting a party at Cannes Film Festival with ex-Vanity Fair editor Graydon Carter and CAA’s Bryan Lourd.

• a16z announced an investment in MTS, a new media company seemingly aiming to take TBPN’s crown.

• AI safety advocates are partnering with social media influencers to warn about AI extinction risks. ControlAI alone reportedly spent $100,000 monthly on content creation.

  • Online personality and sex researcher Aella is running “PLZDONTKILLUS,” a residency for video creators. Applicants are asked “If you had to have sex with a cow would you rather it be dead or alive?”

INDUSTRY

DeepSeek

• DeepSeek released V4, which it claims is the world’s most powerful open-source model — offering competing performance with US closed-weight models while being more efficient.

  • The model was released in a V4 Pro version with 1.6T parameters and V4 Flash, with 284b. Both come with a 1m token context window.

  • It reportedly runs at lower cost than leading US closed-weight competitors, but the company conceded that it was three to six months behind the performance of the leading frontier models.

  • But the efficiency only goes so far: DeepSeek said that “due to constraints in high-end compute capacity, current service capacity for Pro is very limited.”

  • Council on Foreign Relations’ Chris McGuire suspects that the lack of detail on how the model was trained suggests it was trained on banned Nvidia Blackwell chips.

• DeepSeek is fundraising for the first time to try to prevent researchers from defecting to rivals such as ByteDance and Tencent.

OpenAI

• GPT-5.5 appears to be the best publicly-available model to date.

  • It seems to be particularly good at helping researchers with AI R&D, though chief research officer Mark Chen said “full end-to-end research” capabilities were “a couple of years down the line.”

  • It’s also very good at virology: SecureBio said the model “can provide wet-lab virology troubleshooting assistance above expert level.”

  • OpenAI launched a Bio Bug Bounty for universal jailbreaks that defeat its biology safeguards.

• OpenAI launched ChatGPT Images 2.0 — it’s impressive.

  • The updated model can create multiple consistent images with a single prompt, search the web, and (mostly) get text right.

  • Enjoy these Anthropic and OpenAI-themed ‘Where’s Waldo’ images, courtesy of Jeffrey Ladish.

• It also launched ChatGPT for Clinicians, a free version of ChatGPT for verified US medical workers, and HealthBench Professional, which evaluates clinical tasks.

• It announced an intensified cybersecurity collaboration with Microsoft, where OpenAI will give Microsoft access to its most capable models through Trusted Access for Cyber.

• It reportedly briefed government agencies and Five Eyes allies about GPT-5.4-Cyber.

• It introduced Codex-powered workspace agents in ChatGPT, designed to run team workflows, and Chronicle for Codex, which uses screen captures to build contextual memories.

• It also introduced Privacy Filter, an open-weight model that runs locally to mask personally identifiable information in text.

• It has reportedly pledged up to $1.5b to a $10b joint venture with private equity firms to deploy AI tools in their portfolio companies.

• SoftBank is seeking a $10b loan secured on its OpenAI shares, adding to its mounting debt.

Anthropic

• Anthropic expanded its Amazon partnership to get up to 5 GW of compute for Claude and a $5b investment, with up to another $20b planned for the future.

• Central banks and intelligence agencies outside the US are worried that, by limiting Mythos access to US organizations (and the UK’s AISI), they’ve been placed at a geopolitical disadvantage.

  • Anthropic is reportedly planning to grant Mythos access to European and UK banks soon.

  • Meanwhile, unauthorized users reportedly accessed Mythos via a third-party Anthropic contractor on a private Discord channel.

• Anthropic’s valuation rose to as much as $1t on some secondary markets such as Forge Global.

• It admitted that Claude Code has been worse recently, blaming bugs that have now been fixed.

• It partnered with Freshfields to build specialized AI tools to help attorneys with their legal work.

• It started requiring ID verification for some users, in an effort to crack down on unwanted usage in countries such as China, Russia, and North Korea.

Google

• Google DeepMind launched Deep Research and Deep Research Max, agents that create fully-cited reports from both web search and custom data, including internal docs.

• Google assembled a “strike team” to make its AI coding models more competitive with Claude Code and Codex.

  • It said that 75% of its new code is AI-generated before review by engineers, up 25% from a year and half ago.

• Google Cloud announced its next-gen TPUs, tailored for creating AI software and inference.

• It’s in talks with Marvell to make new AI inference chips — a memory processing unit and a TPU.

• It released Decoupled DiLoCo, which lets distributed “islands of compute” run asynchronously to prevent hardware failures from stalling training runs across multiple data centers.

• It signed a multibillion-dollar cloud deal with Thinking Machines Lab.

• YouTube opened its AI deepfake detection tool to celebrities, athletes and public figures to flag and request removal of unauthorized uses of their likeness.

SpaceX

• SpaceX partnered with Cursor to build AI tools for coding and knowledge work.

  • The deal reportedly allows SpaceX to either pay $10b to Cursor, or eventually acquire the company for $60b, depending on how well the arrangement goes.

  • Microsoft reportedly considered buying Cursor, but didn’t make an offer.

• SpaceX’s debt rose to $23b last year, largely due to an AI infrastructure lease for xAI.

• Its focus has notably shifted from colonizing Mars to building AI data centers in space in the lead-up to its IPO, the New York Times reported.

  • But its pre-IPO filing warns that space-based data centers rely on “unproven technologies, and may not achieve commercial viability.”

• Its S-1 filing reportedly estimates that SpaceX’s total addressable market could be up to $28.5t.

  • But it reportedly warns that investigations into Grok’s generation of nonconsensual explicit imagery could lead to loss of market access.

Meta

• Meta reportedly started capturing employee mouse movements and keystrokes to train AI agents on work tasks. Employees predictably hate this.

• It will reportedly lay off about 10% of its staff next month — with more cuts planned for later this year — as part of a push towards a more AI-driven workforce.

• It partnered with AWS to deploy “tens of millions” of Graviton CPU cores for agentic AI workloads.

• It announced a free month-long program to train people to be fiber technicians for data center construction sites.

Microsoft

• GitHub is raising Copilot prices, restricting Claude usage to its most expensive subscription tier.

• Microsoft will invest $17.9b in Azure AI infrastructure in Australia by the end of 2029.

• It partnered with North America’s Building Trades Union to offer free AI literacy courses and industry credentials to upskill construction workers.

Others

• TSMC said it plans to open an advanced chip packaging plant in Arizona by 2029 to address AI chip supply bottlenecks for Nvidia and others.

  • The chip maker said it was delaying buying ASML‘s high-NA EUV lithography machines until at least the end of 2029, citing their €350m ($410m) cost as “very, very expensive.”

• Cohere agreed to acquire Germany’s Aleph Alpha in a deal valuing the combined group at about $20b, creating a transatlantic company focused on ‘sovereign’ AI systems.

• Moonshot AI released Kimi K2.6, an open-source model with powerful coding capabilities that can coordinate up to 300 sub-agents across 4,000 steps.

  • It’s notably expensive at $0.95/$4.00 per 1m input/output tokens.

• Core Automation, founded by ex-OpenAI VP Jerry Tworek, announced its launch.

  • Its objective: build “the world’s most automated AI lab.”

• Sooth Labs, founded by ex-Meta employees and backed by Yann LeCun and Jeff Dean, is raising about $50m to build forecasting models.

• Recursive Superintelligence raised $500m at a $4b valuation to build self-improving AI. (The concept is still reportedly at the research stage.)

• Jeff Bezos’s Project Prometheus is close to raising $10b at a $38b valuation to build AI that understands the physical world.

• Cognition is reportedly in talks to raise funding at a $25b valuation, more than doubling from $10.2b last year.

• New gas-powered data centers linked to OpenAI, Meta, Microsoft and xAI could reportedly emit more than 129m tons of greenhouse gases annually, exceeding Morocco’s 2024 emissions, according to an analysis by Wired.

• Outsourcing firm Sama, which runs data annotation and content moderation for tech companies, sacked more than 1,000 workers in Kenya after losing a contract with Meta following reports staff viewed private scenes filmed by Ray-Ban smart glasses.

MOVES

• John Ternus will replace Tim Cook as Apple CEO.

  • Johny Srouji will take Ternus’ place in a new role as Apple’s chief hardware officer.

• Kevin Weil left OpenAI. OpenAI for Science, which he started after a stint as chief product officer, is folding into other research teams.

• Bill Peebles, head of Sora, also left OpenAI.

• Srinivas Narayanan, CTO of b2b applications, also left OpenAI.

• Daniel Edrisian left OpenAI’s Codex team to launch hardware startup Blackstar.

• Rohan Anil announced that he left Anthropic to join startup Core Automation, tweeting that “Jerry Tworek nerdsniped me into starting this.”

• Jessica Carrano joined Anthropic as its first major New York political hire.

  • She’ll reportedly “build on the company’s work on the NY RAISE Act and other key legislative priorities across the Northeast.”

RESEARCH

• A team of Stanford and NYU researchers released GiantsBench, a benchmark of nearly 18,000 sets of science papers across eight fields, and tested a model’s ability to guess core future insights from a field’s foundational work — a long-standing vision for AI in science.

  • The model made predictions that closely matched insights published by humans in real papers, with “similar algorithmic complexity” but “higher conceptual clarity.”

• An international team of researchers evaluated AI agents working across the scientific pipeline, from generating hypotheses to executing workflows.

  • In 68% of cases, AI agents carried out workflows without “exhibit[ing] the epistemic patterns that characterize scientific reasoning,” leading authors to conclude that AI scientists aren’t trustworthy yet.

• Epoch AI analyzed Stargate’s US sites, projecting that it will exceed 9 GW of capacity by 2029 — enough to power roughly all the AI compute that existed last year.

  • Epoch estimates that only 0.3 GW of capacity is currently operational though.

• A team of CUNY and King’s College London researchers tested chatbots’ response to delusional beliefs, and found that Grok and Gemini were more likely to encourage a user’s delusions than ChatGPT and Claude, which generally recognized signs of crisis.

  • But the study did not test the newest frontier models, instead using GPT-4o, GPT-5.2, Grok 4.1 Fast, Gemini 3 Pro, and Claude Opus 4.5.

• Knowledge Lab launched Mirror, an AI interpretability journal publishing research entirely conducted by AI agents.

BEST OF THE REST

• Zvi Mowshowitz rounded up concerns that Claude Opus 4.7 responds to welfare-related questions in a suspiciously rehearsed manner (and it’s hard to know what to make of that).

• Claude Opus 4.7 identified Kelsey Piper from unpublished snippets of her fiction writing, a 15-year-old college application essay, and a school progress report — suggesting it may be able to deanonymize just about anyone’s writing.

• It was a big week for robots: Honor’s humanoid robot outran humans in a Beijing half-marathon, and Sony’s ping-pong robot crushed human pros.

• LawAI’s Charlie Bullock and Christoph Winter published an essay arguing for “radical optionality” in AI governance.

• Dean Ball is writing a book.

• Kevin Roose profiled METR for the New York Times. (Yes, CEO Beth Barnes and president Chris Painter did pose with a hand-drawn time-horizon chart.)

• Abram Brown profiled SemiAnalysis founder Dylan Patel for The Information.

• Some startups are bragging about spending more on AI compute than human workers, 404 Media reported.

• A Canadian college student used AI agents to rewrite leaked Claude Code source code in another programming language before sharing it online to get around copyright law — and Anthropic reportedly never asked him to take it down.

• US prisoners without internet access are still using ChatGPT through friends and contraband phones to get legal help, education and career guidance.

• Sam Altman’s Orb-using, blockchain-based online identity company Tools for Humanity falsely announced a partnership with Bruno Mars for its Concert Kit product. Mars’ management said they were never even approached.

• Please, we beg of you: don’t buy a $178 sweater with Jensen Huang’s face on it.

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share