A Fable about the future of power
Transformer Weekly: Preemption’s child safety push, OpenAI’s pause preparations, and SpaceX’s IPO
Welcome to Transformer, your weekly briefing of what matters in AI. And if you’ve been forwarded this email, click here to subscribe and receive future editions.
NEED TO KNOW
The White House is reportedly negotiating federal preemption of some state AI laws in exchange for support on social media and AI child protection measures.
OpenAI called for an international organization “to make it possible for the world to take coordinated action, including slowing frontier development when needed.”
SpaceX raised $75b in its IPO at a $1.77t valuation.
But first…
THE BIG STORY
Fable, Anthropic’s latest AI model, is very good at AI research and development. Anthropic’s own researchers use it — and Mythos, its more powerful, still publicly unavailable sibling — to automate much of their work in advancing the frontier of AI.
But if a non-Anthropic employee wants to do the same, they’re out of luck. Fable has “safeguards” built in to prevent it from being used for frontier AI development. It’s a controversial, complicated move — and one that’s indicative of the strange way power works as AI development accelerates.
As we’ve previously covered, the leading AI companies think they are on the cusp of fully automating AI R&D. As they move closer, access to powerful AI models becomes a strategic asset: today’s best model is the key tool for building tomorrow’s.
People in this world have theorized for years that companies would eventually start withholding that access. This week, it stopped being a theory.
There are lots of reasons to withhold that access. The most obvious is profit: why let your competitors use your tools to catch up? Anthropic’s stated reason is national security: the company says it doesn’t want “foreign adversaries” using Claude to “erode [America’s] advantage.” And some suggest the unstated reason is that building a big lead makes an eventual pause more likely — during which one can take costly actions (like devoting compute to alignment research) that hopefully make us all safer.
None of those reasons justify Anthropic’s initial decision to hide these guardrails from users (in an effort to make it harder to get around them) — a move which prompted widespread outrage and a quick reversal. But though the guardrails are now disclosed, they are still present. And the question of whether that is justifiable relies on the impossible: knowing the motivation behind it.
“One of the hard things for Anthropic is that the actions you’d take from sincere safety concern often look exactly like the actions you’d take to entrench your own power,” X user Maja said this week. Anthropic could be completely well-intentioned in withholding Mythos’ capabilities for itself. It also could be the actions of a standard, profit- or power-seeking company. It should not matter. We’re talking about whether Anthropic is doing this for good reasons, but we should be talking about who decides whether it happens in the first place.
There is something deeply uncomfortable about a private company having this much control over what could be the most important technology yet developed. It is not at all obvious what the better solution is: a government forcing a company to sell to its competitors has its downsides, too. One thing, however, is clear. Deciding who gets access to frontier AI models is a new form of power — and we have failed to answer the hard questions about who should wield it.
THIS WEEK ON TRANSFORMER
Making deals with AI sounds crazy. Is it? — Celia Ford explores a strange and new debate in AI safety
OpenAI isn’t being consistently candid about Leading the Future — Veronica Irwin reports on the key details missing from OpenAI’s statements on the super PAC
THE DISCOURSE
Welp, Anthropic employees are done coding:
Nat McAleese: “I always felt Opus 4.5 could barely code; 4.6 was just-about-useful, but I have barely written a line of code since fable.”
Liv Gorton: “I joined anthropic a ~month ago and have written ~no code myself.”
Nerfing drama aside, Claude Fable 5 appears chillingly autonomous:
Felix Rieseberg: “With Fable 5 out in the world, I think a third era quietly started today.”
“I no longer tell Claude to investigate a particular crash report. It runs in a loop … its job is no longer to help me fix a crash, it’s to keep our apps from crashing.”
Ethan Mollick: “With Fable the spell has gotten powerful enough that I am no longer sure I am the wizard. I am closer to a patron. I describe what I want, I pay for it, and I judge the result. The conjuring happens somewhere I cannot watch, in hundreds of small choices I never get a vote on.”
“[It’s possible] that the more capable the model, the less there is for a human to meaningfully do, and the black box is the price of power.”
Miles Brundage is frustrated:
“I eagerly use Anthropic products because the models are good[,] but I do genuinely think they are high on their own supply + unaware of just how many bugs they are constantly introducing due to overreliance on Claude + shipping too many things too quickly.”
Sriram Krishnan had thoughts on AI gatekeeping:
“Just to state the obvious: think there’s a collision course between those who believe research and science should be open and those who believe we are in an accelerating singularity curve. I have many smart friends who have believed both for a while but seeing more and more their realization that these beliefs will be in conflict.”
bayeslord had a more cynical take, in light of the Fable release:
“They didn’t mean pause AI research, they meant pause your AI research”
Joshua Achiam thinks people misunderstand the distinction between OpenAI and Anthropic:
“Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.”
“There are a lot of innocuous and even quite agreeable choices in Claude’s constitution that potentially endow it with a huge amount of authority, maybe even a mandate, to make complex ethical decisions about how to interact with human systems and who to grant power to … cloaked in the language of ethics and virtue there is a sharp and potentially quite lethal double edge to this sword.”
Jasmine Sun wrote advice to the graduating class of 2026:
“I often ask AI folks what they’d tell a normal 22-year-old. Don’t know, they’re screwed, is the non-answer I get most. In that response, I hear depressing defeatism: What can anyone do in the shadow of the technocapital machine?”
“Relish the pivots; ride the waves; recite the Serenity Prayer every morning and chase sunsets at night. I won’t tell you that the future’s smooth sailing. But what a thrill to be alive!”
POLICY
The White House is reportedly negotiating federal preemption of some state AI laws in exchange for support on social media and AI child protection measures. Sen. Marsha Blackburn is leading the negotiations.
Sen. Ted Cruz appears to be involved in the efforts, too, saying that federal preemption and child safety bills are “an element of discussion” for an upcoming markup.
Chief of Staff Susie Wiles, first lady Melania Trump, staff for the Office of Science and Technology Policy and the National Economic Council reportedly also met with children’s online safety groups, including the American Principles Project and Ethics and Public Policy Center, to discuss Blackburn’s KOSA and the App Store Accountability Act.
Trump reportedly blindsided AI company leaders when he announced a meeting to discuss the government taking equity stakes in their firms.
At least a dozen GOP House and Senate offices opposed Trump’s proposal, with Sen. Ted Cruz stating “I don’t think the federal government should be in the business of being an equity holder in private companies.”
On Wednesday, Trump doubled down, saying “if we do [take equity stakes], the public will become very rich.”
Sriram Krishnan is leaving his position as a top AI policy advisor in the Trump administration at the end of the month.
Thomas Lind is also leaving his position as head of policy at the Office of the National Cyber Director.
National Cyber Director Sean Cairncross reportedly asked CAISI to halt public reports on model assessments.
Notably, CAISI wasn’t included in the list of agencies tasked with drafting “standardized AI national security Test, Evaluation, Verification, and Validation methodologies” in last week’s National Security Presidential Memorandum.
Lawmakers from both parties questioned why the Treasury, rather than CISA, was given the lead role in AI cyber defense under Trump’s EO.
House Speaker Mike Johnson, House Majority Leader Steve Scalise and the co-chairs of the House Democratic Commission on AI cast doubt on a bipartisan AI regulation proposal from Reps. Jay Obernolte and Lori Trahan.
Republicans pushed for it to include a broader preemption measure, while Democrats said the bill “does not meet the enormity of the moment.”
Sen. Chuck Schumer said passing AI legislation in this Congress will be “hard,” but that he “would very much like to see that get done the sooner the better.”
The Senate Banking Committee is reportedly weighing an export control markup in the coming weeks, potentially for inclusion in the NDAA.
Sens. Mark Kelly and Jim Banks introduced the AI DATA Act, which would require the Labor Department to track AI’s impact on the workforce through quarterly surveys and annual reports.
Reps. John Moolenaar, Jay Obernolte and Jennifer McClellan introduced the GUARD Act, which would require national security reviews of robots made by “adversaries” (i.e. China) and block those posing threats.
The UK launched the AI Economics Institute, chaired by Nobel laureate Simon Johnson to research AI’s economic impacts and inform policy, with collaboration agreements from Anthropic, OpenAI, Google and Microsoft.
The UK also announced an AI Hardware Plan to provide funding for AI chip companies.
Taiwan is reportedly considering stricter export controls that would restrict AI chip sales to all customers in China and enable prosecution of smuggling as a criminal offense for the first time.
INFLUENCE
Anthropic published a policy framework, calling for the government to have “the legal authority to block or deter” deployment of models that pose catastrophic risks, mandatory third-party testing, and narrow federal preemption of state AI laws.
It also published an economic policy framework, proposing various policy options at different levels of unemployment — including a universal basic income scheme in the case of “unprecedented levels of unemployment.”
Dario Amodei released a policy essay alongside the framework.
OpenAI called for an international organization “to make it possible for the world to take coordinated action, including slowing frontier development when needed.”
Former Andreessen Horowitz partner John O’Farrell criticized AI industry political spending, arguing that PACs like Leading the Future are trying to “intimidate politicians” and silence debate on AI regulation, rather than engaging seriously with policy questions.
OpenAI said it found “PRC-linked influence operations” targeting data center buildouts.
Its report found “no evidence of breakout” from the campaign and that “most of the social media posts … generated little or no observable engagement.”
Nvidia CEO Jensen Huang declined Sen. Elizabeth Warren‘s invitation to testify before the Senate Banking Committee on AI, China, and US export controls.
Congressional staffers drafting federal AI legislation reportedly took an industry-funded trip to meet Google, Nvidia, and other companies lobbying to preempt state AI regulations.
A new Reuters/Ipsos poll showed that half of Americans fear AI could put someone in their household out of work, with Democrats (61%) more worried than Republicans (47%).
INDUSTRY
OpenAI
OpenAI filed a confidential S-1 for a potential IPO.
Sam Altman reportedly told staff that while OpenAI expects to go public “within the next year … the faster the potential RSI takeoff looks like it could be, the more it could be advantageous to delay an IPO.”
Sam Altman and chief scientist Jakub Pachocki outlined the company’s top three goals:
(1) build an automated AI researcher by March 2028,
(2) accelerate the economy, and
(3) give everyone on Earth a personal AGI
It’s planning to overhaul ChatGPT to shift focus from its chatbot to Codex and AI agents.
It’s negotiating a lease on a planned 10 GW data center in Ohio that could cost more than $500b, potentially with credit support from Nvidia.
It’s thinking about slashing prices to lure enterprise customers away from Anthropic.
It’s acquiring Ona, which will provide cloud execution environments for Codex agents.
SpaceX
SpaceX raised $75b in its IPO at a $1.77t valuation. Its shares will list later today.
It reportedly plans to launch initial demos of orbital data centers by late 2027.
Elon Musk came under fire from UK politicians for stoking tensions around riots targeting immigrants following a murder in Belfast.
Former xAI engineer Devin Kim filed a lawsuit claiming he was fired for raising safety concerns about Grok, including its ability to increase discrimination and provide information about WMDs.
The suit says that the engineer’s supervisor, xAI co-founder Jimmy Ba who left earlier this year, ignored safety directives from Musk.
Kim was recently appointed president of the Center for AI Safety. Dan Hendrycks, CAIS’s founder, is an advisor to xAI.
Anthropic
Anthropic launched Claude Corps, a year-long fellowship program placing 1,000 fellows at US-based nonprofits, where they’ll build AI tools with their host organization.
Hosts range from local YMCAs to conservationists to mental health support groups.
Apollo and Blackstone raised $35b to finance its lease of Alphabet chips.
It reportedly signed over a dozen letters of intent to lease data center facilities totaling more than 1 GW of capacity, its first direct data center tenancies.
Dario Amodei reportedly only has one direct report (chief of staff Avital Balwit), freeing him to do other work while Daniela Amodei manages the rest of the executive team.
Apple
Apple announced its newest generation of Apple Foundation Models, built in collaboration with Google.
They include AFM 3 Core Advanced, which runs on-device, and its server-side model AFM 3 Cloud.
The new Siri AI is powered by these models.
Its stock dropped over 3% after the launches.
Google agreed to pay SpaceX $920m per month for access to 110,000 Nvidia GPUs, CPUs and other infrastructure.
It pledged $50m for training trade workers on the kinds of jobs involved in building data centers.
Others
Meta officially cut ties with Manus: Manus can’t access Meta’s internal data system, and Meta can’t use Manus tools.
Amazon disclosed its data center water use, which it claims beats the industry average and is lower than its own previous usage, despite operating more data centers.
Nvidia and SK Hynix partnered to design and manufacture memory chips for AI.
GitHub reportedly disabled 73 repos after hackers injected malware that would steal user credentials when opened in AI coding platforms.
ByteDance spun off its AI drug discovery platform Anew Labs into a separate entity (still largely under ByteDance’s control).
Prometheus, Jeff Bezos’ AI startup for manufacturing and engineering, raised $12b in Series B funding at a $41b valuation.
Former xAI employees, including xAI co-founder Igor Babuschkin, launched River AI, which aims to build AI that “works entirely for you.”
Perplexity reportedly intends to IPO in 2028 whether or not doing so goes well for SpaceX, Anthropic and OpenAI.
Chinese startup Moonshot AI is seeking up to $2b in funding at a $30b valuation, up from a just $4b valuation in December.
MOVES
Clive Chan left OpenAI‘s custom chip program to join Anthropic.
Gabriel Petersson resigned from OpenAI, tweeting that there’s “one last product i need to build before AGI.”
Chris Lovejoy, an ex-medical doctor, joined Anthropic‘s Applied AI team.
Allison Duettmann, CEO of Foresight Institute, joined Mythos Ventures as a Venture Partner.
Leigh Nolan is the Institute for AI Policy and Strategy’s new policy director.
Anissa Gardizy joined The Wall Street Journal as a reporter covering cloud computing and AI infrastructure.
RESEARCH
Researchers at the AI Security Institute were able to jailbreak Claude Fable 5 in just a few hours, and got close to a universal jailbreak within a few days.
Pliny the Liberator published the ~120,000-character Claude Fable 5 system prompt.
Google DeepMind announced a $10m funding call for research focusing on multi-agent systems.
Cosmos Institute announced its new cohort of senior research fellows, which includes Google DeepMind’s Séb Krier and Anthropic’s Matthew Botvinick, among other big names.
Geoffrey Irving launched Sequent, a new nonprofit research organization focused on aligning superintelligence.
A new Google DeepMind paper explored four pathways from AGI to artificial superintelligence, arguing that instead of a “single transformative step change” we might experience “a series of transformative societal changes.”
Indicator’s Alexios Mantzarlis ran an adversarial test on Pangram, which tricked the detector into misidentifying AI-generated text as human between 75-92% of the time.
Despite the finding, Mantzarlis said that he was “relatively impressed at the Pangram’s solidity in the face of adversarial attacks.”
Cornell researchers Sil Hamilton and David Mimno figured out why the fictional “Elias Thorne” (often a lighthouse keeper or clockmaker) keeps starring in AI-generated stories.
GPT-3.5 (which powered the original ChatGPT) was used to make WildChat, a dataset of 1 million ChatGPT conversations that’s since been used to train subsequent LLMs across frontier companies. 166 chats had the name “Elias.”
Alignment training may lead models to prefer a small subset of “safe” WildChat content — including Elias-related stories, apparently.
BEST OF THE REST
Many people appear to be using AI to mask their low literacy, Axios reported.
An expert panel — Daron Acemoglu, Dean Ball, Ethan Mollick, Clara Shih — sat with NYT’s Bill Wasik to talk about how workers should brace for AI’s impact on jobs.
With big IPOs on the horizon, the San Francisco housing market is royally fucked.
Move over, AI 2027. A group of European researchers published Europe 2031, an imagined (but all too plausible) future where Europe “slide[s] into irrelevance” as AI accelerates around it.
METR’s Ajeya Cotra and Understanding AI’s Timothy B. Lee debated how long it will take for AI to become self-sufficient — a glossy continuation of a past Twitter debate.
Helen Toner explained AI chatbots to Oprah.
Someone (maybe for real?) built CrankGPT, a hand-crank-powered computer marketed as a “human-powered, fully local and private AI solution.”
MEME OF THE WEEK
Credit: tomie
Thanks for reading. Have a great weekend.