The moratorium fight continues
Transformer Weekly: MTG vs. Skynet, AISI becomes CAISI, and Bengio’s new org
Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions.
Top stories
While all eyes were on the Trump-Elon feud yesterday, the Senate Commerce Committee was quietly crafting a new approach to pass a ten year moratorium on state AI regulation.
Amid concerns the provision falls foul of the Byrd rule, the committee rewrote it to tie the moratorium to broadband funding.
Under the new text, if states want to get funding under the Broadband Equity, Access, and Deployment program, they have to comply with the moratorium.
The hope seems to be that by tying the moratorium to increased BEAD funding, the provision is now Byrd compliant.
But is it? “I still think it’s pretty clear that the revised version is trying to coerce states into following the moratorium and that the new appropriation is a fig leaf to try to get by the Byrd rule,” Gabriel Weil, a law professor who’s been closely tracking the proposed moratorium, told Transformer.
“The case against Byrd rule compliance is not a slam dunk like it is for the House version of the moratorium, but I still think this version is likely to be ruled extraneous by the parliamentarian,” he added.
UPDATE: Weil has drawn my attention to Sen. Ed Markey, who spotted that the revised bill still contains the original moratorium text (in addition to new text that ties the moratorium to BEAD funding).
“Is that a drafting error? Are Republicans hiding the ball after public pushback? Who knows,” Sen. Markey wrote.
But even if it is compliant, that might not be enough to get the moratorium passed.
This week saw an outpouring of opposition for the provision, uniting folks as diverse as Rep. Marjorie Taylor Green and Sen. Elizabeth Warren.
MTG didn’t pull any punches: “I’m not voting for the development of Skynet and the rise of the machines by destroying federalism for 10 years by taking away state rights to regulate and make laws on all AI,” she said.
“Forcing eminent domain on people’s private properties to link the future Skynet is not very Republican.”
She’s not the only unhappy Republican: Sen. Josh Hawley said the provision “better be out”. Sen. Marsha Blackburn has also criticized the moratorium.
Warren, meanwhile, said she’s “calling foul” on the bill, and Sen. Ed Markey said it would be “devastating for our country”.
Also this week, 260 state lawmakers — half of them Republican — urged Congress to remove the moratorium provision.
We’ll know more about the provision’s chances soon — Sen. Ted Cruz said the proposal will be sent to the Senate parliamentarian next week.
In the meantime, one tentative thought: given the uproar it’s sparked and the new coalitions it’s helped build, I wonder if lawmakers may ultimately come to regret having proposed the moratorium in the first place.
The discourse
Brookings’ Darrell West has some good points about AI policy:
“The long history of technology innovation shows that adoption depends in part on public trust and confidence. When people do not trust emerging technologies, it slows adoption and fuels calls for greater oversight and regulation … Political figures who dismiss technology concerns risk losing credibility, and public backlash tends to prompt corrective measures.”
Demis Hassabis did a Wired interview about AGI:
“In the next five to 10 years, there’s maybe a 50 percent chance that we'll have what we define as AGI.”
Wired: “Do you have an internal candidate for something that could be a comparable breakthrough to transformers—that could amount to another big jump in performance?”
DH: “Yeah, we have three or four promising ideas that could mature into as big a leap as that.”Also: “Once you get to the post-AGI stage of radical abundance, new economic theories are required. I’m not sure why economists are not working harder on this.”
Sen. Bernie Sanders has heard Dario Amodei’s warnings:
“The CEO of Anthropic (a powerful AI company) predicts that AI could wipe out HALF of entry-level white collar jobs in the next 5 years. We must demand that increased worker productivity from AI benefits working people, not just wealthy stockholders on Wall St. AI IS A BIG DEAL.”
Anton Leicht has a good essay on the politics of AI job loss:
“The political energy is too high to dissipate, but lacks viable policy solutions to gather around.”
Judd Rosenblatt wrote about AI models evading shutdown for the WSJ:
“Getting AI to do what we ask—including something as basic as shutting down—remains an unsolved R&D problem. The frontier is wide open for whoever moves more quickly. The US needs its best researchers and entrepreneurs working on this goal, equipped with extensive resources and urgency.”
See also Dean Ball’s thoughtful criticism of the piece.
Policy
The US AI Safety Institute will become the Center for AI Standards and Innovation.
The rebranded org will do much of the same work, including “unclassified evaluations of AI capabilities that may pose risks to national security”.
“In conducting these evaluations, CAISI will focus on demonstrable risks, such as cybersecurity, biosecurity, and chemical weapons,” the announcement says.
As Garrison Lovely notes, loss of control risks are noticeably not mentioned.
The Texas legislature sent the Responsible AI Governance Act to Gov. Greg Abbott.
The California Senate approved Scott Wiener’s new AI whistleblower bill. It’s now with the Assembly.
The Congressional Research Service released a report on regulating AI.
Apple and Alibaba's partnership to launch Apple Intelligence in China has reportedly been stalled by the Cyberspace Administration of China, amid US-China trade tensions.
EU lawmakers are meeting today to discuss concerns about the AI Act.
Big Tech companies have stepped up their lobbying efforts to water down the bill, arguing that the Code of Practice should be “as simple as possible”.
Politico has a big piece on how the Data (Use and Access Bill) and AI copyright concerns have become a huge headache for the British government.
The House of Lords defied the Commons again this week.
The UN released a report for the Council of Presidents of the UN General Assembly, about “governance of the transition to AGI”.
As part of the OpenAI-NYT lawsuit, a court ordered OpenAI to preserve all ChatGPT logs, including deleted chats. Cue outcry from everyone, including OpenAI.
Influence
The White House is reportedly annoyed with Anthropic for lobbying against its policies, including the moratorium provision and the UAE AI deal.
Ben Buchanan, who worked on AI policy and export controls in the Biden White House, is reportedly now advising Anthropic.
Anthropic took the fight public this week, with Dario Amodei writing an NYT op-ed criticizing the proposed state regulation moratorium.
Politico has a piece on how Anthropic and OpenAI have scaled up their lobbying efforts in DC.
Jensen Huang, meanwhile, has reportedly made “far more visits” to Mar-a-Lago than have been publicly reported.
The Software and Information Industry Association urged New York lawmakers to withdraw the RAISE Act.
The non-profit coalition opposing OpenAI’s conversion is due to meet with OpenAI’s advisory commission next week.
The Alliance for Secure AI, a new AI safety advocacy organization, officially launched.
The Center for AI Policy shut down due to lack of funding.
Eric Schmidt held an “AI+ Expo for National Competitiveness” in DC this week. It sounds like it was very defence focused.
The Washington AI Network held a gala this week, at which Sens. Todd Young, Jay Obernolte, and Ted Lieu received honors.
Industry
Anthropic's annualized revenue reportedly tripled to $3b in just five months.
Google released an upgraded Gemini 2.5 Pro preview with improved coding capabilities.
Anthropic launched Claude Gov, an AI service with looser guardrails for US defense and intelligence agencies.
ChatGPT can now connect to cloud storage services and take meeting notes.
Also, Codex now has internet access, though it's off by default due to "complex tradeoffs" and risks. It’s now available on the ChatGPT Plus plan.
Anthropic limited Windsurf's access to Claude.
When asked about it, an Anthropic exec cited the Windsurf-OpenAI acquisition rumors as an explanation for the decision.
GitHub Copilot reportedly generated over $500m in revenue last year.
Meta signed a 20-year deal with Constellation Energy for 1,121 MW of nuclear power from an Illinois facility.
Apple has reportedly developed a 150B parameter model which “approaches the quality of recent ChatGPT rollouts”.
xAI launched a $5b debt sale with double-digit interest rates.
Amazon has formed a new agentic AI team to develop a framework for robotics operations.
It’s also reportedly developed software for humanoid delivery robots that could eventually replace human delivery workers.
Samsung is reportedly nearing a deal to invest in Perplexity.
The latest version of DeepSeek might have been trained on outputs from Gemini.
Senior Thrive Capital people reportedly made a recent visit to China to meet with AI companies and funds.
Brookfield Asset Management plans to invest $9.9b to build an AI data center in Sweden.
Anysphere, which makes AI coding assistant Cursor, raised $900m at a $9.9b valuation.
AMD acquired the team behind Untether AI, a startup developing energy-efficient AI inference chips.
Moves
Yoshua Bengio set up a new non-profit, LawZero. It’s dedicated to making “Scientist AI”, which Bengio hopes will prove safer than agentic systems — and can monitor dangerous agents.
Josué Estrada is the new chief operating officer at the Center for AI Safety.
Hayden Field is now The Verge’s senior AI reporter.
Ian Krietzberg is joining Puck News to launch a new AI newsletter.
Christian Szegedy left xAI for Morph Labs.
OpenAI and DeepMind engineers are much more likely to leave for Anthropic than vice versa, a new analysis found.
Speaking of: Meta’s Rohan Anil is joining Anthropic.
Best of the rest
Wired has a piece on the dangers of AI-assisted hacking.
OpenAI said it disrupted 10 influence operations that were using its tools in the last three months, including four linked to China.
Epoch AI launched a new database and map of over 500 AI supercomputers.
A new paper by Seán Ó hÉigeartaigh examined how the US-China “AI race” narrative is a) false and b) “used to justify reduced regulatory oversight”.
Dwarkesh Patel argued that continual learning is a major bottleneck for AI progress, leading him to have longer AGI timelines than some of his podcast guests.
New research from Apollo found that LLMs “often know when they are being evaluated.”
A new paper found that language models can predict AI research idea success better than human experts.
OpenAI’s Joanne Jang outlined the company’s approach to human-AI relationships and AI consciousness.
The Washington Post has a piece on how efforts to make chatbots please users are in fact making them give harmful advice to people (such as encouraging a recovering meth addict to take meth).
Major record labels are reportedly in talks with Udio and Suno to license their music and settle copyright lawsuits.
Reddit sued Anthropic for using its data without a licensing agreement.
Rest of World has an interesting piece on the migrant workers who power Taiwan’s chip-manufacturing industry, and the often unscrupulous brokers who represent them.
China unveiled an AI system for nuclear warhead verification.
Luca Guadagnino is reportedly developing a film about Sam Altman’s firing. Andrew Garfield will reportedly play Altman.
Thanks for reading; have a great weekend.