The best AI bill yet may not get far
Transformer Weekly: The Obernolte-Trahan AI Act, Trump’s executive order, and Anthropic discusses a pause
Welcome to Transformer, your weekly briefing of what matters in AI. And if you’ve been forwarded this email, click here to subscribe and receive future editions.
NEED TO KNOW
President Trump signed an executive order establishing a voluntary evaluation and early-access framework for frontier AI models.
Senior Trump administration officials have reportedly discussed the government acquiring equity stakes in major AI companies, with Sam Altman floating the idea of OpenAI voluntarily ceding shares to the government.
Anthropic called for the world to have “the option to slow or temporarily pause frontier AI development,” warning that recursive self-improvement might be closer than many believe.
But first…
THE BIG STORY
The Great American AI Act — the long-awaited bill from Reps. Obernolte and Trahan, released in “discussion draft” form yesterday — is the best, most serious AI safety bill yet. But the AI safety community is divided over the big question: is it good enough?
At 269 pages, GAAIA contains plenty of excellent stuff. It would formally authorize the Center for AI Standards and Innovation with a $100m annual budget. It would adopt a transparency framework similar to SB 53 and the RAISE Act. And it would establish a range of initiatives to try to improve our understanding of how AI impacts the labor market.
Its best section is on “independent verification organizations” — a fancy word for third-party audits. Here, GAAIA is stronger than even Illinois’ recently-passed SB 315: it tasks CAISI with establishing a licensing regime for IVOs, and requires frontier AI developers to retain an IVO for regular audits. Those audits assess not just whether the company is following its own safety framework, as in SB 315, but also whether what the company is doing is adequate for tackling catastrophic risks — and if not, what the IVO thinks the company should be doing differently.
As I understand it, the current text doesn’t have teeth to force a company to follow the IVO’s recommendations, but a Trahan aide told me that the bill they’ll introduce will require companies to do exactly what the IVO deems necessary to reduce catastrophic risks. If true, that would create an extremely strong bill — fulfilling many AI safety advocates’ biggest wishes.
But every silver lining has a cloud. And in this case, the cost is preemption — the most controversial part of the bill by far. GAAIA calls for three-year preemption of all state bills concerned with the “development” of AI models, though it still allows for bills concerned with “deployment.” The authors have pitched this as narrowly targeted to frontier-safety, but many disagree: as it stands, the clause could potentially preempt some bills concerning child safety, without providing an adequate federal replacement.
Even if the scope of preemption is narrowed (unlikely, given some senior Republicans already consider it too narrow), it would preclude states from ratcheting up frontier safety bills — forcing any future interventions to happen at the federal level, which has thus far yielded no wins.
Some think that’s a worthwhile trade. As Anton Leicht argues, implementing this bill would begin the process of building up the federal infrastructure that is desperately needed to adequately govern AI. The alternative — hoping that a Democrat-controlled Congress will be able to pass something better — comes at the expense of time. Given the immense urgency of regulating AI, that could be a very expensive price to pay.
In practice, the debate might be academic. AI safety advocates are not the ones who will decide if this bill lives or dies. House Democrats, reluctant to hand Republicans a win before the midterms, have signaled they will strongly oppose GAAIA; House GOP leadership is reportedly skeptical too. Even if GAAIA is a deal worth making, Washington’s dysfunction seems likely to prevent it from going anywhere.
— Shakeel Hashim
THIS WEEK ON TRANSFORMER
Trump’s AI executive order was inevitable — Shakeel Hashim on why opposition to all regulation was always doomed
Do voters care about existential AI risks? One Senate candidate thinks so — Veronica Irwin profiles Mallory McMorrow and her unusually detailed AI agenda
THE DISCOURSE
Policy wonks reacted to Trump signing an AI executive order:
Dave Kasten thinks the new 30-day review period is a “reasonable compromise,” but:
“The bigger question is what transparency the government gets into internal deployments of powerful models: remember, we now know Mythos was released internal to Anthropic about a month before the US government knew of it.”
Dean Ball is “fairly confident this is a mistake”:
“This is a fairly major win for the safety contingent within the Admin, and a significant loss for the Sacks/accelerationist wing, and is surprising to me.”
“I continue to think this EO is a mistake. This is clearly teeing up the infrastructure for a model licensing regime, and the fact that the administration is classifying the details of how this ‘voluntary’ system will work is egregious.”
Zak Kukoff tweeted:
“I suspect one day we’ll look back at this admin as the last to have both safetyists and accelerationists under one roof — the same way we view Nixon as perhaps the last Republican to house both liberals and conservatives in his administration.”
Anton Leicht and Dean Ball make the case for betting on human agency as AI disrupts the labor market:
“The role of humans in future economies is not something we simply discover as it occurs. How we distribute tasks between humans and machines is largely downstream of a web of complicated economic incentives and technical features…and when policy makers ask ‘what will happen,’ they fail to see that they’re among the central live players in this question.”
“The attitude we suggest you take on this issue is uncomfortable … it asks you to bet on humans to figure out what to do, but not to idly sit back and watch it play out. Instead, we ask governments to take their thumb off the scale wherever they currently hinder human experimentation, and build the capacity to remain watchful enough to steer the trajectory back on track if we must.”
Kevin Roose overheard this question inside an AI company:
“How are you spending the last 300 days of work?”
roon tweeted:
“the frontier labs don’t have ‘comms problems’. reality right now has a comms problem. what is happening is a little scary and there’s no nice words anyone could say, especially not those profiting from it, that’ll make it feel that much better”
“you dont have to believe in existential risk or job loss for this to be scary: ai is real, you can replicate human thoughts in machines. it is redefining what it means to be human. even if they are strictly corrigible tools that do what we ask, this can be traumatic”
L. M. Sacasas argued that while AI can feel demoralizing, it doesn’t have to:
“We’ve made machines that can fly faster and farther than the swallow-tailed kite, but in no way does it follow that the kite should cease from its flight or that it is somehow diminished because of the advent of flying machines.”
“Why should I cease from inhabiting the playground of language because a machine can pretend to play in it as well? Why should I abandon the exercise of judgment or the pursuit of knowledge? … Do not resign from life. Let us do what it is ours to do because it is good for us to do it.”
POLICY
President Trump signed an AI executive order, creating a voluntary regime for government testing of frontier models and early-access to models with advanced cyber capabilities.
The signing was seen as a victory for Chief of Staff Susie Wiles, Treasury Secretary Scott Bessent and National Cyber Director Sean Cairncross, who had revived the EO after it was pulled by Trump at the last minute last week.
David Sacks, who had led attempts to quash the EO, tried to spin this one as a win.
OpenAI confirmed it will comply with the EO.
The NSA is reportedly using Mythos to conduct offensive cyber operations, possibly with the help of several Anthropic staff embedded at the agency.
It is the latest sign that the Trump administration’s fight with the company is easing.
Senior Trump administration officials have reportedly discussed the government acquiring equity stakes in major AI companies.
Sam Altman has reportedly pitched Trump on the idea of OpenAI voluntarily ceding shares to the government.
Earlier this week, Sen. Bernie Sanders announced legislation to create a sovereign wealth fund by taxing major AI companies 50% of their stock.
He follows other proposals to tax AI companies in various ways from Sen. Elizabeth Warren, who is pursuing a data center tax, Rep. Greg Casar, who proposed a token tax, and Sen. Ron Wyden’s tech company levy for worker displacement programs.
OpenAI’s Joshua Achiam claimed that the public already owns about 26% of OpenAI through the OpenAI Foundation. (He got a lot of pushback.)
The Bureau of Industry and Security issued guidance to “clarify” that licenses are required for advanced AI chip exports to China-headquartered firms outside China.
Sen. Elizabeth Warren pressed Nvidia on export control compliance after Supermicro’s co-founder was indicted for allegedly smuggling Nvidia chips to China.
At least seven Chinese universities with military ties, including two blacklisted by the US Commerce Department, are reportedly seeking access to Nvidia’s H200 chips through third-party brokers and compute leases.
Sen. Elissa Slotkin introduced legislation to bar the Defense Department from using AI to spy on Americans or launch nuclear weapons, with the aim of incorporating the bill into the 2027 National Defense Authorization Act.
Sens. Coons and Reed plan to introduce a similar bill next week.
The House Energy and Commerce Committee thinks China is fueling data center opposition in the US, and asked the White House to look into it more.
The New York state legislature passed a bill that would establish a one-year moratorium on data centers, co-sponsored by NY-12 candidate Alex Bores.
Illinois Gov. JB Pritzker suspended tax breaks for data centers.
Florida Attorney General James Uthmeier sued OpenAI and Sam Altman, alleging the company has knowingly caused harm in its “insatiable quest to win the AI arms race and amass large fortunes.”
Florida GOP gubernatorial frontrunner Byron Donalds said he disagrees with Trump about federal preemption of state AI laws.
Scott Wiener finished first in his primary to represent San Francisco in the House, a seat currently held by former Speaker Nancy Pelosi.
The election now goes to a runoff with city supervisor Connie Chan.
The EU unveiled a tech sovereignty plan requiring governments to store critical data on EU-owned cloud services and tripling data center capacity, aiming to reduce dependence on US and Asian tech companies.
Separately, the European Commission also appointed a 60-member Scientific Panel and a 174-member Advisory Forum to support enforcement of the EU AI Act.
The panel includes many names familiar to Transformer readers, including Yoshua Bengio, Miles Brundage and Markus Anderljung.
China expanded trade secret protections to include data and algorithms, targeting tech leaks.
Canadian Prime Minister Mark Carney unveiled an AI strategy projecting 250,000 new jobs by 2031 and a 3% GDP boost, including a C$500m fund for homegrown AI firms.
Argentine President Javier Milei proposed legislation creating “non-human corporations” operated by AI agents with limited liability protections.
INFLUENCE
Sam Altman stopped in DC to meet Trump admin officials, Speaker Mike Johnson, Minority Leader Hakeem Jeffries, and Sen. Bernie Sanders, among others.
The meetings came as OpenAI released a frontier AI policy blueprint.
It calls for CAISI to perform a mandatory evaluation process of “the most capable frontier models,” but says it should not be allowed to “approve or block deployments.”
Sam Altman, Dario Amodei and Demis Hassabis signed a letter urging Congress to require screening of synthetic DNA/RNA orders to prevent AI-enabled bioweapons.
OpenAI also released a biodefense action plan this week.
Lots of AI safety groups came out against the Obernolte/Trahan bill, including Americans for Responsible Innovation and the Alliance for Secure AI.
Accelerationist super PAC Leading the Future and pro-safety super PAC Public First both plan to spend on behalf of Rep. Kevin Hern in Oklahoma.
Leading the Future was also caught operating sockpuppet Twitter accounts, including a fake anti-AI activist that put out violent rhetoric.
Leading the Future’s 501(c)(4) group fessed up to it, saying they were “parody meme accounts run by an outside vendor.”
Earlier in the week, OpenAI distanced itself from Leading the Future, saying it “does not direct the activities of LTF, or have visibility into their operations.”
It also said that “groups that are advocating on AI should … not use tactics like astroturfing.”
Sam Altman, meanwhile, said he would “love to see money out of politics in general,” but when pressed on Greg Brockman’s donations to LTF he blamed the need to “fight back” against OpenAI’s competitors.
(Notably, Leading the Future was established before the Anthropic-backed Public First network).
Ten Trump administration officials may hold up to $43.8m in SpaceX or xAI stock ahead of SpaceX’s IPO.
Americans for Responsible Innovation, the AI Policy Network and the Alliance for Secure AI urged congressional leaders to add a “human in the loop” provision for autonomous weapons to the NDAA.
The robotics lobby also pressed for NDAA provisions to reduce Pentagon reliance on Chinese robotics and restrict purchases of humanoid robots.
Labor unions pushed back against state-level data center restrictions, helping sink regulatory bills in Illinois, Colorado, Maine and Pennsylvania.
GLAAD CEO Sarah Kate Ellis warned that AI trained on biased data reinforces harmful stereotypes and spreads misinformation about LGBTQ+ people.
Americans showed the lowest support (26%) for AI data center construction among 15 large economies.
The Pentagon is reportedly operating an AI-generated propaganda site targeting Latin America with pro-US military content.
INDUSTRY
Anthropic
Anthropic confidentially filed for an IPO, but didn’t provide details about the timing or size.
With its $900b valuation, going public will create an unfathomable amount of employee wealth as early as this fall.
It’s formalizing its Claude Partner Network, which helps third-party providers implement Anthropic’s enterprise tools, to show “business-readiness” before going public.
It called for the world to have “the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.”
It said this was due to the possibility that recursive self-improvement will “come sooner than most institutions are prepared for,” though it conceded that “we are not there yet, and recursive self-improvement is not inevitable.”
According to internal data, Claude now writes 80%+ of Anthropic’s codebase.
It launched a team dedicated to “AI and the rule of law.”
It expanded Project Glasswing to about 150 new organizations across over 15 countries.
OpenAI
Sam Altman will reportedly attend the G7 summit alongside heads of state, after Emmanuel Macron extended an invite.
The OpenAI Foundation announced its AI Resilience program.
It will grant over $130m to organizations focused on four areas: pandemic preparedness and biosecurity, cyber-resilience, making models safer, and AI’s impact on young people.
ChatGPT officially hit over 1b monthly active users faster than any other app (even TikTok).
OpenAI started building a new data center in Michigan.
It committed to protect locals from electricity price hikes, create union construction jobs, fund the refurbishment of a local rec center, and give free Codex credits to college students.
It launched new Codex plugins for enterprise tasks.
It upgraded ChatGPT’s “dreaming” feature, which synthesizes memories across many conversations.
OpenAI’s second highest token user is reportedly burning through 100b tokens per month … somehow.
SpaceX
SpaceX is seeking $75b in its IPO with its existing share price suggesting a market cap of nearly $1.77t, making it bigger than the vast majority of existing companies in the S&P 500.
Not everything is going smoothly however: S&P Global effectively ruled out a fast-tracked entry for the company to the S&P 500, which would have put SpaceX’s stock straight into the holdings of the biggest institutional investors including the world’s pension funds.
The IPO might get a boost, however, as Fidelity is making it easier for smaller (and less sophisticated) investors to buy into the IPO, dropping its normal requirements for them to hold accounts of up to $500,000 down to $2,000.
The Verge’s Elizabeth Lopatto called the IPO “financial nihilism’s final form,” noting that SpaceX’s total addressable market was listed as greater than the GDP of the US.
The Commissioners Court of Grimes County (yes, Grimes County) gave SpaceX a property tax exemption for its planned Terafab semiconductor facility, ignoring the pissed-off rural community’s demands.
Microsoft
Microsoft unveiled a new wave of AI initiatives at its annual Build conference.
Since “the drama-filled marriage [between Microsoft and OpenAI] slowly devolved into a situationship,” Hayden Field wrote for The Verge, “this year’s Build had the vibe of a freshly single divorcée posting a thirst trap on Instagram.”
It launched seven new AI models, including MAI-Thinking-1, its first advanced reasoning model.
It’s behind the frontier, but it’s cheaper on some tasks (and reportedly trained entirely with Microsoft’s own IP).
It announced Scout, an OpenClaw-like agent that works like an office assistant in Teams, among other tools for AI agents.
It’s also partnering with Mayo Clinic to build a narrow AI model for healthcare, specifically trained on medical data.
Nvidia
Nvidia’s new Vera Rubin platform will go into full production later this summer.
Anthropic, OpenAI and SpaceX will reportedly be some of the first Vera CPU users.
It announced Cosmos 3, a world model built to simulate physical actions, and the blueprint for a hulking 6-foot humanoid robot.
It acquired Kumo AI, a startup that makes predictive AI models, for over $400m.
Meta
Meta is still stalling the Muse Spark API release.
But it did release an enterprise AI agent, aptly named “Meta Business Agent.”
It’s reportedly adding more privacy protections and exemptions to its widely-despised employee tracking tool.
Meta’s AI support assistant readily helped hackers break into high-profile Instagram accounts, 404 Media reported.
Alphabet is raising $85b to fund data center construction, despite existing construction projects falling behind schedule.
Google committed to better “water stewardship,” including funding $500m in water utilities upkeep and reporting water use transparently.
Employees are reportedly making tons of anti-AI memes about the company’s tools.
Others
DeepSeek is reportedly raising $7.4b in its first funding round.
Lila Sciences is reportedly raising $2b at an $8.5b valuation to build AI for autonomous scientific discovery.
Jeff Bezos invested $50m in Flourish, a startup trying to make AI better at continual learning and energy efficiency by emulating the human brain.
SaaS-pocalypse fears are driving big VC bets on robotics and physical AI, the Wall Street Journal reported.
Foxconn reportedly partnered with Intel to build AI equipment for data centers, factories, and robots.
CC Wei, head of TSMC, is worried that their chip supply won’t meet customer demand for “a long time.”
Kevin O’Leary said he will halve the 40,000-acre footprint of his proposed Utah AI data center after lawmakers told him to cut it back by 75%.
It means the project will no longer be the “largest data center in the world.”
Airbnb CEO Brian Chesky is reportedly starting a new AI lab which may focus on user interaction and design.
He will remain in charge of Airbnb and won’t take the CEO role at the new startup.
MOVES
Helen Toner was named Executive Director of CSET, where she’s served as Interim Executive Director since September.
Brian Landsman, ex-Salesforce executive VP, joined OpenAI as VP of global partnerships.
Guy Rosen is leaving Meta, where he served as its chief information security officer.
Ex-Biden admin natsec advisor Anne Neuberger joined Andreessen Horowitz as its first head of global affairs.
Weijie Su, a statistics professor at Wharton, joined OpenAI.
Kirsty Innes is joining Imperial College London to build a new Centre for AI-Driven Innovation.
Brian Christian joined UC Berkeley’s Center for Human-Compatible AI, where he’ll study how AI systems represent and shape human preferences.
Amy Tam joined xAI from Bloomberg Beta.
Erin Woo announced she’s now covering OpenAI (in addition to Google) for The Information.
RESEARCH
MIT’s Mert Demirer and others found that AI-driven coding productivity gains were not filtering down into software releases and adoption of new applications, likely due to bottlenecks in existing structures and marketplaces.
The research found explosive growth at the top of the funnel, with coders creating or editing 3x more files, but just a 30% increase in releases, and no increase in downloads of new apps.
Neo Research, Asia’s first independent AI safety research group, evaluated DeepSeek v4 Pro.
DSv4 Pro’s general capabilities and cybersecurity risk are roughly 3-6 months behind the Western frontier.
Researchers didn’t find much evidence of misbehavior, but verbalized evaluation awareness is rising across DSv4 Pro and other Chinese models.
The Forecasting Research Institute published another round of expert AI forecasts, illustrating how participants’ views evolved between summer 2025 and last month.
It quantifies how impactful people think AI will be in 2040 on a “Technological Richter Scale” (TRS), where Level 5 is “technology of the month,” like a cool new app, and Level 10 is “technology of the epoch,” like the rise of humans.
On average, AI experts and superforecasters nudged their TRS levels up by ~0.2-0.4, suggesting that respondents think AI will be a bigger deal than they’d guessed last year, with the majority predicting it would reach a “technology of the century” level equivalent to electricity.
Perhaps unsurprisingly, the study found that the public views AI much less optimistically than experts.
MIT FutureTech and the University of Queensland got 272 AI experts from 37 countries to identify the most likely harms of AI, and figure out who should be in charge of preventing them.
Their top five causes of harm: dangerous capabilities, AI-enabled weapons and cyberattacks, competitive dynamics, power centralization, and misinformation.
They argue that developers, governments and regulatory bodies are most responsible for addressing these risks.
Harvard Business Review analyzed how people actually use AI in the wild. Two major categories of use:
“Thinkslop,” or letting AI think for us in all kinds of settings (therapy, brainstorming, flirting…)
Work, often secretly. Researchers found that “shadow usage” is common.
University of Toronto researchers demoed an AI-powered “worm” that can autonomously copy itself across computer networks and tailor its attacks to each machine (terrifying).
University of Cambridge researchers said AI entirely designed the key component in a new vaccine that could protect against all coronaviruses, with early trials finding a “modest” immune response.
The team is already working on applying a similar approach to Ebola and flu.
BEST OF THE REST
SemiAnalysis published an incredibly in-depth analysis of whether building data centers in space is totally insane (TLDR; it could be much less insane soon), complete with a model that estimates when they’ll be economically viable.
Consciousness research is trending — Google DeepMind and Meta recently joined Anthropic in hiring philosophers, ethicists, and psychology researchers to think about digital minds and AI welfare.
Facebook is being flooded with AI-generated slop about … not building data centers.
The Atlantic’s Charlie Warzel thinks we’re experiencing an AI-driven “crisis of agency.”
Mathematicians published the Leiden Declaration on AI Mathematics in light of recent headlines about AI-powered math results. In it, they question whether tech companies getting involved in math research risks prioritizing the wrong questions.
Big week for AI in Hollywood: Martin Scorsese partnered with (and publicly endorsed) Black Forest Labs for storyboarding in preproduction for an upcoming film.
Founders Fund has launched a reality TV show in which notable tech figures play a game of Mafia. Episode 1 features Palmer Luckey, Dylan Field … and Sam Altman.
MEME OF THE WEEK
Thanks for reading. Have a great weekend.
GAAIA's independent verification mandate runs into a practical question the bill doesn't answer: who pays for the evaluations? The organizations best positioned to do frontier model testing — METR, Apollo, SaferAI — are overwhelmingly funded by the labs they'd be evaluating, or by philanthropy with deep ties to those labs. If 'independent' just means 'not the company's own team,' this codifies the existing dependency rather than fixing it.
And Joshua Achiam had an interesting response to some of the pushback:
"(..) if your contention is that the public needs to be able to elect representatives to the OpenAI Foundation, well then you're out of luck trying to dunk on me because I agree with that and I want it to happen."
https://x.com/jachiam0/status/2061655108241228087