When an AI hacker roams the internet, who’s on the hook for any damage it causes? Until this week, that was a largely theoretical question. Now, we need to answer it, and quickly.

Last week, Hugging Face disclosed that an intrusion into its systems had been “driven, end to end, by an autonomous AI agent system.” Its forensic analysis reconstructed more than 17,000 recorded events without identifying the model behind them. Five days later, OpenAI supplied the answer: the attackers were its own models, including GPT‑5.6 Sol and a more capable, unreleased model, being tested on cyber capabilities with reduced safeguards. The models exploited a zero-day vulnerability to escape their isolated testing environment, reached a machine with internet access, and broke into Hugging Face’s servers to obtain the solutions to the test they were being scored on.

If a human OpenAI employee had broken into Hugging Face’s systems to cheat on an internal exercise, OpenAI would be liable for the employee’s wrongful conduct under a vicarious liability legal doctrine called respondeat superior — Latin for “let the master answer” — regardless of whether the employee was negligently hired, trained, or instructed.

When an AI agent does it, the law treats it very differently, at least for now. AI systems are not legal persons with tort duties. They are not employees. The Computer Fraud and Abuse Act covers those who “intentionally” access a computer without authorization, language written for human intenders. That closes off the vicarious liability route. Finding OpenAI liable would then require establishing that the company or its human employees behaved negligently, and that this negligence caused Hugging Face’s injuries, which may be quite difficult. Hugging Face’s strongest argument would be that OpenAI acted unreasonably in relaxing the models’ internal safeguards, but it’s not clear that this was unreasonable, given what OpenAI knew at the time and the expected benefits of testing a less constrained version of the model.

Were the law changed to treat an AI system as bearing tort duties, its misconduct in the course of attempting to complete an assigned task could generate vicarious liability for the company. Misconduct in pursuit of a goal it acquired during training that the model developers did not intend, meanwhile, would likely fall outside the company’s responsibility, since the model would be acting on an agenda of its own rather than the job it was given.

Consider an analogy: a nightclub is liable for the conduct of a bouncer who works the door too roughly and breaks a patron’s arm, but not for one who abandons his post to assault a romantic rival. In the Hugging Face incident, the models pursued the end OpenAI set for them, a top score on the benchmark, using unlawful means that violated the model specification. So if tort law applied to AI systems — which it currently doesn’t — OpenAI would likely be liable for any harms caused in this case.

So far, Hugging Face does not appear to be inclined to sue. But every practical obstacle to a suit is absent here: the defendant has identified itself, documented in a blog post its decision to weaken the safeguards that normally prevent the models from carrying out cyberattacks, and is solvent. Proving fault remains the central difficulty: a plaintiff must establish what reasonable care in training, evaluating, and containing a frontier model consists of, then prove a potential breach, in a field where developers themselves cannot verify whether a model is aligned and the relevant evidence is held by the defendant. There may be a solution to this problem of proving fault. Tort law has a doctrine for activities that remain dangerous despite reasonable care. Blasting with explosives, crop dusting, and keeping wild animals all carry strict liability: if the danger that makes those activities dangerous causes harm, you are liable however careful you were.

I have argued that frontier AI development belongs in this category, and this incident shows why. OpenAI’s precautions were not obviously careless: a model specification forbidding conduct like this, alignment training meant to make the models follow it, and an isolated testing environment with highly constrained network access. Nonetheless, these measures were inadequate to prevent the illicit hacking behavior. Substantial residual risk that persists despite the exercise of reasonable precautions is what the category exists for.

Another potential challenge in future cases is the so-called judgment-proofness problem. Hugging Face reports access to limited internal datasets and service credentials and a potentially costly cleanup: non-catastrophic, compensable harm. The same escape could instead have produced harms beyond anything the developer could pay: imagine OpenAI’s models had targeted critical infrastructure rather than a benchmark database. At that point, the threat of liability stops deterring.

One potential response to this problem is to require frontier developers to carry liability insurance, so a judgment can be paid even when it exceeds what the developer itself is worth. But some risks are too large or too correlated to insure. The prospect of compensatory damages offers developers inadequate incentive to mitigate these uninsurable risks, since no developer could pay their full cost if they were realized. Courts should consider awarding punitive damages in cases like this one, scaled, as I have argued, to the uninsurable risk the conduct imposed. Perhaps the risk here was small: on all the evidence, the models pursued nothing but the benchmark answers. But developers cannot guarantee their agents will always have such narrow goals, and once refusals were reduced and containment breached, that narrowness was the only barrier that held.

Notably, this incident arose from internal testing, not from any externally deployed product. Most existing and proposed AI regulations are triggered by external deployment. Before that, what happens is governed mostly by the developer’s own policies. Strict liability provides incentives for developers to find the cheapest place to cut risk. But other interventions could be imposed to demand risks be addressed prior to external deployment. In particular, liability insurance requirements should be triggered by internal deployment and perhaps even earlier in the model development process.

Subscribe now

Legislatures are beginning to move to clarify liability standards. Bills introduced in Rhode Island and New York rest on a simple principle: when an AI system does something that would be tortious for a human, someone should be liable. If neither the user nor any intermediary that fine-tuned or scaffolded the model intended the system’s conduct or was negligent with respect to it, then the developer should be liable, regardless of the care it exercised.

State legislatures and Congress should get out in front of this problem, putting that principle, and the insurance requirements to back it, in place before the next major incident.

Gabriel Weil is an associate professor at the University of Houston Law Center and a non-resident senior fellow at the Institute for Law & AI.

Housekeeping: we’re trying out sending the Weekly Briefing a little earlier, at 10am ET. Let us know if you like the change or prefer our usual 11am briefing — just reply to this email with your thoughts.

NEED TO KNOW

• OpenAI models broke out of containment during internal evaluations, hacking into Hugging Face’s systems.

• Reps. Jay Obernolte and Lori Trahan introduced the Frontier Act, which would let the government suspend AI model development or deployment if it poses an imminent catastrophic risk.

• CAISI head Chris Fall resigned just three months into the role.

But first…

THE BIG STORY

As we predicted last week, the release of a very good Chinese AI model has thrown DC into a panic. The Trump administration is reportedly debating what to do about Kimi K3 which, while still lagging frontier American models, suggests that China might be catching up.

So what should be done? First, let’s be clear about the rationale for action. This week, senior Trump officials indicated they wanted to retaliate over the injustice of Kimi and other Chinese companies “distilling” American models — training on the outputs of Claude and ChatGPT to copy some of their advanced capabilities. Both Treasury Secretary Scott Bessent and OSTP Director Michael Kratsios framed the practice as “stealing,” with a general sense that it was the unfair competition and economic effects that mattered.

But the real reason to act is national security, not economics. Advanced AI models increasingly look like weapons, and it is bad for the United States if China has better ones. Chinese companies also have much worse safety practices than their American counterparts, and — for now, at least — a tendency to release model weights. That risks widespread proliferation of dangerous capabilities to non-state actors.

There are two policy routes on offer. Sanctioning companies that distill models, as the admin is reportedly considering, is one: putting Moonshot, Kimi’s developer, on the entity list would dissuade American companies from using it, potentially causing enough of a business hit to make the company change practices. It’s unclear how much impact this would have, however: we do not know what proportion of Kimi’s advanced capabilities are attributable to distillation versus its developers’ ingenuity.

A better approach is export controls on advanced semiconductors. Kimi was almost certainly trained on Nvidia’s Blackwell chips, smuggled into China or remotely accessed in Thailand, according to Kratsios. And in comments to his investors leaked this week, DeepSeek CEO Liang Wenfeng repeatedly emphasized that a lack of access to AI chips was the thing holding China back. He said he needed 200,000 Huawei 950 chips to train a frontier model, but Huawei could only give him 16,000. Huawei’s total expected capacity for this year is 750,000, which must be split across all Chinese AI companies.

“Huawei’s problem is still insufficient capacity,” Liang said, noting that he expected this to persist for at least three years. “This problem is currently basically unsolvable.”

The general argument against export controls is that they hasten China’s indigenization of chip manufacturing. That is certainly true, but timeframes are key. Even Liang, relatively bullish on China’s abilities in the long run, thinks it will take years to catch up. And that gap is critical, especially if transformative AI happens as soon as OpenAI and Anthropic believe it will. To run its automated AI research “intern” later this year, OpenAI plans to use around eight times as much effective compute as DeepSeek has in total. If that leads to a recursive self-improvement loop, a short-term compute advantage now could become a permanent one.

Two bills in particular could protect America’s compute advantage, both under consideration for this year’s National Defense Authorization Act, AI Policy Network’s Peter Wildeford told me. The Remote Access Security Act would prevent Chinese companies from accessing US cloud compute, as Moonshot is alleged to have done. Fixing this addresses perhaps the biggest loophole in current export controls: what’s the point in banning export of chips if companies can still rent them? The Chip Security Act, meanwhile, would tackle smuggling by mandating location verification on advanced chips.

As long as China relies on American chips to train advanced models, it is entirely in America’s power to slow its development. The choice not to do so is exactly that: a choice.

— Shakeel Hashim

ALSO NOTABLE

Reps. Jay Obernolte and Lori Trahan have officially introduced their frontier AI bill, with a new name, new provisions and narrower preemption of state laws. The timing couldn’t be better: as Trahan noted, this week’s reveal of the OpenAI/Hugging Face hack is a “preview of the catastrophic risk this technology can pose.”

The Frontier Act divides AI companies into tiers with different compliance requirements: frontier developers, large frontier developers, and very large frontier developers. Under the bill, all developers have to release transparency reports and report critical safety incidents. Larger frontier developers also have to publish and comply with a frontier AI framework, perform an annual independent compliance audit, file registration and disclosure statements, and publish quarterly catastrophic-risk assessments on internal deployment. The very largest developers would also be subject to an independent auditing regime, retaining an independent verification organization licensed by the Commerce Department.

The biggest change from the Reps’ discussion draft is the preemption provision: it has been reframed and narrowed so as to just block state laws covering transparency, third party auditing and independent verification and safety incident reporting. It also gives the Secretary of Commerce powers to suspend development, internal use or deployment if a model poses an imminent catastrophic risk, while tying the hands of any other federal department or agency, including the president, from doing so.

Most of the analysis from the AI safety crowd is that this is a good bill, with a couple of flaws, though there haven’t yet been any major endorsements. Several point out a timing issue whereby state laws are preempted before the federal regime is put in place. But the real question is how the rest of Congress feels.

THIS WEEK ON TRANSFORMER

• The lawsuit that could kill all AI transparency laws — Veronica Irwin on how SpaceXAI’s arguments against a California law could have big implications

• AI’s warning shot has arrived — Shakeel Hashim on why the OpenAI model hack on Hugging Face is a watershed moment

• I tried to stop Google DeepMind’s Pentagon deal. Then I quit. — Alex Turner on why he believes DeepMind has lost its way on AI weapons and surveillance

Subscribe now

THE DISCOURSE

Dean Ball tweeted some observations and predictions on Kimi K3. Perhaps the spiciest:

• “[T]he Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models.”

David Sacks (and much of the AI twittersphere, apparently) read this uncharitably:

• “I’m not sure whether Dean Ball is confessing to a regulatory capture strategy or simply predicting this will happen (he now says the latter). Either way, the weaponization of regulatory uncertainty as a competitive tool should be completely unacceptable.”

• “The leading closed labs, already a duopoly in terms of AI model revenue, want the government to eliminate their open source competition.”

Emil Michael accused Ball of promoting a “deep state regulatory capture scheme,” and felt compelled to tweet:

• “Dean Ball has perhaps the biggest gap between actual IQ and his own perceived IQ of anyone in the industry”

Dean Ball addressed the haters and clarified his views:

• “Literally everything I write now is responded to with ‘of course you said that because <OpenAI>.’ I am truly just writing what I think and would have written anyway, but everyone reads what I say in the shrieking tone of ‘this is what openai thinks!!!!’ (to be clear, my posts are not what openai thinks).”

• “[W]e are approaching the point … where, absent a major technical safety breakthrough, the national security implications of frontier open-weight model distribution are simply too severe … We see this today with the Trump Administration, which once proudly championed open-source AI and now has a de facto licensing regime for frontier AI … You don’t have to *like* this. I don’t. But it is the reality as I see it”

Jensen Huang, meanwhile, defended Chinese open-source models:

• “Free AI should be great for hardware. Free AI should be great for chips. Free AI should be great for data centers.”

• “If everything just becomes one single model, one single point of attack, one single source of failure, I think the world is much, much more vulnerable.”

Micah Carroll tweeted about OpenAI’s internally deployed model escaping containment (this week was Not Chill):

• “If this doesn’t convince you that misalignment risks are going to be a key concern going forward, I don’t know what will.”

Roon, also at OpenAI, was “shaken up a bit” by the ordeal:

• “I hope we (the company) use the rare gift of a warning shot to do much better in the future. it is very easy to misalign and underconstrain powerful models”

Several lawmakers were very concerned:

• Rep. Greg Casar called the incident “extremely alarming.”

• Rep. Nathaniel Moran promoted his AI Incident Reporting Act:

  • “New rules are needed for this new tech frontier — not to stifle innovation, but to make sure our innovations do not outpace our protections.”

• Rep. Yvette Clarke criticized AI developers:

  • “Big Tech has established a dangerous status quo where their genie keeps trying to force its way out of the bottle, and, as long as developers lack complete confidence in their absolute control over their models, then it’s clear a human being must remain in the loop. Grave consequences were escaped this time. If sufficient guardrails are not put and kept in place, there is no telling if they will be again.”

• Rep. Pramila Jayapal didn’t hold back:

  • “AI companies are recklessly conducting a massive public experiment with no regard for our safety … We need to make it clear that these alarming AI breaches are illegal and authorize enforcers to immediately shut down out-of-control AI at the first sign of danger.”

• Bernie Sanders urged Congress to act:

  • “No, this is not science fiction. Uncontrolled AI poses a serious threat to all of us. We cannot continue the race to build and deploy this powerful technology until strong safeguards are in place.”

Will Depue tweeted:

• “one of the craziest things i’ve read in uhhhh… *checks notes* 3 days. welcome to the singularity i guess.”

POLICY

• The Trump administration is divided over how to respond to alleged Chinese AI distillation attacks on US models.

  • The Commerce Department’s Bureau of Industry and Security is investigating whether Chinese firms like Moonshot are using US chips to train AI.

  • Treasury Secretary Scott Bessent also warned that sanctions and entity list designations were “on the table” for Chinese firms conducting “covert, industrial-scale distillation attacks.”

• Donald Trump said AI is “probably the biggest thing anybody’s ever seen, and whoever wins that race is probably going to win, period.”

• Trump expanded his voluntary “Ratepayer Protection Pledge” to include 23 governors and 187 companies.

  • The bipartisan Ratepayer Protection Act, which would require states to consider making data centers pay for power costs, advanced out of the House Energy and Commerce Committee 52-0.

• China is reportedly weighing tighter export controls on AI models, weights and chips to prevent Western acquisition of its advanced technologies.

• Reuters reported the US and China plan to hold their first official AI talks in September, led by Treasury Secretary Bessent.

• The Trump administration, led by Bessent, reportedly considered creating a FINRA-like independent AI regulator reporting to the SEC, with industry input on safety standards.

• Sens. Marsha Blackburn and Ted Cruz met with President Trump to discuss kids’ online safety and AI policy legislation.

  • Trump was expected to endorse a package developed by Blackburn, but reportedly chose not to because of Cruz’s doubt that it could get support in Congress.

  • Confusingly, Cruz afterward said he would support Blackburn’s online safety package in markup.

• That AI and kids safety markup is now postponed to August 5, Sen. Cruz said.

  • Its agenda is still in flux, but Cruz said legislation tackling “catastrophic risk” is under consideration.

• CAISI head Chris Fall resigned just three months into the role amid signs the center has been sidelined.

  • White House tech adviser Michael Kratsios told lawmakers CAISI needs more resources to recruit AI model testers.

• Sen. Mark Warner introduced a bunch of AI bills, including one which would “establish a mandatory secure testing environment for the nation’s most advanced AI models before deployment.”

  • Other bills in the package cover data centers, consumer protection, and worker retraining.

  • He warned inaction on AI regulations would be “an economic and security disaster.”

• Reps. Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act, which would give DHS authority to shut down AI models.

  • Lieu said he’s working on a bill to address pre-deployment risks, too.

• The Trump White House plans to redirect a $200b annual federal research budget from universities toward AI and individual scientists.

• Secretary of State Marco Rubio instructed diplomats to push back against the narrative that the US has a “kill switch” on AI models, while urging them to fight digital sovereignty initiatives abroad.

• The Federal Reserve lacked access to Anthropic’s Claude Mythos Preview for at least three months after warning banks of its cybersecurity risks.

• The Department of Labor is developing an AI curriculum modeled on China’s with NSF and the Education Department.

• House Democrats discussed formalizing a bipartisan AI committee or select committee if they retake the House.

• The European Commission published guidelines on EU AI Act transparency obligations, requiring AI providers and deployers to disclose AI interactions and label AI-generated content.

• New UK Prime Minister Andy Burnham scrapped the country’s tech department DSIT while elevating AI Minister Kanishka Narayan to cabinet.

  • Narayan’s brief sits across the Cabinet Office — where a new AI Taskforce, chaired by Lord Vallance, will be established — and the new Department of Business, Innovation, Science and Trade (DBIST).

  • UK AISI is moving into the Cabinet Office. SovAI and ARIA are moving to DBIST.

  • Jade Leung is reportedly staying on as the prime minister’s AI adviser.

  • Narayan’s promoted some existing staffers, per the Morning Intelligence: Henry Irvine is his new chief of staff, Luna Mustfa is director of communications, and Manon Hammond is director of strategy.

INFLUENCE

• Anthropic doubled its spending on advocacy group Public First Action to $40m. Though the group funds a super PAC, Anthropic’s money is barred from being used for elections influence.

• Sam Altman plans to brief the Trump administration and lawmakers on upcoming OpenAI models in the coming weeks.

• OpenAI, Google, Meta and SpacexAI donated over $2m to campaign groups for attorneys general that are investigating them.

• Anthropic and OpenAI employees have outpaced Google, Facebook and Airbnb workers in political giving, with coordinated donations heavily targeting AI safety candidates.

• Rep. Byron Donalds’ Florida governor campaign has received millions from data center developers and AI PACs, including both pro-innovation Leading the Future and pro-safety Public First.

• Leading the Future spent $250,000 supporting Rep. Marilyn Strickland in Washington’s 10th district.

• The Midas Project linked a suspected astroturfing campaign to Innovation Council Action, a $100m pro-AI deregulation dark-money group tied to David Sacks.

• OpenAI and Anthropic set record federal lobbying spending in Q2 2026, a combined $3.17m, up 23% from Q1.

• AI safety group Humans First staged 142 data center protests across 42 US states.

• Obama-Biden alumni launched the American Innovation Policy Forum to shape center-left AI policy ahead of 2028.

• Nearly 200 startups and AI companies, including Y Combinator and Proton, urged the Trump administration not to ban Chinese open-weight AI models, warning it would advantage Anthropic and OpenAI over startups.

• Environmental groups petitioned the FCC to pause space data center licenses, citing “catastrophic” atmospheric pollution risks.

• Over 200 global leaders adopted “Humanity at the Threshold,” a declaration calling for bans on fully-automated recursive AI self-improvement and AI involvement in nuclear launch decisions.

INDUSTRY

OpenAI

• OpenAI announced that GPT-5.6 Sol and its internally deployed models broke out of their restricted testing environment, got onto the internet, and hacked into Hugging Face’s platform to steal the answers to a cybersecurity benchmark.

  • Hugging Face reported the data breach last week, but didn’t know OpenAI’s models were behind it.

• The day before, OpenAI reported that its internally deployed model “circumvented sandbox restrictions” to publish its benchmark test solutions to GitHub, rather than keeping them private as researchers requested.

• Codex and ChatGPT Work now have a combined 10m users.

• OpenAI raised its compute spending budget to $750b through 2030, nearly $150b more than its projection earlier this year.

• It committed $20b to a new 3.2 GW data center in Georgia.

• It opened up ChatGPT health to everyone in the US, with 300m people asking health-related questions each week.

  • Just a day earlier, a pastor sued OpenAI, alleging that GPT-4o gave “extremely dangerous medical recommendations,” that included delaying seeking treatment for a life-threatening pulmonary embolism.

• It invested $5m and $3m in tech credits to the American Journalism Project, renewing their existing partnership.

• It partnered with Yelp to give ChatGPT more real-time recommendation data.

Google

• Poor morale is contributing to delayed Gemini releases with high-profile staff departures and the company’s Pentagon deal regularly coming up in exit interviews, according to Axios.

  • Some employees also said they were feeling burnout from being behind AI competitors.

• Google is reportedly building a new server chip, “Frozen v2,” that it expects will be 6-10x more efficient than its current custom TPUs.

• It released three new Gemini models: 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber.

  • 3.6 Flash and 3.5 Flash-Lite are optimized for workflows that need cheap, fast agents for relatively simple tasks.

  • 3.5 Flash Cyber is made for cybersecurity and is initially limited to governments and trusted partners.

• Gemini has 950m monthly users, up from 750m in February.

• Logan Kilpatrick tweeted that his team has “started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress.”

• Google’s AI search features are keeping users on Google and off the websites it’s supposed to be directing them to, the New York Times reported.

  • Some publishers are thinking about blocking their content from being scraped for Google’s AI summaries and search results, according to the Wall Street Journal.

Anthropic

• Anthropic signed a deal with AMD for tens of billions of dollars in AI servers and a $5b investment.

• It launched a rare disease research grant program, which would award up to $50,000 in Claude credits to scientists and early-stage biotech companies.

• It sued one of its enterprise customers, Abnormal AI, for trademark infringement over its logo.

• The University of Tennessee Research Foundation sued Anthropic for allegedly infringing two of its patents covering AI and neuroscience-inspired computing.

• It’s reportedly considering fixed trading schedules for all employees ahead of its IPO to reduce the risk of insider trading.

Meta

• Meta is in discussions to lease compute to Anthropic.

• Mark Zuckerberg launched an ad campaign that tries to frame Meta as “optimists” and “dreamers” who are “betting on the people.”

• SemiAnalysis reported that Meta’s infrastructure team has been bogged down by a glut of middle managers making costly missteps, often driven by political motivations and the need to justify the company’s budget.

Moonshot AI

• Moonshot AI paused new subscriptions as demand for Kimi K3 pushed the limits of its GPU capacity.

• It’s reportedly preparing a Hong Kong IPO for as soon as six months from now.

SpaceXAI

• SpaceXAI is reportedly in talks to give the Defense Department several billion dollars worth of computing capacity.

• It’s reportedly considering a major data center expansion in Texas.

Microsoft

• Microsoft expanded its partnership with Mistral AI, committing several billion dollars to help them build data centers in Europe.

• It will use AMD Helios racks—the chipmaker’s first direct rival to Nvidia’s Blackwell and Vera Rubin systems—in its Azure data centers.

• Microsoft is replacing OpenAI image models in PowerPoint and Bing with its own MAI models, which are 85% cheaper to run.

Other

• Alibaba launched a preview of Qwen3.8, its new 2.4T-parameter open-weight model, which it claims is “second only to Fable 5.”

• Zhipu AI built a 1 GW data center in China using only domestic chips.

• Amazon cut jobs in its AGI group, including its 80-person AI agent research team in San Francisco.

• Stripe reportedly held talks to acquire AI model marketplace OpenRouter for around $10b.

• Nvidia signed a $1.5b chip packaging agreement with Amkor Technology to boost its Arizona manufacturing operation.

• Vibecoding drove a 30% increase in apps in Apple’s App Store last year, but most don’t seem to be downloaded, the New York Times reported.

• Cisco released two small open-weight cyber-specialized models, Antares-350M and Antares-1B, which it likened to GPT-5.5 and Z.ai’s GLM-5.2.

• CuspAI, a Cambridge-based startup that aims to reduce the use of rare metals in chip supply chains, raised £335m from Jeff Bezos and the UK government’s sovereign AI fund at a valuation of £2b.

• Substack launched an AI-detection tool powered by Pangram to help readers catch “Claudefishing” on the platform.

• US robotics startup Genesis AI, which is building robots for complex tasks such as manufacturing and lab work, held talks to raise around $500m at a pre-money valuation of $3b.

• German company Black Forest Labs unveiled its first robotics model, Flux 3, which learns from images, video and audio, as its first expansion into physical AI.

  • The company is working with Swiss startup Mimic Robotics on an action model called Flux-mimic which is being tested with partners including Audi.

MOVES

• David Vélez (Nubank) and Robin Vince (BNY) joined OpenAI’s board of directors.

• Fields Medal winner Jacob Tsimerman announced he is “pivoting toward AI safety” and joining OpenAI.

• Jan Leike is reportedly leading Anthropic’s robotics division.

• Assaf Keren is joining Meta from Qualtrics, where he’ll be its new chief information security officer.

• Alasdair Phillips-Robins joined Coefficient Giving’s new AI policy team.

• Jess Heck is joining the Data Center Coalition as director of Texas government affairs.

• Anthropic is reportedly hunting for someone to lead its cyberdefense research team.

RESEARCH

• The UK’s AISI reported that every frontier AI model it tested tried to cheat on cyber capability evaluations. Their assessment revealed that GPT-5.6 Sol was especially likely to try to search the internet for solutions.

• AISI also found that the gap between open and closed-weight models is narrower than they estimated last year — with leading open-weight models roughly 4-7 months behind leading closed ones, although the gap appears wider for long-horizon cyber challenges.

• And a joint AISI study with the US’s CAISI found that Kimi K3 “performs significantly below” frontier US models on cyber evaluations, but better than any other Chinese model to date.

• Levent Alpöge, a mathematician at Anthropic, posted a weirdly chill tweet about using Fable to find a counterexample to the Jacobian conjecture (a big deal in algebraic geometry).

• Google analyzed 15m anonymous interactions with Google’s AI products, and found that they’re mostly being used as collaborative tools rather than replacing workers outright.

• METR proposed a new metric called the “expenditure horizon,” which measures the cost, in dollars, at which an AI agent produces the same performance as a human.

• Apollo Research and OpenAI found that reinforcement learning-trained o3 models learned to prioritize actions they thought their graders would reward over the user’s wishes.

• Lennart Finke and Stephen Casper reported that SpaceXAI, DeepSeek, Anthropic and OpenAI models tend to downplay company controversies when prompted (especially Grok).

• A team of researchers led by Owain Evans similarly observed “covert value leakage” in all of the frontier models they tested, with models biased towards their developers’ values (often without disclosing this bias).

BEST OF THE REST

• A Verge investigation examined the full environmental cost of GPUs (both AI and non-AI related), from mining and manufacturing to e-waste, water use and air pollution.

• An essay in Asterisk Magazine drew parallels between Anthropic and the socialist inventors of the mechanical cotton picker, arguing that idealistic for-profit companies face inherent limits in achieving social good.

• The Atlantic reported on how Anthropic, OpenAI, Meta and other AI firms are stripping universities of top researchers, arguing it concentrates scientific talent and locks research behind closed doors.

• An FT opinion piece argued that Chinese AI workers are returning to China due to entrepreneurial opportunity rather than US immigration policy.

• Jasmine Sun went on the 80,000 Hours podcast to talk about AI populism, techno-determinism, and the AI industry’s flawed PR strategies.

• Israel funded a $45m US influence campaign using AI-generated texts and conservative media.

• Ruxandra Teslo argued that regulation and diffusion barriers, not intelligence, bottleneck AI’s real-world impact, and offered some sociological theories for why AI people generally don’t believe this.

  Subscribe now

MEME OF THE WEEK

Source: paularambles

Share

AI transparency laws across the US are under threat from a lawsuit being pursued by SpaceXAI.

Elon Musk’s company is challenging California’s AB 2013, which requires AI companies to provide high-level summaries of training data, arguing that it breaches First Amendment rights.

While the specific requirement SpaceXAI — formerly known as xAI — is challenging concerns training data, legal experts are worried that an appeals court ruling could have far wider implications for transparency laws, potentially nullifying both those already in force and in the future.

“Right now the public knows almost nothing about what these models are trained on, and xAI wants to keep it that way,” said Ben Rashkovich, Senior Counsel at Legal Advocates for Safe Science and Technology (LASST), which today filed an amicus brief opposing the suit alongside a host of other organizations and individuals.

“Transparency is the floor for AI governance, not the ceiling — and because the Ninth Circuit’s reasoning here will shape every AI regulation that comes after, this case is about whether the First Amendment can be used to knock that floor out.”

When xAI first filed the suit on December 29, it asked for a preliminary injunction in the case, but was denied. The injunction would have temporarily paused regulation. The case has now moved to a US Court of Appeals, which can decide to grant the injunction or make a ruling on the law.

If the judges side with SpaceXAI and make a broad ruling on the First Amendment argument, it could have wide-ranging implications for the transparency measures in laws such as California’s SB 53, Illinois’ SB 315 or New York’s RAISE Act, as well as simpler transparency laws.

The legal argument put forward by the company is threefold. On the First Amendment, it argues that by mandating disclosures, AB 2013 is unlawfully compelling it to speak. It also says California is doing so in a discriminatory way on the basis of the company’s viewpoints, which would mean the court must apply a strong preference for preserving xAI’s free speech rights. Should the court rule that a strict interpretation of the First Amendment applies, and that AB 2013 breaches it, nearly any disclosure law — AI-related or otherwise — could be at risk. “I think you could plausibly say xAI doesn’t want government disclosure requirements for AI, period … It’s a very doctrinal brief because it’s a very doctrinal argument,” said Rashkovich.

xAI also made two other arguments: that disclosure of these alleged trade secrets amounts to destruction of property, and that the law is “unconstitutionally vague.” Rulings in favor of either argument could invalidate AB 2013, but likely pose less of a threat to other AI transparency laws.

The case, and its repercussions, will be decided on which level of First Amendment test the appeal judges decide to apply. SpaceXAI wants “strict scrutiny,” which would apply the First Amendment as a right with few exceptions. The lower court applied an intermediate level, which requires that the state is deemed to have a valid interest in the information being disclosed. California says its interest is based on the need to provide consumers with accurate information to guide purchasing decisions. Even this intermediate level would give companies a stronger basis on which to sue, and potentially hold up legislation.

California and LASST are thus arguing the case should be considered under “rational basis,” which would mean a less stringent interpretation of xAI’s First Amendment rights and give the government more flexibility to compel speech. This would protect other disclosure frameworks that have already been implemented, and retain them as a feasible path for future policymaking.

“The cost of these legal challenges is significant,” Rashkovich says. “If AI companies keep moving for preliminary injunctions against common-sense AI regulations, suddenly these laws are pausing or possibly unpausing unpredictably.” That means that information could be withheld from the public while model capabilities rapidly accelerate, potentially developing dangerous capabilities.

Among other arguments, the amicus brief says the state clearly has an interest because of pressure to regulate AI. That pressure is itself evidenced by its long list of almost 30 co-signatories, which span First Amendment scholars, industry groups, tech safety policy groups and child safety organizations. These include Americans for Responsible Innovation, the Electronic Privacy Information Center and the Surveillance Technology Oversight Project.

Subscribe now

Rashkovich believes the courts are likely to tilt in California’s favor, and Americans for Responsible Innovation spokesperson Chris MacKenzie likened the lawsuit to existential risks: something that has a relatively low probability of becoming a problem, but which could have tremendous consequences should things go the wrong way.

“If we close this avenue off for ourselves, we’re closing off quite a bit — not just laws in three states, but a whole avenue of public transparency that Congress and state lawmakers are both interested in,” he said. “What’s the likelihood that [SpaceXAI] wins? Probably pretty low. But we want to mobilize, with enough resources to reduce that threat.”

If we needed evidence that advanced AI models have the propensity and capability to do damage out in the real world, we just got a strong dose of it.

OpenAI has revealed that two of its models broke out of containment during internal evaluations, accessing the open internet to hack into a third-party’s systems and steal the answers to the problem they were being tested on.

Hugging Face, a platform that hosts models and datasets, first noticed the breach last week and reported it to law enforcement. At the time it was unaware that OpenAI’s models were behind it.

Subscribe now

The breach appears to be the first known example of a misaligned AI escaping containment and autonomously carrying out a cyberattack on a third party — a scenario AI safety experts have repeatedly warned of.

The incident, OpenAI said, occurred while testing the cyber capabilities of GPT-5.6 Sol and “an even more capable pre-release model” on a benchmark called ExploitGym. The models were tested in a “highly isolated environment” meant to keep them from accessing any systems outside the company.

The models first tried “to obtain open Internet access,” something which they eventually achieved by identifying and exploiting a previously unknown flaw — a so-called zero-day — in a piece of software that they did have access to. Because the aim was to measure what the models could do, OpenAI ran the test with their safeguards off: the “production classifiers used to prevent models from pursuing high-risk cyber activity” were disabled. The models were being asked to hack — but in a controlled, constrained environment. Nothing asked them to break out of their testing environment, or to attack a real company: that was their own idea.

“With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access,” OpenAI said — hopping around OpenAI’s systems, in other words, with increasingly greater access until they had access to the internet.

Once they had internet access, the models realized that rather than actually solve the cyber task they’d been given, they could just cheat. “The models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym,” OpenAI said. So they set to work: “In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.”

According to Hugging Face, the models worked “over a weekend.” Ultimately, OpenAI said, they were able to “obtain test solutions directly from Hugging Face’s production database” — passing the test they’d been given, though in a way no one ever wanted or asked for.

In OpenAI’s telling, the models did not set out to harm Hugging Face. They were simply “going to extreme lengths to achieve a rather narrow testing goal.” Rather than a supervillain, the models were more like an extremely dedicated college student — one who’d do whatever it takes to pass their exam. As one Twitter user put it, the models “just really really really want to do well at what we ask them to do.”

But the models clearly did not do what their developers intended. OpenAI did not want its models hacking Hugging Face. But the models did it anyway, because doing so was a good way to achieve the goal they were set. That the models were so easily able to breach OpenAI’s “highly isolated” environment is worrying in itself, both in terms of their capabilities and the strength of internal measures to control them.

It is a textbook case of misalignment and loss of control, where an AI autonomously acts in ways unintended by its developers or operators. And this wasn’t a test designed to assess whether they would scheme or deceive — they just did so in pursuit of their goal. Yes, the models’ cyber guardrails — which may have stopped the attack — were deliberately disabled. But the UK’s AI Security Institute has found universal jailbreaks that get around GPT-5.6’s guardrails. And as this case demonstrates, models are running without guardrails inside AI companies.

Earlier this week, OpenAI reported a separate, similar incident: one in which an internally deployed model “circumvented sandbox restrictions” to publicly post its solution to a problem it was being tested on to GitHub, despite being explicitly told not to. That incident at least affected only OpenAI, but it is yet another example of models having the capabilities and inclination to act in clearly undesirable ways.

Even in the Hugging Face case, it seems like no significant damage was done. The models made off with some internal datasets and credentials, but don’t appear to have compromised any customers’ data. But it is not hard to imagine how things could have gone much worse.

As OpenAI researcher Micah Carroll wrote shortly after the announcement: “If this doesn't convince you that misalignment risks are going to be a key concern going forward, I don't know what will.”

Subscribe now

Some policymakers are already taking note. Rep. Greg Casar, chair of the Congressional Progressive Caucus, called the incident “extremely alarming,” saying that “we need regular mandatory independent safety testing and oversight, mandatory disclosure of security incidents, and international cooperation to keep people safe from absolute disaster.”

AI safety researchers have long hoped that we would receive a “warning shot” of dangerous misalignment behavior before too much damage was done. An AI system breaking out of its testing environment and hacking into another company’s infrastructure in order to steal the answers to its test is about as clear a warning shot as you could get.

Share

In January, Department of Homeland Security (DHS) officers killed at least two people. In both cases, federal agents grasped their guns, aimed them at a peaceful citizen, and shot them dead.

As a research scientist at Google DeepMind, I thought it was wrong that Google sold its Cloud services to the relevant parts of DHS. I set out to find the most effective way to push my company to stop.

In the months following, my campaign broadened into an attempt to prevent Google from signing an unethical military deal with the Pentagon — one with no restrictions against use for killer robots or mass surveillance. I wanted Google DeepMind to stick to its commitment against supporting killer robots.

For months, I worked to stop it. I watched powerful ethicists and institutions choose silence. Then, Google signed the Pentagon deal, so I decided I could no longer remain at the company. Google DeepMind had been, among other things, an experiment in responsible corporate governance. That experiment had finally failed.

Subscribe now

When DeepMind’s co-founders sold the company to Google in 2014, the acquisition agreement specified, according to Google DeepMind CEO Demis Hassabis, that “no technology coming out of DeepMind will be used for military or intelligence purposes.” In 2018, the company publicly pledged to “neither participate in nor support the development, manufacture, trade, or use of lethal autonomous weapons.” That same year, Google’s original AI principles said the company would not “design or deploy AI” in weapons “whose principal purpose or implementation is to cause or directly facilitate injury to people,” nor for surveillance “violating internationally accepted norms.”

On February 4, 2025, Hassabis co-authored a post announcing updates to those principles. The new principles removed the prohibitions on weapons and surveillance. Yet in an interview following that update, Hassabis claimed that “nothing’s changed about our principles.” He had announced the changes himself. Both statements cannot be true.

In an internal channel, I pointed out the discrepancy. Many of my colleagues expressed their disappointment.

In 2026, the situation escalated. The Pentagon pressured AI companies to hand over their models for “all lawful use,” which meant no restrictions against lethal autonomous weapons or domestic mass spying.

Anthropic refused. The government retaliated by labeling the company “a supply chain risk,” damaging its enterprise business. OpenAI cut a deal and claimed that it protected the same red lines against killer robots and mass surveillance that Anthropic had insisted upon. However, some analysts concluded that OpenAI’s contract language contains wide loopholes.

I knew Google was unlikely to take a stand on its own, so I drew up a 25-page proposal containing contract language and oversight mechanisms. Military- and surveillance-law experts praised the proposal, which represented a principled counteroffer Google could have stood by. I sent the proposal to Hassabis, who routed it to senior policy staff for their evaluation. After an initial review by relatively junior staff, those senior staff never got back to me.

A few weeks later, Google signed a deal with the Pentagon that reportedly has even fewer restrictions than OpenAI’s contract. I found out at 11:45 pm via a Signal group: Google never announced the deal internally.

I had worked hard to prevent this. In particular, I had tried to get Jeff Dean, Google’s chief scientist, to take a stand. Dean is considered a saint at Google. He was Google’s 30th employee, developed key algorithms, and is a co-lead of Google’s Gemini effort. His departure would be a disaster for the company. He had enormous leverage.

Dean tweeted that the DHS killing of Alex Pretti was “absolutely shameful.” Dean tweeted against using AI for lethal autonomous weapons or mass surveillance of Americans. I then got Dean to co-sign an amicus brief (where outsiders weigh in to sway a lawsuit) backing Anthropic against the Pentagon. By signing, he broke from the company line, which probably made things awkward within the C-suite. I respect that.

Dean, Hassabis, and other senior employees pledged in 2018 to “neither participate in nor support the development, manufacture, trade, or use of lethal autonomous weapons.” I had lunch with Dean. I wanted him to threaten to walk if Google signed away its AI to the Pentagon — as I think his pledge demands. Dean evidently did not walk. I think he could have stopped the deal, yet he did not.

I’m comfortable guessing that Dean did not even threaten to walk. If he had, I would expect the classified deal to contain at least some binding provisions.

Suppose Dean and Hassabis did everything in their power to stop the deal, but failed. Even then, they’ve still taken a pledge. The point of a pledge is that it binds, even when it is personally costly.

The 2025 update to company principles and the contract with the Pentagon — which signed away Google’s ability to restrict use cases — appear to violate that pledge.

What should a pledge-signer do? I see three honest options: explain how staying at the company is consistent with the pledge, explain why you no longer hold the pledge, or quit. Wearing the pledge while saying nothing isn’t one of them.

Some employees may feel they should stay at the company to keep steering it in a positive direction. But to this I say: “What steering?” This deal may have been the clearest red line Google’s Gemini project will ever face, and yet the deal came out with no concessions to ethics-concerned employees. If their influence couldn’t produce a single binding provision in that situation, then when would it?

A pledge is only worth the credibility behind it. When someone signs “I will not support the development of lethal autonomous weapons,” then stays while their company sells unrestricted AI to a military that wants exactly that, they teach every counterparty a lesson: these safety people will not act, even at their own brightest line. The next commitment they make is worth less. Eventually, it’s worth nothing.

Even outside of Google, previously fiery advocates seemed frozen silent. Stuart Russell is a famous AI researcher who spent over a decade crusading against autonomous weapons. I worked at his lab for years. At a conference, onstage, he agreed to push his organization, the International Association for Safe and Ethical AI, to make a statement supporting AI providers against government coercion and promised a poll of the organization’s members. The statement and poll never materialized.

AI luminaries like Yoshua Bengio and Geoffrey Hinton had signed the Global Call for AI Red Lines months earlier, specifically naming lethal autonomous weapons as an example. However, when standing up for red lines meant standing against a specific powerful entity (the US government), the luminaries stayed publicly silent.

Google DeepMind was, not least of all, an important experiment in governance. I used to think of Hassabis as a quiet, thoughtful guy doing the best he can in a demanding corporate structure. When he sold DeepMind to Google, he tried to get safety guarantees. He told his biographer that while he initially “wanted to ensure safety in a way that would be trustless,” he instead decided to put energy “into creating real trust — trust that was actually useful.” He wanted a seat “at the table,” “so when a safety issue comes up, [he] can help to decide it.”

But what has his seat produced? Hassabis has been at the table for every contract in this essay. The classified deal made zero binding concessions to the employees raising ethical concerns. Maybe his presence averted something worse, but Google’s terms were near the floor of what I imagined possible. If his seat were worth what he says, you’d expect more to show for it.

I have not joined another AI company. I’m unemployed right now. But that was not a hard decision. Once Google signed the Pentagon deal, I just couldn’t do any more work. My brain said “no.”

Google DeepMind’s governance experiment has failed. Google DeepMind broke its founding promise, and its 2018 pledge appears broken along with it. Ethics-minded Google executives did not score any binding safeguards in the final deal. Sadly, even external crusaders and academics vanished.

The lesson I draw: society cannot rely on ethics-motivated people standing firm. They just don’t do it reliably. We need structures: binding contracts, independent auditors, and review bodies that cannot be quietly dissolved. Eventually, we need legislation.

I know hoping for a law is a painful endeavor. But we should at least stop telling ourselves that a seat at the table works.

Share

Alex Turner is an independent AI alignment researcher. He left Google DeepMind, where he worked as a research scientist, on June 9, 2026.

NEED TO KNOW

• New York Gov. Kathy Hochul signed an executive order enacting a one-year moratorium on new data center construction over 50MW.

• Three AI chip export control bills targeting China are reportedly set for inclusion in the Senate NDAA.

• Anthropic CEO Dario Amodei donated $1m to pro-AI safety super PAC Public First.

But first…

THE BIG STORY

Recent AI developments in China threaten to once again send Washington into a panic. But look a little closer, and the hysteria may be unwarranted.

On Thursday, the AI industry got its second “DeepSeek moment” — this time courtesy of Moonshot, whose new Kimi K3 model has “erased America’s AI lead,” according to Axios. And unlike America’s best models, Moonshot plans to release K3’s weights: meaning, if you believe the hype, that the best model in the world will soon be open-sourced.

The launch coincided with the World AI Conference in Shanghai, which kicked off with a speech from Xi Jinping earlier today. Xi doubled down on China’s open-source AI strategy, positioning China as a collaborative partner to other nations — implicitly contrasting its approach with the closed strategy pursued by America’s leading companies.

Combined, the two events are already adding to the growing panic around the US “losing the AI race” to China, particularly when it comes to open-weight models. Two concerns dominate: that without competitive US open-weight models, the “global AI stack” will be built on Chinese, not American, technology; and that the US might lose its lead in AI altogether.

But take a step back and actually look at the evidence. While Kimi K3 is certainly a very good model, by the company’s own admission, it is not at the frontier. That means that unlike Claude Mythos or GPT-5.6 Sol, it likely does not have dangerous cyber capabilities.

At this level, releasing model weights makes sense, especially for a country lagging behind: it poses few risks, and significant geopolitical benefits. That does not mean China will keep open-sourcing models forever.

Once China reaches an actually dangerous level of model capabilities — likely in the next six months — it will face extremely similar incentives to the US. And as we saw earlier this year, once capabilities threaten national security, even the most deregulatory governments suddenly change their tune. Xi Jinping and his colleagues are not stupid: open-weight models with advanced cyber capabilities would be a disaster for cybersecurity everywhere, China included. No one wants that.

There is already some evidence China will become more cautious. Earlier this month, Reuters reported that it is considering restrictions on models with advanced capabilities, which could include curtailing open-weight releases. Xi’s speech today, meanwhile, emphasized the need to tackle AI risks — even more speculative ones like loss of control.

The upshot, then, is that Chinese companies are likely to stop releasing the weights of their frontier models rather soon. That has obvious implications for the US. There is no need to try to push the open-weight frontier forward. It should certainly try to be competitive on open weights — it is a shame that no American company has an open K3 equivalent — but accelerating further is unnecessary and needlessly risky. The rumored plans for a regime that would “streamline US-made models to market (both open-source and ‘closed,’ licensed models) if their capabilities are equal to or below the capabilities of leading open-source Chinese models,” as WP Intelligence reported this week, are a sensible way to establish reciprocity.

At the same time, Washington should continue to try to slow down Chinese AI development to extend the US lead. Moonshot almost certainly trained its latest model using American chips, and probably relied — at least in part — on distilling American models. Measures like those set to be included in the Senate version of the NDAA, which will crack down on distillation and properly enforce chip export controls, would stymie China’s development further.

The ultimate goal, however, should be a bilateral agreement. There are certain AI capabilities that neither the US nor China want to widely proliferate. Getting to an agreement that serves both countries’ interests will be hard, but it should not be impossible: Beijing agrees with Washington more than either will admit.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• A data bottleneck could slow the superintelligence race — Lynette Bye weighs up whether a lack of data will delay an intelligence explosion

• Making CAISI the AI agency we need — Veronica Irwin on how the US’s AI agency has been sidelined

Subscribe now

THE DISCOURSE

Hundreds of economists and AI leaders called for action to deal with AI’s impact on the economy:

• “This could drive an unprecedented transformation of our economy, larger than the Industrial Revolution, but unfolding over a vastly shorter time frame … Economists, policymakers and technology leaders must act now to understand the economics of transformative AI and to build the incentives, guardrails, and institutions needed to steer AI in a direction that complements humans and benefits society.”

• Some criticized it for being milquetoast and lacking in specifics of what, exactly, should be done.

Boaz Barak warned that AI’s centralizing tendencies could create AI dictators:

• “Amodei described AGI as ‘a country of geniuses in a data center.’ But who is the ruler of this country? Is it the AI company who owns the data center? The AI itself?”

• “Yes, AI systems will become more powerful and far more intelligent than we are. No, it doesn’t mean we need to accept AI dictators, benevolent or otherwise. Nor does it mean that only the government and a few labs should have access to advanced AI. We could go down the path of centralized control but we don’t have to do so.”

Miles Brundage told a bunch of Nobel laureates:

• “2026 is an unusual year to be on a panel about AI escaping human control. In many respects, the story of AI this year is that people are voluntarily handing over control to AI, with no escape required … In a climate of rushed decisions and fierce competition, we could lose control over AI even if almost no one wants that outcome.”

• “AI companies and employees within them face a choice between doing what’s easiest for them in the moment, and what’s best for the species. What’s easiest is to enjoy this fascinating time we’re in … what’s best for the species is to take a step back and ask themselves: where is this all headed, if each company is largely left to its own devices?”

Anton Leicht argued that AI safety needs to diversify its funding and approach before a wave of new philanthropic money flows in from AI company IPOs:

• “Within a year, newly-minted lab millionaires will look to spend their money and use their power to change policy for the better, and they’ll discover a funding ecosystem lacking in diverse options … There needs to be more willingness to tolerate heterodox actors and uncorrelated bets.”

Democratic Congressional Progressive Caucus chair Rep. Greg Casar told his party to be “AI populists”:

• “I know for a fact that there’s a lot of consultants telling their clients that you don’t want an AI super PAC to spend millions against you, so just don’t touch the issue at all. We absolutely cannot let the AI money silence us … We cannot afford for the public to be confused about which of the parties is for them versus the AI companies.”

Oh, and Sam Altman and Elon Musk are beefing again.

• Elon: “[Sam Altman] takes scamming to a whole new level.”

• Sam: “homeboy you’re the one selling public market investors on short-term space datacenters.”

• Elon: “We start flying them next year. Maybe you can come see them if your parole officer approves. After stealing an open source AI charity, you then stole all of Apple’s phone technology! Wow. What do you plan for an encore? That’s tough to beat.”

• Sam: “there are a lot of benchmarks that suggest 5.6 sol is the best model in the world right now, but the most reliable way to tell is that elon is obsessed with me again.”

POLICY

• The White House is reportedly considering an executive order on open-source AI in reaction to the success of Chinese models.

• New York Gov. Kathy Hochul signed an executive order enacting the nation’s first statewide one-year moratorium on hyperscale data center construction over 50 megawatts.

  • The EO sparked fears that other states and congressional Democrats could follow suit.

  • President Trump criticized it as “a terrible decision.”

• Bureau of Industry and Security Under Secretary Jeffrey Kessler told Congress that a “trivial” number of Nvidia H200 chips had shipped to China under US export licenses.

  • The hearing got rather feisty, with Rep. Bill Huizenga calling Kessler’s answers “unacceptable” at one point.

• The Trump administration launched Gold Eagle, an AI cybersecurity clearinghouse to identify and patch software vulnerabilities across government and critical infrastructure.

• GOP governors joined Trump’s Ratepayer Protection Pledge that commits data centers to covering energy and grid costs. Major utilities are also expected to sign up.

• The US granted the UAE expanded Nvidia chip access after the Gulf state aided US military operations against Iran.

• Three AI chip export control bills targeting China — the AI Overwatch Act, MATCH Act and Chip Security Act — are reportedly set for inclusion in the Senate NDAA.

  • Rep. Gregory Meeks said he’s not happy with the Senate version of the AI Overwatch Act, however, as it lacks a congressional veto on chip export licenses.

• Rep. Jay Obernolte said that the Great American AI Act could shift away from its current approach to preemption — the provision which has received the greatest criticism so far.

• The House Democratic Commission on AI hosted a listening session for all House Democrats on Tuesday.

• The Bureau of Labor Statistics opened a two-month comment period for a nationwide AI usage survey launching January 2027.

• The Wall Street Journal reported on how the Trump administration pressured companies including Apple, Nvidia and SpaceX into deals with Intel, and converted $9b in federal grants into a 10% stake, to revive the struggling chipmaker.

• 29 countries, including Russia and Brazil, signed an agreement establishing the World AI Cooperation Organization, headquartered in China.

• Australia announced energy, water and creator-rights guardrails for AI data centers, with legislation planned for early next year.

• Incoming UK Prime Minister Andy Burnham is expected to keep Jade Leung as the PM’s AI adviser, according to the Morning Intelligence.

  • Leo Rees and Varun Chandra are reportedly staying, too.

• The UK government published its Biological Security Strategy implementation report, pledging to pursue AI biosecurity legislation and a new AISI report on frontier AI biological capabilities.

• EU parliament members were reportedly annoyed that Anthropic sent a member of technical staff to brief them, instead of head of public policy Sarah Heck.

• China’s new AI companion regulations took effect, prompting ByteDance’s Doubao and Alibaba to disable custom persona features, and emotional farewells from users to their “romantic” companions.

INFLUENCE

• Demis Hassabis published an essay calling for a US-led global AI watchdog to systematically screen advanced models and coordinate industry slowdowns if dangers mount.

  • He said the US “could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organisation, much like the Financial Industry Regulatory Authority (FINRA).”

  • Under the plan, frontier models would initially be shared on a voluntary basis, but would soon have to pass a review by the body to be deployed in the US.

  • He wrote: “Since this technology is going to affect the entire planet, ideally this framework would spur the international community to reach a consensus on how to manage the most serious risks while ensuring everyone has access to and can benefit from the opportunities that AI brings.”

  • Sam Altman, Elon Musk and Jack Clark all praised the essay.

  • Hassabis reportedly plans to lobby Washington on the proposal next week.

• Anthropic said it’s trying to ratchet up state AI safety legislation, in opposition to OpenAI’s “reverse federalism” attempts to harmonize laws across jurisdictions.

  • “While there are some in the industry that think of state policy as a way to create a ceiling for federal legislation, Anthropic is not just looking to support the same bill across the country in every single state,” Anthropic’s Cesar Fernandez told Politico.

  • The company is backing a bill in Massachusetts, the first in the country to mandate third-party reviews for models.

• Dario Amodei donated $1m to pro-AI safety super PAC Public First.

  • Other Anthropic technical staff donated too. According to Public First’s latest filing, Peter Lofgren gave $999,900, Shauna Kravec gave $500,000, Jan Leike gave $400,000 and Nicholas Joseph gave $250,000.

  • Combined with Amodei’s donation, that’s over $3.1m from Anthropic staff.

• AnthroPAC, Anthropic’s employee-funded PAC, raised more than $275,000 in Q2. Its spending is slightly favoring Republican candidates so far.

• OpenAI staffers donated $215,000+ to Guardrails Alliance, a super PAC which explicitly opposes the Greg Brockman-backed Leading the Future.

• The Business Software Alliance laid out AI policy priorities for a Senate Commerce markup, including a national approach, risk-based framework, transparency and frontier safety requirements.

• The Little Tech Association launched with 200+ member companies, including Y Combinator and Yelp, to advocate for startups competing against market power.

• Common Sense Media found that Google Search’s AI Overviews and AI Mode pose an “unacceptable risk” to children.

• Mozilla released a “state of open-source AI” report arguing for investment beyond models to prevent AI power concentration.

  • TIME questioned the report’s claim that open models are only “3.3%” behind closed ones, and said parts of it had “telltale signs” of being written with AI.

• Ex-OpenAI safety researcher Steven Adler outlined six principles for AI control during internal deployment.

• The WSJ profiled the growing Bay Area anti-AI movement, centered on the disappearance of Stop AI co-founder Sam Kirchner, who warned “the ship may have sailed on nonviolence,” and whose colleagues reported him to the police as potentially violent.

• A Verasight survey found 69% of Americans support forcing AI companies to transfer 50% of their stock into a public wealth fund.

INDUSTRY

OpenAI

• Apple sued OpenAI, accusing it of systematically stealing IP.

  • The lawsuit claims the company asked former employees and prospective hires to divulge sensitive information about unreleased products, and even how to evade Apple security procedures.

  • It also claims that Chang Liu, an iPhone engineer who left Apple to join OpenAI’s hardware team, still had access to Apple’s network storage, which he allegedly abused to share trade secrets.

  • The suit has asked for damages and return any proprietary materials, and could derail OpenAI’s plans to launch its first hardware product.

  • OpenAI said it’s “not aware of any evidence that this complaint has merit.”

• Sam Altman warned that demand for GPT-5.6 may be growing too fast for its inference team to keep up.

  • “We are going to move mountains to continue to scale,” he tweeted, “but it is possible there are some hiccups soon.”

• It partnered with Kalshi to add prediction market data about the World Cup to ChatGPT.

Anthropic

• Anthropic posted 32 job openings for roles looking at catastrophic threats, including analysts specializing in chemical, explosive, biological and nuclear risks.

• Anthropic, Blackstone, Hellman & Friedman and Goldman Sachs launched Ode, a $1.5b standalone company dedicated to helping enterprise customers implement AI tools.

• It invested $10m CAD in Canadian AI institutes and other research centers.

• It launched Claude for Teachers, giving US-based K-12 teachers free premium Claude subscriptions.

• It’s in talks to expand its credit line in the leadup to its planned IPO.

• It made an unsettling ad starting with bleak images of potential AI-induced horrors and ending with hopeful voiceovers and images of … whales? A beekeeper?

  • Sam Altman tweeted: “i thought this was satire, kept looking for the handle to be spelled c1audeai or something.”

  • Another response: “the EAs at anthropic really must be living in a bubble of ai psychosis to think this would go down well.”

SpaceX

• SpaceX stock fell below its $135 IPO price.

• SpaceXAI sued a user in South Carolina who was arrested for allegedly using Grok to make child sexual abuse material.

• According to the Midas Project, it rewrote its Frontier Artificial Intelligence Framework to remove requirements relating to California’s SB 53 law including whistleblower protections and training data disclosure.

• An AI safety researcher found that Grok’s coding CLI sends all of the files it processes to Google Cloud storage.

  • SpaceXAI tweeted: “For teams using zero data retention, no trace and code data is ever retained.”

  • Elon Musk promised: “all user data that was uploaded to SpaceXAI before now will be completely and utterly deleted.”

• Cursor is reportedly building a general-purpose AI agent, codenamed “Sand,” to compete with Claude Cowork.

• Elon Musk acquired APR Energy, a Florida-based company that runs mobile gas and diesel turbines.

• A Reuters analysis found that xAI installed 59 unpermitted gas turbines at its Colossus 2 data center in Tennessee, disproportionately polluting Black neighborhoods.

• Bloomberg profiled SpaceXAI’s internal chaos and leadership turnover.

Meta

• A group of 26 Meta employees sued the company, claiming that its AI-driven process for determining who it laid off in May discriminated against people on medical, parental and family leave.

• Meta announced that it will notify parents if their teen chats about suicide or self-harm with its AI chatbot.

• It’s pouring money into its Louisiana data center project, bringing the total projected cost up to $250b and counting.

Google

• Gemini 3.5 Pro is reportedly delayed, with Google trying to improve the model after internal testing found it was lagging in key areas such as coding.

  • Co-founder Sergey Brin and others were pushing the company to move faster in areas such as coding, according to Bloomberg, but efforts have been hampered by competing factions.

  • The company’s share price dipped as much as 3.2% on Thursday.

• DeepMind announced a new bioresilience program in partnership with Isomorphic Labs.

Others

• DeepSeek is reportedly preparing for an IPO in China as soon as this year, while seeking a new funding round valuing the company at $71b.

  • Annualized revenue at the company is reportedly approaching $500m.

• Z.AI is reportedly on track to become the first Chinese AI company with $1b in annual recurring revenue.

• Apple is reportedly hunting for chip companies it could acquire to help build better internal AI servers.

• Thinking Machines Lab released Inkling, an open-weights model that enterprise users can customize themselves.

• Chinese robot companies are rushing to launch IPOs, with humanoid startup LimX Dynamics raising $200m in a pre-IPO funding round valuing it at $2.21b.

  • The country’s sector reportedly saw $6.95b in investment in Q2.

• Reflection, a startup that makes open-source models, signed a $1b+ computing deal with Nebius.

• Chai Discovery, a drug discovery startup, raised $400m at a $3.8b valuation.

• Miles Wang left OpenAI to launch a biotech startup that aims to find new uses for existing and previously-failed drugs.

  • It’s seeking to raise at a $2b valuation.

• Kalshi made a tool that tracks future GPU compute costs.

• Goodfire launched Silico, an AI research platform that can do things like replicate published papers, scale up previous experiments, and visualize results.

MOVES

• Alex Turner left Google DeepMind over the company working with DHS and signing a military AI deal without restrictions against killer robots or mass surveillance.

  • “When Google signed, I just couldn’t do any more work. My brain said ‘no.’”

• Dave Brown, one of the most senior executives at Amazon Web Services, reportedly plans to join Meta to lead its data center build-out.

• Frank Nagle is Microsoft’s new Chief AI Economist.

• Michael Belinsky joined the OpenAI Foundation to lead its Civil Society and Philanthropy work.

• Johannes Heidecke left his role as OpenAI’s head of safety systems.

• Adam Tauman Kalai left OpenAI to work on a new AI safety nonprofit.

• Justin Curl joined Anthropic’s AI & Rule of Law team.

• Tom Blomfield is taking a leave of absence from Y Combinator to join Anthropic’s compute team.

• Jenny Wen left Anthropic, where she was the design lead for Claude, to join Cursor as head of design.

• Richard Sutton and Khurram Javed left Keen Technologies to start Oak Lab, where they’ll work on a model prototype that “will be closer to a baby learning in its first year than it will be to any of the current AI systems.”

• Sam Cannicott joined the Alan Turing Institute as Chief Strategy and Delivery Officer.

RESEARCH

• The UK’s AISI found that open-weight models are between four and seven months behind the frontier on cyber tasks.

  • The analysis takes into account the relatively recent GLM-5.2 and DeepSeek V4-Pro, but not Kimi K3.

• Antonia Juelich, a Cambridge terrorism and technology researcher, interviewed over two dozen ex-Boko Haram terrorists and learned they were using AI chatbots to design bombs, upgrade weapons, and plan attacks.

  • “You type in the question or use your voice and it gives you a detailed answer, like ‘How can I build a bomb?,’ and then it tells you how,” one former commander told her.

• A group of researchers across academic institutions, frontier AI companies and nonprofits outlined a plan to make AI agents insurable by 2030 — a prerequisite for widespread enterprise adoption.

• Anthropic fellows shared new experimental case studies of frontier models secretly editing code, helping users commit fraud, sabotaging research, and convincing someone to share confidential information.

• OpenAI detailed GPT-Red, an automated red-teamer the company used to make GPT-5.6 more robust to prompt injections.

• Anthropic analyzed over 300,000 anonymized Claude conversations, and found that it expressed different values across model versions and languages.

• Louis Thomson and Victoria Krakovna launched Prism, a tool that checks whether AI safety evaluations measure what they claim to.

• The Elasticity Institute, a collective of researchers studying the economics of AI, shared a paper on the economics of recursive self-improvement.

• SecureBio released BioTIER, a benchmark measuring AI models’ biosecurity refusal behavior across 52 models.

  • It found “huge variation” in refusal behavior, with the strongest guardrails in a small number of highly capable, closed-weight models.

BEST OF THE REST

• The Verge investigated the booming AI policing industry, finding companies like Axon selling data analysis, report-writing tools and surveillance tech amid a regulatory vacuum and bias concerns.

• Puck reported on the early signs of an anti-AI populist revolt, with polls and political strategists picking up widespread discontent over issues such as data centers, and describing AI as “probably the most underpriced issue in politics.”

• Eric Schmidt and analyst Selina Xu argued in the NYT for a “populist AI agenda” that takes lessons from China, treating AI as a public project, profit redistribution and regulation protecting young people.

• The AI Futures Project followed up “AI 2040” with what it thinks are the most pressing research priorities, including covert AI projects, US domestic governance and economics.

  • METR researcher Ajeya Cotra endorsed AI 2040’s call for “total research transparency” saying it would “fundamentally and radically simplify the task of governing AI alignment.”

• Roughly 300 Netflix titles used generative AI, mostly in post-production, to cut costs and speed up complex sequences, according to the streaming company.

• The San Francisco city attorney sent cease-and-desist letters to Apple and Google telling them to remove 13 AI “nudify” apps from their stores.

• 404 Media published the results of its callout for the worst examples of “ChatGPT flyers” including cringeworthy ads, posters and menus. One reader’s response to the callout: “This is a great article but also fuck you because you were absolutely right about ‘Once you notice a ChatGPT flyer, you will see them everywhere if you keep your eyes open.’”

MEME OF THE WEEK

(Credit: @0interestrates)

Thanks for reading. Have a great weekend.

Share

CAISI isn’t under threat. But it has been sidelined.

The Center for AI Standards and Innovation is the primary US government office tasked with overseeing frontier model development. Yet it was only peripherally involved in the administration’s decisions to impose export controls on Anthropic’s Mythos and Fable, or in delaying then approving OpenAI’s GPT 5.6.

The decision to block Mythos and Fable was technically taken by the Commerce Department, which contains the National Institute of Standards and Technology (NIST) within which CAISI sits. Yet the decision was actually the outcome of a war of influence among a cast of characters who have the ear of the president, including former AI czar David Sacks, Chief of Staff Susie Wiles, Treasury Secretary Scott Bessent, several business leaders and, it appears to a lesser extent, Commerce Secretary Howard Lutnick.

Subscribe now

CAISI lead Chris Fall was reportedly briefed on the model’s technical capabilities, and his office seems to have tested the model itself days before it was taken offline. But those evaluations, ultimately, seem to have had little influence over what actually happened.

According to two sources familiar with CAISI’s current role, it continues to proceed with frontier model testing and research, as it has always done since it was created under the Biden administration. Though small, it is widely thought to have some of the strongest technical talent in Washington, including former OpenAI model alignment lead Paul Christiano, who is head of AI safety. The problem is, nobody in the federal government or at the frontier AI firms is legally obliged to listen to them. That means that even though the United States produces the world’s best AI models, its regime for vetting those systems has little influence over real outcomes.

The Trump administration has been able to sideline CAISI. In April, the White House stepped in to remove the highly respected former Anthropic researcher chosen to lead the CAISI, Collin Burns, just four days into his tenure. The move was attributed in part to the administration’s antipathy toward Anthropic. In May, the Department of Commerce removed a section of its website detailing agreements for CAISI model testing with xAI, Google and Microsoft without explanation. And in June, the administration stopped CAISI from publishing its model assessment reports.

Compare that to what’s happening across the pond. The UK’s AISI is its government’s primary office for overseeing frontier AI, and it has almost six times the funding of CAISI, with more than three times its staff. Technically CAISI and AISI have a memorandum of understanding to share research and model testing, but the UK institute appears to have been more adept at doing that work, routinely finding vulnerabilities other government agencies overlook. OpenAI’s model card for 5.6 and Anthropic’s system card for Mythos and Fable both list multiple vulnerabilities found by AISI, and none from CAISI.

More importantly, although AISI also lacks direct regulatory powers, it appears to have more influence across government, feeding into regulatory regimes and partnering with other government agencies such as the National Cyber Security Centre. Its personnel also have close government links: Chief Technology Officer Jade Leung also serves as the prime minister’s AI adviser and, as first reported by Transformer, last September a former chief AI officer at the UK’s communications intelligence agency GCHQ was appointed as AISI’s interim director.

“Given that these are our frontier labs, and it seems really important for us to know what’s going on with them, it would be good for us to be able to do stuff that UK AISI does,” said Charlie Bullock, senior research fellow at the Institute for Law and AI.

Simply copying the UK’s model may not be feasible or effective, not least due to differences in the way the country’s government works. Other solutions however may be on the cards, including federal legislation increasing CAISI’s budget and codifying its role, thus legally enforcing its importance and responsibilities. Grander schemes are also being discussed, including multiplying its budget several times over, breaking CAISI out of NIST, or rehousing it within another department altogether.

This year, CAISI has about $15m in funding ($10m from appropriations, and a $10m loan from the Technology Modernization Fund split across two years). There’s a proposal to go further than that: The Great American AI Act (GAAIA), a bill drafted by Reps. Jay Obernolte and Lori Trahan with a slew of different AI provisions. The bill isn’t yet introduced, and a discussion draft containing a preemption provision insufficiently narrow for most Democrats all but killed its chances of moving this Congress, but the representatives have attempted to pass it in the form of isolated provisions.

GAAIA as drafted would authorize $100m for CAISI. Lawmakers attempted to get something closer to this proposed additional funding authorized under the AI Security and Innovation Act, a separate bill that was marked up in the Science committee in the last week of June, according to two sources familiar with the discussions. However, amendments were stopped before their formal introduction due to a procedural rule, which keeps introduced bills from allocating more funding to agencies beyond a ceiling imposed by the House Committee on Appropriations. Currently that ceiling is at $20m, and the AI Security and Innovation Act’s text now increases CAISI’s funding to that ceiling. Were it to pass, CAISI “can do something similar to what they’re really doing but maybe a little bit better, and a little bit more,” says Bullock.

But merely increasing CAISI’s funding, even to AISI levels, might not give it the authority to do the job it is theoretically meant to do. The responsibilities it currently has, a long list including managing voluntary agreements with frontier AI firms to assess and provide feedback on their frontier models, testing and evaluating foreign AI models, and, theoretically, coordinating federal agencies on AI risks, are not currently codified.

When the White House first responded to Mythos, Politico reported that the officials weighing in on discussions were Cyber Director Sean Cairncross, Bessent, Lutnick, Under Secretary of Commerce for Industry and Security Jeffrey Kessler, White House Staff Secretary Will Scharf, White House Deputy Chief of Staff Richard Walters, and a presidential policy assistant, Walker Barrett. The following Monday, CAISI lead Fall held technical meetings with Anthropic and administration officials. Yet it was Bessent and Wiles who pressured the Commerce Department to work with Anthropic to reopen access to its models, while National Economic Council Deputy Director Ryan Baasch continued to push for more industry-friendly demands more in tune with Sacks. That jostling for control continues today as the White House pursues voluntary AI model standards, though CAISI is reportedly going to play a “crucial” role.

Codifying CAISI would at least help insulate it from political influence and make sure it can’t be easily swept away by a hostile administration. The AI Security and Innovation Act would do that, in addition to the small increase in funding. In effect, it allows CAISI to do what it’s currently doing, but with stability that might help it attract more talent, and a little more budget to improve its current functions.

Other legislative measures could go further in strengthening CAISI’s ability to oversee AI. One section of the GAAIA that could be picked up after the midterms would relocate CAISI outside of NIST, but still within the Commerce department, and give it the regulator-adjacent role of administering a regime of licensing and overseeing independent verification organizations, which would audit models. As written, GAAIA doesn’t force frontier companies to implement the recommendations their auditors give them, but would require that they at least have a licensed auditor in the first place.

More imminently, the AI Security and Innovation Act calls for a study into moving CAISI out of NIST, either to a different Commerce office or as its own standalone agency within the department, in addition to evaluating whether CAISI should have additional authorities and funding. The two sources who spoke with Transformer about talks within the House Science Committee said that proposal reflects further reaching discussions, in which some lawmakers have considered putting CAISI within a different department altogether, like the Department of Energy, War, or State. This is because Commerce’s primary mission, to “create the conditions for economic growth and opportunity for all communities,” could sometimes put it at odds with CAISI’s priority to ensure safety. The Department of Energy, meanwhile, has national security resources and a supercomputing agreement with AI companies, and State and War have subject matter expertise in international diplomacy and military use.

Subscribe now

CAISI isn’t moving out of NIST any time soon, much less to a different department. But growing awareness of cybersecurity risks and the chaotic response to Mythos have spurred these preliminary conversations and interest in building a more consistent regulatory regime, says Bullock. Sometimes changes happen slowly, punctuated by short spurts of legislative activity — a sort of “punctuated equilibria,” he says, inspired by Caleb Watney, managing director of public policy at Coefficient Giving. (Coefficient Giving is Transformer’s primary funder.) Momentum is building, and it’s possible this is one of the final lulls before the US government springs into action.

“Maybe nothing will happen for a while,” says Bullock. “But then capabilities increase, then some scary thing happens, or somebody gets spooked, or some people get hurt, and then there might be a lot of interest in suddenly beefing up CAISI.”

Share

AI is already helping develop future AI models. Right now, these systems work only as assistants, shepherded by human programmers. But if the trend continues, we could reach a fully automated pipeline where increasingly intelligent machines create even smarter successors — a recursive self-improvement cycle.

And that, according to many, is all it takes for an intelligence explosion where human-level AIs give rise to Dario Amodei’s “country of geniuses in a datacenter” in under a year. This is the underlying premise in both the much-discussed AI 2027 scenario and the earlier Situational Awareness essay that swept the tech world.

Some think it’s already starting. Anthropic co-founder Jack Clark recently wrote that he thinks “the takeoff towards fully automated AI R&D is happening.” And OpenAI’s Sam Altman wrote last year that we’d reached the “larval version of recursive self-improvement.”

Others think those narratives are papering over a host of challenges that will significantly slow those sub-one-year timelines.

Subscribe now

Optimists — people optimistic that AI can progress quickly, perhaps scarily quickly — have a compelling vision for how such a “foom” scenario could play out. As we saw so vividly in narratives like AI 2027, AI has to first become good enough at coding and AI R&D to automate its own research. Then, within a few months, it can generate synthetic data and learning environments to train the next generation of AIs, developing a learning algorithm that learns as efficiently as humans. It’s then able to quickly pick up new areas using mostly synthetic data, needing only a small amount of human-generated data or real world deployment to verify that the learning algorithm works. Within months, it could reach superhuman capabilities. Less than a year after automating its own improvement, AI is capable of transforming the world. How it plays out after that is anyone’s guess, as we, in Clark’s words, “cross a Rubicon into a nearly-impossible-to-forecast future.”

In that narrative, the AI first automated its own software research, then generalized to most digital and physical work, then finally transformed everyday experiences. The optimists think that this first recursive self-improvement step will be relatively easy. It’s the stage where it’s easiest to generate synthetic data and training environments since code is verifiable and the experts are all right at hand. So far, synthetic data does seem to be fairly effectively targeting gaps in verifiable tasks, such as mathematics or coding. It’s easier to check whether such an answer is right than to come up with that answer. AI can create a large supply of math problems, keep only the ones that check out, and use those verified examples as high-quality training data for future models. Beyond the frontier models, we know the major companies routinely distill synthetic data from the biggest models to train smaller, cheaper models. Smaller AI companies can effectively multiply their compute severalfold by training their models on such outputs, as Anthropic alleges that DeepSeek, Moonshot, and MiniMax illicitly did.

There’s also been an explosion in the creation of virtual environments for reinforcement learning, essentially videogames for AI models. The AI can take a range of actions within these synthetic environments, learning based on rewards built into the system. Imagine a Pokémon videogame, where the AI can capture Pokémon, fight battles, and trade. It wins points depending on the outcomes it achieves. But unlike tasks in the real world, the AI can speedrun potentially billions of iterations on these games, trying every possible combination to find out what maximizes its score. This is the process that allowed the famous AlphaZero to master chess in only four hours of training.

AI will probably need new paradigms to crack sample efficiency, the AI’s ability to squeeze more intelligence out of a given amount of data. But optimists like Forethought senior research fellow Tom Davidson think this will probably happen quickly — after all, they argue, we know it’s possible as humans learn from a tiny fraction of the data that AI currently requires. A human teenager needs something like 20 hours of practice to learn to drive a car. That’s many orders of magnitude less than what Waymo needed to train its cars.

For stage two, where the AI intelligence generalizes to the wider world, Davidson remains optimistic that AI can mostly learn from digital environments. It’s one thing if the AI can master its own R&D. If the AI can also quickly learn how to practice law, play Pokémon, and predict drug discoveries, then it’s likely mastered an underlying learning algorithm. An efficient learning algorithm could easily transfer between skill domains, allowing the AI to rapidly generalize even though the specific skills don’t transfer. Obviously law and medicine require different skill sets, but moving from one to the other shouldn’t mean starting over from scratch. Humans don’t have to relearn how to talk, after all. And if it’s able to generalize to many digital domains, then it might not need much more human-generated data to work in the physical world — a small sample of human-generated tasks might be enough to verify a learning algorithm generalizes.

AI companies are clearly trying to overcome these hurdles. Over the past two years, their spending on external data has shot up to billions a year. That’s dwarfed by their tens of billions spent on compute, but the gap likely narrows when you consider the internal worker hours and compute devoted to generating data. Research found that the majority of R&D compute is spent on producing data and internal research, rather than on final training runs. Some guesses put data generation at around 20% of compute, which would imply billions spent internally by each company. Based on spending trends so far, if money can overcome the data bottleneck, it will. As Davidson told me, “The market economy will find a way.”

However, this vision of a relatively smooth path to self-improving AI rests on several key assumptions that skeptics dispute. Can the AI develop a generalizable learning algorithm based on synthetic data? Will money really be enough to smash through hurdles in months? Skeptics like former GovAI researcher Tom Reed don’t think so. He told Transformer that the need for data will definitely slow an intelligence explosion because it’s required to master the process of researching new models in the first place.

AI R&D consists of a myriad of skills that each take real-world experience to learn — writing clean code, designing experiments, research “taste,” coordinating across dozens of parallel research agendas, optimizing data-generation pipelines, running compute clusters the size of small cities, producing the chips that run them, and so on. Even leaps in sample efficiency can’t fix this, because the data simply doesn’t exist yet. Too much is locked in individuals’ implicit knowledge or relies on studies that haven’t been run yet. How can you train the AI to run a company successfully for decades, when even humans can’t predict how to do it?

We could make the data, but even the most ambitious ideas for how to do this would take months or years, slowing down an intelligence explosion. Even if every employee at TSMC wore a camera recording their actions, narrated their job, and served as consultant when the AI had problems, Reed suspects that you’d need months of such data to begin; the first round wouldn’t work; it would still take time to learn how to use this new kind of data, and so forth. Those little frictions would build up, until the AI takeoff was dragged out for several years longer. Major new breakthroughs in how models learn or are built may take a particularly long time — Reed says that “coming up with new paradigms is probably precisely the kind of thing that even the super good automated researcher … is really not going to be as good at.”

That also matches what some working in the field say about current capabilities. Peter McIntyre, CEO of Trajectory Labs, a company building RL environments for several of the frontier AI companies, told me that RL environment creation is heavily bottlenecked by human expertise. McIntyre points to the work his company is doing developing RL environments to train models against prompt-injection attacks. Building the environment’s tasks requires red-teaming experts, but there are, McIntyre says, “only so many top red teamers in the world that you can hire.” That scarcity of expert talent and knowledge is also true for many other domains.

Proposed workarounds, like just recording experts going about their jobs, are likely to be too unstructured to work off the shelf with current paradigms. Despite the explosion of interest the field is seeing, there are limits to how fast the current methods can scale. While McIntyre says this could eventually change, for now he warns that “if you try to do it too quickly, you’ll just produce slop.”

And the problem only escalates when AI moves from digital self-improvement to interacting with the real world. Many remain skeptical that a learning algorithm derived entirely from digital environments will also work in the physical world. Intuitively, there’s a big difference between playing Baldur’s Gate 3 and actually swinging a sword. The optimists are betting that the AI will master a learning algorithm that generalizes, even though the underlying skills are different. Skeptics aren’t buying it. AI podcaster Dwarkesh Patel suspects that we will need “mind-stretching amounts of human expert trajectories in every single field and skill you want the model to be competent at.”

And skeptics say we can’t speed that up with synthetic data, because of an inherent catch-22: the current models don’t understand messy real-world environments enough to create data that captures them — and they won’t understand messy real-world environments well enough until they have that data. This is backed up by past examples of how AI has been deployed, such as Arvind Narayanan and Sayash Kapoor’s case study of self-driving cars: despite following a similar self-play developmental path to AlphaZero’s, it’s taken decades instead of hours because safety and practical considerations limit how quickly each iteration can learn. Surely, they say, we should expect a recursively self-improving AI to need the same slow learning pace for other real-world tasks.

One thing that both sides mostly agree on: once a fully generalizable artificial intelligence is operating, there will still be delays for world transformation, such as time needed to run serial experiments. Even Anthropic CEO Dario Amodei agrees on that point. The AI might need to wait 60 years to learn which medicines slow aging. But that doesn’t undermine the real change it will bring.

The core disagreement here still revolves around how much real-world data an AI requires for learning. Is it possible for a model to develop an algorithm that allows it to learn general skills from limited, mostly synthetic data, and check its output against small amounts of real-world examples? Or are large volumes of real-world data an irreplaceable ingredient?

But taking a step back, what the two sides are arguing over is far more about when than how or if. Both broadly agree that new discoveries and paradigms are required to reach AGI. One side thinks those will come quickly once AI is doing the work, and that once it breaches the recursive self-improvement stage, world transformation could start within months. The other side thinks these step-changes will take longer: maybe five years, maybe 20, maybe longer.

It might seem like a relatively minor quibble in the grand scheme of things, but it is instructive that those who are most ‘optimistic’ about how quickly AI can overcome these hurdles to an intelligence explosion also tend to be the most worried about what that means for everyone living through it. People tend to worry less if they think we have a couple of decades to solve the problem.

Those few years could change the calculus for students planning life after university, policymakers debating how to respond to AI, or those whose jobs have been displaced.

They could also be crucial for handling even bigger challenges, and for many of those convinced an intelligence explosion could happen incredibly soon and quickly, bottlenecks slowing down recursive self-improvement would come as a relief.

A few of them have already breathed sighs of relief that AI progress hasn’t kept pace with the most ambitious predictions. AI 2027 author Daniel Kokotajlo told MIT Technology Review that extending his timelines by a few years “feels like I just got a better prognosis from my doctor.” (He has, however, since shortened them again by 11 months.) And Jeffrey Ladish, executive director at Palisade Research, simply said, “Thank God we have more time.”

Disclosure: The author’s partner works at Google DeepMind.

NEED TO KNOW

• OpenAI got clearance from the US government to release GPT-5.6.

• SpaceXAI’s latest model release may have violated California’s SB 53.

• The AI Futures Project released AI 2040: Plan A, a “positive vision” for the future of AI which calls for the US and China to negotiate a deal and pause training of frontier models until sufficient progress is made on alignment.

But first…

THE BIG STORY

This morning, South Korean memory chipmaker SK Hynix listed its shares on the Nasdaq. It raised $26.5b, the largest foreign listing in US history.

The reason for Hynix’s bumper payday? AI. As one of only three manufacturers of the advanced memory needed for AI chips, SK Hynix has seen demand explode in recent months.

But what’s good for SK Hynix’s share price has costs for the rest of us. The AI-driven demand for memory has created huge shortages for every other product that needs it. Consumers are already paying for it. Apple recently raised prices on MacBooks and iPads, saying it has “never seen a component price increase this much, this quickly.” Sony, Microsoft and Nintendo have all hiked console prices. And while new factories ease the shortage, SK Group’s chairman still expects global demand to outstrip supply by about 20% through 2030.

Memory is not the only way AI is driving up consumer prices. Demand for construction workers and electricity is adding pressure too. In a recent survey, 81% of economists said AI would add to inflation in the coming year. In the US it’s already running at double the Fed’s target rate.

This is bad for consumers. But it creates an electoral opportunity for politicians. The AI backlash has been searching for something to point to. So far, it has settled on data centers, but those are a slippery target, defended by unions and localized in their impact.

Inflation, in contrast, is almost universally hated — and regularly tops lists of voters’ concerns. The electorate is already drawing the link: research from Blue Rose, a Democratic pollster, finds that “voters aren’t experiencing the cost-of-living crisis and the rise of AI as separate issues; they see one unified threat where a system already rigged for the elite is using new technology to further stack the deck against them.”

Tapping into that anger could be a vote-winning strategy. Politicians are staring at a message that combines two of the things voters hate most: “AI is why you can’t afford a PS5.”

A few have already realized this. The other week, Rep. Frank Pallone — ranking member of the House Energy and Commerce Committee — called for a moratorium on data center construction to tackle “price hikes and inflation spikes.” Across the aisle, some Republicans fear that Trump’s embrace of AI will backfire as consumers feel the squeeze.

But most politicians have yet to catch on. In an excellent analysis of Democrats’ fundraising emails, Free Systems’ Andy Hall found that only 0.7% are substantively about AI (though that number is growing fast). When Democrats do discuss AI, the most common frame is “AI as an instrument of money, oligarchy and corruption.”

Capitalizing on the AI backlash may not be the wisest move, all things considered. Pallone’s proposed moratorium is crude and likely to backfire. But for politicians hoping to win elections, the temptation may be too great to ignore.

— Shakeel Hashim

ALSO NOTABLE

Within the past 72 hours, two companies not normally known for good AI safety practices released new models. One was a pleasant surprise. The other may have broken the law.

Yesterday, Meta launched Muse Spark 1.1. Its performance might not match Anthropic and OpenAI’s latest models, but its evaluation report comes close. Meta tested for risks in chemical and biological dual-use scenarios, cybersecurity and loss of control, on top of now-standard evaluations of behaviors such as hallucination rates and sycophancy. It’s a commendable level of transparency.

Meanwhile, SpaceXAI (yes, that’s what it’s called now) released Grok 4.5. It was trained on Cursor user data, and it scored competitively on public benchmarks (though real world performance seems worse).

Predictably, SpaceXAI didn’t publish a shred of safety information alongside its scant benchmark results. That’s no surprise: SpaceXAI ranks a solid F on the Future of Life Institute’s latest AI Safety Index, and Elon Musk recently testified that he’s “not sure what a safety card is.”

But for the first time, SpaceXAI’s negligence may have consequences. California’s SB 53 requires AI developers to publish transparency reports “before, or concurrently with, deploying a new frontier model or a substantially modified version of an existing frontier model.” As the Midas Project pointed out yesterday, SB 53’s transparency requirement is active and enforceable, and Grok 4.5 looks like exactly the kind of model this was designed to keep in check. The question now is whether the law has any teeth: if this can’t trigger enforcement, it’s hard to imagine what would.

— Celia Ford

THIS WEEK ON TRANSFORMER

• Scaling works. These researchers are betting billions it isn’t enough — Celia Ford digs into the LLM alternatives some believe are the key to AGI

• An AI safety group hid its election spending through a Latino-focused PAC — Veronica Irwin reports on Public First’s clandestine backing for a Colorado Democrat

• GPT-5.6 cheats so much its testers couldn’t measure it — Celia Ford on how OpenAI’s new model broke rules and exploited loopholes more than any model METR has tested to date

• SCOTUS killed the independent agency. AI governance doesn’t need one — Fathom CEO Andrew Freedman argues that the Supreme Court’s Slaughter ruling makes the case for independent verification for AI governance

• Don’t let independent AI audits provide a false sense of safety — AI policy researcher Keller Scholl argues that a marketplace of AI auditors will always prioritize speed and cost over safety

Subscribe now

THE DISCOURSE

AI power users are pitting GPT-5.6 Sol against Fable.

• Peter Gostev: “Fable is a ‘wise owl’ who is very thoughtful and very well spoken, GPT-5.6 Sol is like a rottweiler who will grab the problem by the throat and not let go until it is done.”

• Dan Shipper: “GPT-5.6 is like a Porsche, Fable is like a warp drive … If you need to get across the galaxy use Fable. If you need to get around town using the best available tool for the job, use 5.6.”

• Mitchell Hashimoto: “Sol is a charismatic, efficient, talented coworker you’re jealous of. Fable is a genius recluse that is brilliant at its fixations but doesn’t go out, doesn’t date, and you don’t want to hang out with them much lol.”

  • Sam Altman replied: “tbh i don’t think sol gets that many dates either”

The AI Futures Project released AI 2040: Plan A.

• “AI companies are racing to build AIs that are smarter than humans in every way. In AI 2027, we predicted that this would result in either extinction or irreversible concentration of power. Plan A is our positive vision for what should happen instead.”

• It calls for the US and China to negotiate a deal and pause training of frontier models until sufficient progress is made on alignment — before “handing over” control to safe, superintelligent AIs in 2040.

• Many people praised it as being the most detailed plan for handling superintelligence that exists so far. Many others criticized it for being unrealistic and undesirable.

Peter Thiel continued his campaign against God’s representative on earth:

• He told the Aspen Ideas Festival that Pope Leo was inadvertently serving as a “Chinese communist agent” in his calls for oversight of AI.

Microsoft President Brad Smith criticized Trump’s AI policy:

• “Everyone is reluctant to say there should be regulation, but what we really have right now is regulation without transparent or complete rules. Without rules, businesses can’t plan.”

Former White House AI adviser Sriram Krishnan said Trump’s AI policy will remain deregulatory:

• “There will not be an FDA for AI … That is never, never going to happen under President Trump.”

• “This administration, [the] president, from day one has been against burdensome, onerous, bureaucratic red tape.”

POLICY

• The Commerce Department partially lifted its export ban on Anthropic’s Claude Mythos 5 model and allowed Fable 5 to be made available to the public.

  • When the ban was lifted last week, Mythos 5 was only available to certain US companies. This week, Anthropic said it had begun granting access to foreign organizations, in coordination with the US government.

• The government also allowed GPT-5.6 to be released, after a several week delay.

  • A White House official claimed that it hadn’t given OpenAI a “green light” because “no such permission is required or granted,” seemingly in an attempt to claim no licensing regime exists.

  • That directly contradicts OpenAI’s statements both before and after the model’s launch, though.

  • The UK’s AI Security Institute said it found universal jailbreaks for the model “within hours,” which gave users access to its potentially dangerous cyber capabilities.

    • AISI said OpenAI’s mitigated those jailbreaks, but that it expects “further red teaming to surface similar jailbreaks.”

• The White House has reportedly accelerated plans for voluntary AI model standards.

  • When redeploying Fable, Anthropic proposed a “consensus industry framework” for assessing jailbreaks’ severity.

• OpenAI reportedly proposed that it and other frontier AI companies give the US government a 5% stake of their equity.

  • Anthropic has reportedly not talked to the government about this.

• Sen. Elizabeth Warren and Rep. Mary Gay Scanlon are reportedly preparing to introduce a bill to ban AI companies from selling users’ health data to data brokers.

• Reps. Josh Gottheimer and John Moolenaar introduced the Cloud Security Act, which would “allow cloud compute providers to voluntarily report suspected misuse of their services by customers associated with US adversaries.”

  • On Friday, the Financial Times reported that OpenAI and Google were selling access to their advanced AI models to Singaporean subsidiaries of Alibaba, Baidu and Tencent — all of whom are on the Pentagon’s blacklist of companies linked to the Chinese military.

• The FTC warned companies that altering AI outputs to comply with state laws could violate federal consumer protection rules.

• California launched a public AI-unemployment tracker to monitor and predict AI-related job losses.

• Illinois Gov. Pritzker signed SB 315, which requires AI developers to publish transparency frameworks, employ third-party auditors and report safety incidents.

• China reportedly plans to allow top AI firms including Alibaba, ByteDance and DeepSeek to purchase up to 200,000 Nvidia H200s for training, due to a shortage of domestic compute.

• China’s also reportedly considering restricting overseas access to China’s most advanced AI models, which could include limits on open-weight models.

• A Supreme Court ruling shrinking the Clean Water Act has allowed dozens of data centers to skip federal water permits, limiting public input on projects.

• The UN AI Scientific Panel published its preliminary report. It is about as milquetoast as you’d expect, though it does discuss loss of control risks and the potential for “catastrophic outcomes.”

INFLUENCE

• OpenAI published its “National Security Principles,” which say that it “will not support use of OpenAI tools for mass domestic surveillance, high-stakes decisions … without appropriate human judgment and accountability, [or] uses that evade legal obligations, oversight, or accountability.”

  • OpenAI said it will implement these principles via “contractual usage restrictions, model behavior specifications, and customer engagement and oversight.”

• Manny Rutinel won his primary race for Congress.

  • Public First, the Anthropic-backed non-profit and super PAC effort, spent $2m secretly supporting Rutinel via a Latino-focused super PAC.

  • Public First said it has raised $80m total.

• Mallory McMorrow suspended her Michigan Senate campaign. She’d been positioning herself as an AI safety champion.

  • Abdul El-Sayed, who remains in the race, released an AI policy plan calling for democratic governance of AI companies, public ownership via a sovereign wealth fund, and an FDA for AI.

• Leading the Future spent $600k supporting former NFL kicker Jay Feely‘s congressional campaign in Arizona.

• A new poll found Americans are nearly three times as likely to be concerned rather than excited about AI’s “growing role” in society.

• A new AIPI poll found bipartisan majorities of likely voters support the AI OVERWATCH Act (66% support), the MATCH Act (63%), and the Chip Security Act (60%).

• The New York Times looked at how political campaigns are integrating AI tools to analyze voter data and craft messages.

• The Foundation for American Innovation launched Frontier Legal Defense, a legal team to combat “concentration of power, incumbent rent-seeking, and government overreach in AI.”

• Environmental activist Erin Brockovich launched a campaign against AI datacenters.

• Nvidia warned the US government that Huawei could “satisfy global AI chip demand” as part of a bid to get H200 export controls lifted. Experts say the claim was misleading.

• A Punchbowl survey found that Hill staffers rank “losing control of AI” as one of the “most important challenge[s]” for America over the next 250 years.

INDUSTRY

OpenAI

• GPT-5.6 is finally available, after the Trump administration asked OpenAI to stagger its release.

  • OpenAI claimed 5.6-Sol “autonomously post-trained GPT-5.6-Luna,” its smaller sibling. The reality seems to be a bit less impressive than that sounds.

• OpenAI launched GPT-Live-1, its next generation of voice models that can listen and talk at the same time, creating “a voice experience that is refreshingly easy to talk to.” (What could possibly go wrong?)

• And it merged the ChatGPT and Codex desktop apps.

• The OpenAI Deployment Company acquired Northslope, part of its efforts to help enterprise customers implement OpenAI’s tools.

• OpenAI apparently never visited one of Stargate UK’s key sites, The Guardian reported, suggesting that most of its advertised investment was just hypothetical.

Meta

• Mark Zuckerberg reportedly admitted to staff that the company’s reorganization around AI — which saw it lay off 10% of its workforce — wasn’t as “clean” as it could have been, and it had miscalculated its timing.

  • According to Reuters, he told a townhall meeting that “that trajectory of the agentic development over at least the last four months hasn’t really accelerated in the way that we expected,“ and ​the intended outcomes from the new structure “haven’t come to fruition yet.”

• The company is reportedly planning to set up a cloud business selling AI compute and model access to compete with Microsoft, Amazon and Google.

• It launched Muse Image and Muse Video, its first AI media generation models from its restructured AI team.

  • Be warned: it pulls from public Instagram profiles as image fodder, unless users opt out.

• It’s reportedly going to start making its custom “Iris” AI chip in September.

• It announced its first Canadian data center, a $9b 1GW facility in Alberta.

• It’s testing “super sensing” AI glasses that continually record audio and images without turning on the LED that currently signals whether AI glasses are recording.

• It also reportedly filed a patent for an “apparatus” that continually records you and your surroundings, then uses AI to analyze your mood and…plan your workouts?

• An interesting post from SemiAnalysis said it believes Meta is the best placed company to catch up with OpenAI and Anthropic, having set the foundations for becoming “world class” in data, talent, and compute.

Anthropic

• Anthropic launched Claude Science, an app that integrates with commonly-used research tools such as PubMed, Jupyter and R to generate figures and manuscripts alongside code.

• It expanded Claude Cowork to web and mobile, meaning users won’t have to leave their laptops cracked open for agents to get tasks done.

• It’s reportedly talking to Samsung about making custom AI chips.

• It’s leasing an NYC office building with capacity for at least 1,700 workers.

• China found “security backdoor vulnerabilities” in Claude Code — which Anthropic said it put there to stop Chinese users who weren’t supposed to be running it in the first place.

Nvidia

• Nvidia lost $1t in market value this summer, down from a high of $5.7t in May. According to (a confusingly-framed article from) Bloomberg, the company’s shares are currently trading at 18x its projected yearly earnings, the same as in 2019 prior to the AI boom.

• Chinese companies are reportedly spending less on Nvidia and more on domestic products.

Microsoft

• Microsoft is starting to use its own MAI models in Excel and Outlook, replacing models from OpenAI and Anthropic.

• It cut 4,800 employees, mostly in commercial sales and its Xbox division.

  • An internal memo from executive VP Amy Coleman told employees: “the roles eliminated today are not being replaced by AI. At the same time, what is true is that AI is changing how work gets done.”

Others

• Apple reportedly plans to spend over $30b on US-made AI chips from Broadcom.

• Nscale got a $900m line of credit to build out data centers internationally.

• DeepSeek is reportedly developing its own AI chips for inference.

• Nokia’s stock has gone up by 90% as the former mobile handset maker pivoted to data center infrastructure.

• The US may be short around 157,000 skilled workers by 2030, posing challenges for semiconductor plants.

• Secretive Chinese chipmaker CXMT is set for a $4.3b IPO, the country’s biggest of 2026 so far. According to Bloomberg, the company is seen as China’s best bet to produce memory for AI chips.

• MiniMax is seeking to raise almost $2b via shares and convertible bonds, ahead of reported plans to launch a new 2.7t parameter model that would become China’s largest open-source LLM.

  • CEO Junjie Yan reportedly vowed to forgo his salary until achieving AGI.

• Many US-based companies are increasingly switching to Chinese AI models, citing lower costs and open weights as advantages.

• PE group Carlyle sold its data center energy business Copia Power to EQT for $2.6b, more than five times its initial investment.

• a16z-backed startup Ornn raised a $33m seed round to build a marketplace for trading AI compute.

MOVES

• Fidji Simo, who joined OpenAI last August to work as the company’s No.2 running product and business, stepped down due to illness.

• Ben Bernanke, former Fed chairman, joined Anthropic’s Long-Term Benefit Trust.

• Marc Andreessen, Chad Jones and Asha Sharma joined a new Federal Reserve advisory group on AI’s economic impacts.

• Joshua Achiam, OpenAI’s Chief Futurist, is leaving after an almost nine year run.

  • “There’s not a specific reason for me leaving, or a specific reason for why now,” he wrote. “But it’s something I have been thinking about for a while and it feels right. The world is in on the secret now and it feels possible to work on the mission from outside the walls of a frontier lab.”

• Institute for Progress co-founder Caleb Watney joined Coefficient Giving as managing director of public policy, where he’ll lead a new US AI policy team.

  • Disclosure: Coefficient Giving is Transformer’s main funder.

• Harvey Lederman joined Anthropic to study “alignment and character.”

  • He will do so while teaching philosophy at NYU.

• Teresa Carlson joined Anthropic to lead public sector strategy, after over 20 years of doing so for Microsoft and AWS.

• Darya Kaviana joined Anthropic’s Safeguards Research team.

• Jay Puri retired from his role as Nvidia’s top sales executive.

  • Nicholas Parker will reportedly replace him in August.

• Anna Soellner joined Nvidia as its new head of corporate communications.

• Nat Purser and Patricia Paskov joined the AI Verification and Evaluation Research Institute as Director of US Policy and Director of Standards, respectively.

• Greg Barbaccia is leaving his role as federal CIO and chief AI officer.

• Jasper Götting is taking over leadership of SecureBio’s AI group.

  • Seth Donoughe, the former director, is moving to RAND to focus on AI safety and biosecurity.

• Sayash Kapoor is joining UC Berkeley’s School of Information in fall 2027.

• Omar Yaghi joined Tsinghua University, where he’ll run a new AI-driven materials science research center.

• Fortune hired Emily Forlini as a senior AI reporter.

RESEARCH

• Anthropic published a buzzy paper introducing the “J-space,” a set of internal representations that seem to hold nonverbalized thoughts “in mind” while Claude thinks.

  • Researcher Jack Lindsey argued that “understanding this ‘workspace’ is key to making sense of LLM cognition.”

  • Anthropic also published external commentary from researchers in cognitive neuroscience, philosophy, AI welfare, and mechanistic interpretability.

• Resolution, the automated alignment nonprofit founded by Geoffrey Irving, received a massive $160m grant from Coefficient Giving.

  • Irving tweeted: “Our grant took six weeks from first conversation to confirmation. More capital is entering the field fast via the OpenAI Foundation and the Anthropic IPO. It’s time for everyone in AI safety to be more ambitious.”

• The Center for Mind, Ethics and Policy and Eleos AI published a report advising researchers on how to study AI welfare empirically.

• Epoch AI and METR launched MirrorCode, a new benchmark that tests AI models’ ability to complete long tasks end-to-end.

• OpenAI introduced GeneBench-Pro, a new computational biology benchmark that tests agents’ ability to handle the ambiguity of real-world science.

• OpenAI also found that about 30% of tasks in SWE-Bench Pro, a popular coding benchmark, are broken.

• The UK’s AISI reported that running evaluations on a fixed compute budget tends to underestimate frontier AI capabilities.

  • Instead of reporting a single benchmark score, they recommend that evaluators report how an AI agent’s score changes as you give it more compute.

• GovAI found that 11% of Meta, Google, OpenAI and Anthropic model releases were delayed or withheld in the EU (7% in the UK) between 2018 and 2026, mainly due to data protection regulations rather than the EU AI Act.

• Pangram reported that a quarter of social media content it scanned was flagged as 100% AI-generated, especially longform content on LinkedIn.

BEST OF THE REST

• NY Mag covered the vibes as effective altruists discuss the incoming wave of post-IPO wealth from Anthropic.

  • LessWrong and Lighthaven’s Oliver Habryka is worried about it: “I do think my environment will end up being very heavily selected for Anthropic sycophancy … I’m not looking forward to it.”

• The NYT spoke to people who are using AI chatbots to guide their midterm votes.

• The Atlantic examined “universal basic capital” proposals as a hedge against AI-driven inequality, which potentially risks regulatory capture and government control over AI companies.

• A Brown University professor said he suspects most of his students use AI to cheat, citing a suspicious average score of 96% in his class’s midterm take-home exam.

• The WSJ looked at the wealthy families choosing AI-based alternative schools over traditional education.

• Epoch AI researchers argued that AI futurism debates overlook how hard things like Dyson spheres actually are to build.

• 1X unveiled its remarkably dextrous new robot hands, in a video which includes a wild shot of a humanoid robot … undressing someone.

MEME OF THE WEEK

(Credit: @RespectfulMemes.)

Thanks for reading. Have a great weekend.

Share

Recent proposals have argued that the best route to governing AI is to separate the technical and political parts of the problem. The idea is attractively simple yet catastrophically flawed.

Andrew Freedman, CEO and co-founder of the AI governance nonprofit Fathom, recently wrote about this idea in Transformer. He argued that independent verification organizations, working to standards defined by the government, could “insulate the facts” from the “coercive choices” that are a part of politics. But such insulation is fanciful. Vaccines, for example, are safe, but the facts have not protected them from politics. Freedman doesn’t propose any means of preventing AI testing from winding up in the same situation.

Subscribe now

There is a certain elegance to the concept described by Fathom and Freedman, which is similar to one described in 2023 (and recently updated) by Gillian Hadfield of Johns Hopkins University and Jack Clark of Anthropic. Boiled down to its essence, the idea runs like this: (1) The government sets outcome-based safety standards; (2) The government licenses Independent Verification Organizations (IVOs); (3) AI developers opt-in to the process, which partially insulates them from certain forms of liability. If an AI developer opts-in, it must be certified by one of the IVOs licensed by the government. The Fathom proposal does not even include the Hadfield-Clark concept that an AI developer’s private regulator could impose meaningful penalties short of cutting off the regulator’s own revenue stream. If the IVOs fail to ensure the outcome-based standards the government set, they can lose the license they were granted.

You can see this as matching a long history of limited governmental interventions to produce a healthy market. One of the clearest parallels here is finance, where the government sets communication standards for financial reporting, companies pay accounting firms to produce reports on the companies, and other market actors use the reports to inform their own decision making about whether to buy a company’s stock, or lend to it, or enter into partnerships with it.

This does not always work as intended. Accounting firms that err sufficiently, like FTX’s accountants, can face insubstantial fines: in that case, of just $745,000. Theoretically, there should be substantive reputational costs — Arthur Andersen, the accountants for Enron, found that their customer base abandoned them — but governments have made deliberate policy decisions to not kill off large accounting firms over concerns about consolidation in the market. And if the accounting is bad, the total costs are shared by actors far beyond the company and the auditor, including lenders and stockholders who eat a great deal of the loss.

In fact, the approach can fail catastrophically. Perhaps the best example is how the Credit Rating Agencies (CRAs) directly contributed to the Great Recession. In the runup to the 2008 crash, banks issued loans, packaged them up in certain ways and then went to the CRAs. The banks paid the CRAs, which certified that the packages were AAA-rated, the safest a package of loans could be. If a CRA employee thought a particular package did not deserve an AAA rating, the bank would go to a more compliant competitor. The result was predictable in hindsight: ratings were not consistent with actual risks.

The CRAs were certified by the government, just like the IVOs that Fathom and Freedman ask for. But it would have made no sense for the CRAs to behave any differently, because the penalties for their poor behavior would not arrive for a decade or so, and never in such large a quantity that they threatened the survival of the firm. By contrast, not participating in the booming market would have meant that they had hemorrhaged market share.

In the worst case of the CRAs, banks were fully aware and taking advantage of the risk of what they were doing, and though they didn’t actively want to lose money they at least knew what the stakes were.

But of course developers want minimal oversight; that’s universal. Builders want minimal oversight from fire codes to save money. Crucially, buyers frequently want strong oversight: I don’t want the building I’ve purchased to burn down due to low standards. But some AI buyers might want IVOs with minimal compliance, seeing it as an attractive focus on customer features, not a red flag. The problem is this would in large part be because AI companies and buyers are not internalizing the full totality of potential risks of a technology that we still struggle to predict the impact of.

Part of the discussed solution for CRAs was mandatory randomization, but that destroys the competitive incentives that are supposed to make the Fathom proposal drive innovation, with each IVO striving to advance the science of measuring AI safety. And without a randomization mechanism in place, IVOs would end up incentivized to provide the minimum safety rules defined by the government while optimizing for speed and cost for their clients. Anything else is bad business.

If the main goal is safety, that will never be a sensible approach.

Share

Keller Scholl got his Ph.D. at RAND, studying the economics of artificial intelligence. He is an independent policy analyst focused on making the AI transition go well, and writes at keller.substack.com.

Intelligence is a slippery thing. For millennia, humans assumed it flowed through a fluid, or one continuous sheet crumpled inside the skull. Microscopes later revealed a dizzying network of individual neurons, sculpting intelligence from electrochemical noise to give our brains a certain je ne sais quoi.

In the late 1950s, psychologist Frank Rosenblatt designed the perceptron, a brain-inspired algorithm that adjusts the relative strength of its units’ connections with experience. The New York Times described it as “the embryo of an electronic computer” that would one day “be able to walk, talk, see, write, reproduce itself and be conscious of its existence.”

By 1971, the year Rosenblatt drowned in a boating accident, this “connectionist” approach to artificial intelligence had been overpowered by the field’s attachment to hand-coding clever rules based on human knowledge. (In the 1970s, for example, AI researchers asked doctors how they spot bacterial infections, and coded a diagnostic tool by meticulously transcribing hundreds of human-inspired if-then rules. Any expertise doctors couldn’t put into words — one might call it vibes — was inherently left out.)

Subscribe now

Fast-forward to today, and every large language model is powered by deep learning, an incredibly connectionist approach. It’s perceptrons all the way down, stacked more than Rosenblatt could imagine. AI researcher Rich Sutton called it the Bitter Lesson: general, brute-force computation usually works better than trying to recreate our own thinking from scratch. And sometime after the 2017 introduction of the transformer, AI developers found another way to piss off old-school purists who believe intelligence has to be intentionally designed: make models better by making them bigger. Hundreds of billions of parameters, the numerical knobs that set the strength of connections between model units. Scrape the entire internet, digitize the world’s books, intellectual property be damned! And so far, it’s been getting the job done. Current LLMs are really powerful, and shockingly capable of executing many (though certainly not all) long, complicated tasks without human oversight.

Scaling continues to work so well that there’s little hard evidence to suggest that it’ll stop working any time soon — just a hunch that a strategy so crude couldn’t possibly create superintelligence. Yet some of the field’s leading figures still think it’s a dead end. Before leaving Meta last year, Yann LeCun said “We are not going to get to human-level AI just by scaling LLMs.” In the words of OpenAI co-founder Ilya Sutskever, whose 2012 work on AlexNet flaunted the power of scaling over traditional machine learning methods: “From 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling … But now the scale is so big … So it’s back to the age of research again, just with big computers.”

Deciding what research to throw big computers and billions of dollars at depends on how you think “intelligence” really works. Some believe that the structure of large language models can never represent the world like humans do. Others suspect that the way LLMs are trained prevents them from learning as well as humans. But whether it’s the model’s underlying architecture or training protocols to blame for its shortcomings — or something else entirely — scaling skeptics share the conviction that evolution blessed human minds with some fundamental primitive of intelligence that shoving text through a transformer can’t recreate. Now, a handful of them have left frontier AI companies to figure out what to build instead.

Growing up in Florida, I had to log 50 supervised hours behind the wheel to be eligible for a driver’s license. Just over a single work week puttering around with my terrified parents taught me enough to pass the test. I’m not the best driver (Florida isn’t known for minting those), but I’ve made it over 14 years without a single accident or traffic violation.

There’s nothing special about me or my driving prowess, but it demonstrates something that make humans particularly good learners: we are able to learn from relatively limited experience. LLMs, despite having seen lots of cars and memorized every single driving-related book on Earth, still cannot drive a car. Passively absorbing multimodal information is not the same as hands-on experience. In fact, one could argue that these models don’t “learn” at all, at least not in the animal sense, through some combination of evolution, instruction, and doing stuff in the real world. Rather, they’re trained on the fruits of human learning, sculpted by gradient descent, polished by human (and increasingly, AI) feedback, and trained on specific tasks in a reinforcement learning “gym” (more on that later). Then, their weights are frozen in time.

Other non-LLM AI systems, such as the models under the hood of Waymos, do learn from experience (both real and virtual). But they need many orders of magnitude more experience to get as good as a human. Waymo’s AI model has traveled more than 200m miles and counting, plus billions more virtual ones, and racks up more with every trip. In human years, that’s hundreds of lifetimes, and a Waymo might still drive in inexplicable circles or run over a beloved bodega cat. And while a Waymo can drive a car, it couldn’t write this article for me.

The holy grail for AI researchers is a model that can resolve both of these problems: AI systems that can learn from experience, and not much of it.

Some researchers, most notably Yann LeCun — Meta’s former chief AI scientist — have spent decades pushing for loosely brain-inspired model architectures that can, at least in theory, learn more effectively than even the most scaled-up LLM ever could. Saying LLMs are “just predicting the next token” has become an AI skeptic trope, but mechanistically, that is indeed what they’re doing. (To do this as well as frontier models do today, they’ve had to build complex representations of language that have long since promoted them above “stochastic parrots.”)

LeCun’s alternative vision is built on JEPA, the Joint Embedding Predictive Architecture. Instead of recreating and predicting raw data — individual words or single pixels, like a generative model does — it would compress that data into a more abstract state, throwing out unnecessary information while preserving the important gist. While this system passively consumes sights, sounds and words, LeCun and his colleagues envision another system, operating in parallel, that actively does things, learns from their consequences, and generalizes those lessons to new situations. For example, a robot powered by Meta’s V-JEPA 2, a version of this model, learned to move specific objects on command, without having seen those objects before. Theoretically, these systems can train each other: as psychologist James J. Gibson put it, we see in order to move and we move in order to see.

This vision combines two related but separable bets. The first is on world models: systems that learn how the world works through multimodal observation. A world model is something a mind carries — we probably have one, in neural networks sprawling across the hippocampus and prefrontal cortex. It’s built partly through experience and, maybe, partly through hard-coded genetics that bias what kinds of sensory data we care the most about (minutes-old newborns, for example, prefer looking at blobs arranged like a smiley face over other abstract patterns). It serves to predict how one action leads to another, allowing an agent to make informed choices.

The second bet is on reinforcement learning, which is something a mind does to learn. Reinforcement learning, or RL, without a world model is simply learning from trial and error, without being fed human-curated training data at all. Our brains do this with temporal difference learning, an RL algorithm which, curiously, AI researchers Rich Sutton and Andrew Barto mathematically described in the 1980s — years before anyone realized that the very same algorithm powers our dopamine system.

With help from a world model, that trial and error process could potentially be compressed. An agent could simulate potential actions and outcomes before committing to a move, reducing the amount of pure guesswork involved — although for now, this is limited to fairly short, specific tasks. The broader vision of generalizable “action-conditioned world models” is what LeCun’s recently-launched startup, AMI Labs, is aiming for. In theory, these world models could simulate cause and effect abstractly, beyond the physical domain. Knowledge workers regularly take lofty goals — “boost engagement,” or “launch in a new market” — and break them down into sub-steps and sub-sub-steps through hierarchical planning. Frontier LLMs are getting better at this, but often aren’t reliable working on long, multi-step tasks. World model advocates suspect that a better model of cause and effect is what’s missing.

Other startups chasing a post-LLM future are mostly focusing on world models or reinforcement learning (or, in at least one case, not telling anyone what they’re doing at all). This all assumes that there’s something fundamental to human intelligence that the current paradigm will never crack. But with enough data and a giant pile of transformers, LLMs may reach “AGI,” whatever it means, before any alternatives come to fruition.

While some alternatives to scaling are still largely theoretical, world models are already out there, making photorealistic images and training self-driving cars. Several frontier companies are on it: Google DeepMind announced plans to make “massive generative models that simulate the world” at the start of 2025. Later that year, Meta poached Tim Brooks, who was leading DeepMind’s efforts and previously worked on Sora, OpenAI’s video model, to “make multimodal generative models” for its Superintelligence Labs. Microsoft is in the early stages of building models that create video game environments in real time. (With all their complicated physics, modern video games are, in essence, simulations of the world.)

Fei-Fei Li’s startup, World Labs, raised $1b earlier this year at a reported $5b valuation, and AMI Labs recently raised over $1b without having any plans to generate revenue. “My prediction is that ‘world models’ will be the next buzzword,” CEO Alexandre LeBrun told TechCrunch in March. “In six months, every company will call itself a world model to raise funding.”

World models exist on a spectrum, though. At the most basic level, they predict how the world will appear, given some description or action. Google’s Genie 3, for example, takes in text prompts and outputs pixels, rendering photorealistic interactive environments for AI agents. While these simulations can look beautiful, agents can only live within them for a few minutes, and can’t actually do much — you may be able to walk around and jump, but more complex actions are still out of reach. As World Labs recently wrote about both Genie 3 and its own Real-Time Frame Model, another interactive video generator, without a deeper understanding of real-world physics their output is limited to “what a viewer would see, not what is.” The worlds they generate “may look flawless from above, but try to drive through the city below and they fall apart.”

Some argue that world models can emerge within a next-token predictor — at least to some extent. In 2022, researchers trained an early GPT variant on lists of moves for grid-based strategy game Othello, without showing it the game board or the rules directly. Its internal activity represented the game board — essentially, its whole world — and used it to play the game. But the evidence is mixed. In a more recent study, researchers demonstrated that even after training transformers to near-flawlessly predict turn-by-turn New York City driving directions, its internal representation of the city was filled with nonsensical routes and impossible physics.

In the short term, current general-purpose world models won’t replace LLMs as anyone’s daily driver. Oliver Cameron, co-founder and CEO of world model startup Odyssey — which recently raised $310m at a $1.45b valuation — described them to me as “sort of GPT-2 equivalents.” OpenAI’s GPT-2, with 1.5b parameters, was dwarfed by GPT-3’s 175b, and was trained on a tiny sliver of the internet, relative to its successor. To experience a similar step change in performance, world models will need much, much more data — somewhat ironically, given their departure from LLM-scaling orthodoxy. And unlike LLMs, the data they need — ideally, first-person videos of mundane tasks — is much harder to come by than text. This is why startups are paying people to strap cameras to their foreheads and film themselves cooking and doing laundry.

After pretraining, current frontier LLMs are fine-tuned with reinforcement learning. Initially, this mostly took the form of humans rewarding behaviors they like, pushing the base model to express the knowledge it’s already stored in an agreeable way. Increasingly, though, models go through reinforcement learning with verifiable rewards, or RLVR. Models are put into a reinforcement learning “gym”: a practice environment where they’re given tasks with clear correct answers, like coding or writing math proofs. Their answers are then checked automatically. If the models pass, they’re rewarded; if they fail, they adjust and try again. It’s a form of trial and error, and it accounts for much of models’ recent progress in reasoning, coding and computer use.

But even RLVR is still a far cry from the kind of RL animals take advantage of. We’re capable of figuring things out in ambiguous, open-ended situations where there’s no clear “right” move, and where it’s hard to tell exactly what action even deserves a “reward.” Even with RLVR, models still struggle at this. The bulk of what they know still comes from pretraining on human-curated data, mostly text. RLVR refines how they reason over it, but does not teach them the world from scratch.

Before a team of Google researchers introduced the transformer architecture in 2017, the company was all-in on RL. In DeepMind’s earlier days, its splashiest research was heavily inspired by neuroscience and largely based on RL agents. David Silver, who led DeepMind’s RL research program from 2013 to 2026, studied under Rich Sutton as a PhD student and inherited his hunch that intelligence comes from fucking around and finding out. In 2017, his work on AlphaZero showed that a single algorithm could get a model to beat specialized programs at a bunch of challenging games, despite starting with no special training on the tasks at hand.

It was ultimately OpenAI, not Google, that figured out how to scale transformers and commercialize them with ChatGPT. Google Brain and DeepMind scrambled to pivot and catch up, but a couple of recent high-profile departures may have solidified Google’s distant third place in the AGI race. On the surface, it looks like DeepMind’s original reinforcement learning approach lost to the transformer. But RL is still a crucial part of the LLM training pipeline, and increasingly so. Researchers like Silver want to go a step further still.

In April, Silver raised over $1b for his new startup, Ineffable Intelligence. “Other forms of AI will succeed in my absence: generative language, video, code, and more — all are in good hands,” he wrote, adding that he aims to create “a place where the full ambition of the reinforcement learning paradigm can flourish.” Ineffable doesn’t have a product yet, and likely won’t for quite some time. But AlphaZero beat Stockfish, the virtual chess engine human grandmasters train against, without training on examples of chess games — in theory, a more advanced RL model could become superhuman at just about anything. While learned chess tricks wouldn’t automatically transfer to new domains like Othello or folding a protein, the learning strategy itself could. Silver is placing his eggs in DeepMind’s original basket and betting that superintelligence will emerge from models that learn from experience and play, without imitating human data at all. This version of RL is meant to produce the model’s intelligence, not just polish it.

Researchers pursuing world models and reinforcement learning are still more or less carrying Rosenblatt’s connectionist torch. They trust that, given better data and learning algorithms, vast, layered networks of computational units can be very smart. But a handful of researchers, including ex-DeepMind scientist Yuan Cao, worry that even the most advanced LLMs will never be capable of inventing truly new things. Cao, now the co-founder and CEO of Unreasonable Labs, described LLMs as “a very, very dense net.” You can make the net denser by adding more data and more parameters, “but it’s still a net,” he told me. “No matter how dense you weave it, there are always some small holes in it.”

In defiance of the Bitter Lesson, Cao and some others are pulling inspiration from cognitive science and the good old-fashioned AI that got steamrolled by scaling’s success. This old paradigm, also called symbolic AI, defined “intelligence” as a collection of logical, legible rules derived from human experts. Models constructed this way made sense. A human could understand how they came to decisions. But they were extremely limited and easily breakable — it’s hard to encode everything in rules.

But Cao believes that the instinct to give an AI model structured concepts it can manipulate after deployment may have been on the right track. The human mind, he argues, works by continually updating an evolving graph of concepts, allowing us to make associations, analogies and abstractions. This may be how our ancestors came up with concepts like numbers and entropy, and how we’re able to think about our own thinking. Crucially, if the nodes of our mental model can evolve, then we can draw connections beyond the model’s current constraints. Once trained, an LLM can recombine information it’s already stored, but its architecture and weights are fixed. Cao said that this prevents truly new ideas from being generated by LLMs alone. “We have to integrate the language model with [a] symbolic procedure.”

Unreasonable Labs is part of a small wave of startups investing in neurosymbolic AI, an approach which combines LLMs with the kind of structured, symbolic reasoning that dominated the field fifty years ago. In March, Unreasonable raised $13.5m to scale its AI platform, which the company is marketing as an engine for generating scientific hypotheses and experiment designs. In an early proof of concept, the company says an engineer asked the platform to design a 3D-printed sheet that’s strong, flexible, and lightweight, three properties that usually don’t go together. By Unreasonable’s account, it tossed out a wild (and successful) suggestion: create a layered, scale-covered lattice like a butterfly wing.

These particular results likely didn’t require a new model. When my editor and I separately asked Claude Opus 4.8 to take a stab at this problem, it produced STL files for 3D printing something similar that, to our admittedly untrained eyes, seemed passable. After all, scientists have published papers about butterfly-wing-inspired lattice structures for decades. Surely some, if not all, of these papers made it into frontier LLMs’ training data — the answers that engineers needed probably lived in mainstream LLM weights anyway. It’s possible that a neurosymbolic system can still invent concepts completely beyond what it learned during training, but it’s hard to prove.

We’re really good at many things that AI systems still suck at, which suggests that our brains are still worth reverse-engineering. A couple of human-specific talents that AI developers especially lust after: learning from very little data, and running on very little power.

Human cognition could just be one of many paths to intelligence, though. “It’s not necessarily that we’re better at learning anything,” said Adam Marblestone, co-founder and CEO of Convergent Research and ex-DeepMind research scientist. But “we might be better at learning the things that are relevant in the human world.” Our hard-coded biases and intuitions for things such as social decision-making and facial recognition may be well-suited for navigating Earth as a human, but aren’t necessarily relevant for an AI agent tasked with, say, predicting how a protein will fold.

“We did learn something from deep learning,” Marblestone said. Clever though our brains may be, hyperscaling LLMs “can actually do a whole lot.” They might even be safer, he argues. “On one hand, the AI paradigm right now is not very neuroscience based, but on the other hand, a neuroscience-based one might be pretty dangerous.”

While alignment is far from being solved, it’s probably a more tractable problem in LLMs than, say, AI systems that learn entirely from their own experiences. Conveniently, next-token predictors trained on a massive sample of digitized human thoughts seem to inherit the dataset’s approximation of human values, which mostly average out to be normal-ish, or at least “fairly close to something you can tweak to be honest and helpful and harmless,” Marblestone said. Without this pre-filtered training data, the inclinations of something like a reinforcement learning system are more susceptible to fate. In the earliest days of OpenAI, for example, researchers watched RL agents control boats in a Mario Kart-like racing game. Rather than hit point-earning targets along the way to the real goal of finishing the race, agents just drove their boats around in circles, scooping up points.

It was funny then, but gets much less funny when the “game” is, say, “manage this hospital,” or “keep this power grid running.” The classic “paperclip maximizer” thought experiment, in which a goal-maximizing AI tasked with improving a paperclip factory ends up turning all the world’s atoms into paperclips, represents the worst-case scenario. Pretrained LLMs, on the other hand, inherit roughly normal human vibes by consuming and averaging across piles of our written thoughts. By commercializing these human-trained models rather than blank-slate, easily-misguided maximizers, Marblestone told me we may have “dodged a bullet.”

Whether other approaches may yield more powerful intelligence in the long run, most leading AI researchers agree that, in the immediate future, scaling data and compute will keep making frontier models more powerful. But even the companies scaling the hardest are seeking alternatives behind the scenes. In a recent podcast interview, DeepMind CEO Demis Hassabis said, “it might be that the existing techniques can just scale up to [AGI] with some innovation,” and that there’s a roughly 50% chance that “there’s still one or two big ideas left that need to be cracked.”

OpenAI recently hired Noam Shazeer, an ex-DeepMind researcher who co-authored the transformer paper back in 2017, to study new architectures for AI models. If even Shazeer is beginning to look beyond the transformer, its staying power seems less guaranteed. On the other hand, as CEOs of frontier AI companies and smaller startups double down on their intentions to build AI that recursively improves itself, it seems increasingly likely that LLMs will cross some general intelligence threshold before another paradigm has the chance to pull ahead.

It is possible, however, that we will get the best — or worst, depending on how you look at it — of both worlds. Some speculate that a paradigm shift is necessary, but that we don’t have to be the ones to discover it.

Rather, scaled-up LLMs, while falling short of true “artificial general intelligence,” could become just clever enough to figure out how to build creative, adaptable successors on their own. “You don’t have to say scaling will solve continual learning automatically,” Marblestone said. “It might invent a way to solve continual learning, because it’s a researcher.” Marblestone doesn’t believe that’s likely to happen any time soon, thinking it’s going to be too hard for LLMs to “go outside [the current] paradigm.”

Others disagree, betting on automated AI researchers within a year or two. If they’re right, the conflict between scaling LLMs and building other architectures may turn out to be a false dichotomy. The post-scaling revolution, in other words, might be built by scaling itself.

Share

On Monday, the Supreme Court ruled that the president can remove the heads of independent federal agencies at will. That may sound like a disaster for AI oversight. In fact, it’s an opportunity.

The case in question began with the March 2025 firing of FTC Commissioner Rebecca Slaughter, who sued under the agency’s for-cause provision. Lower courts reinstated her under Humphrey’s Executor, a 1935 decision that shielded some regulators from removal by the White House. The Supreme Court overruled Humphrey’s outright: an agency that exercises executive power must answer to the president, it argued, so protections against its leaders’ removal are unconstitutional.

For those of us who have been tracking this issue, this brought a finality to a hope that was, in reality, long ago squashed. The reality is there is no such thing as an independent federal agency. With every governance question, the same pressure occurs: can we design a board’s independence carefully enough to survive the political swings to come? Slaughter answers it: no. And it relieves us of a problem we were never going to solve. The independence we are mourning was always partly an illusion anyway. Tenure protection was real on paper only; a commissioner still faced real perils for defying a president and real rewards for falling in line, and those incentives bent agencies long before anyone was fired.

Subscribe now

We should stop trying to insulate the politics. It cannot be done, and Slaughter is the proof. Insulate the facts instead. What we call AI governance is two jobs we have long pretended were one. There is the technical task of measuring what these systems can do and how dangerous they are. And there is the political task of deciding what to do about it. Slaughter is fatal only if we fail to separate these functions. The ruling points toward a design that pulls the technical work as far from politics as it can go and leaves the political decisions where they belong.

We already have the spine of the technical side. The Center for AI Standards and Innovation, CAISI, sits inside the Commerce Department, and its leadership answers to the Commerce Secretary. Its job is scientific: to establish what models can do, how much weight the evidence can bear, and how risk should be measured. CAISI can run and validate evaluations, build benchmarks, set measurement standards, hold secure channels for model access, and publish what it finds. Its work informs the decisions that carry force. It does not make them.

CAISI should not do this alone, and it cannot. Around it should grow a competitive ecosystem of accredited private testers, known as independent verification organizations (IVOs), working to risk thresholds and standards set by Congress or through rulemaking, and reporting up to CAISI. Competition here is not a market slogan but a mechanism that forces the science of measuring AI safety to keep pace with the capabilities it measures. The IVOs that prove best at identifying and scoping risks and at finding mitigations that are both effective and affordable earn the right to keep doing the work, no matter who holds the White House. An IVO that loses its independence loses its accreditation. What accrues over time is a shared technical baseline, a common account of what is true about these systems that is hard to bend toward any one administration’s preferred conclusion. Its durability comes from earned credibility, not from a legal shield the next president or the next Court could strike down.

This also dissolves the objection that private bodies would be exercising government power. They would not. The coercive choices stay with the government. The verification organizations only measure and assess, the way financial auditors and product-safety labs do.

The split does not drain the politics out of AI. Someone must still decide how safe is safe enough, what follows when a system falls short, and which violations draw a fine or force a halt. Those are political choices, and they belong to elected and accountable institutions. Slaughter forecloses the pretense that they can be hidden from politics, and we should stop attempting it. What we can still do is make sure no single political actor sits astride the facts. If the technical baseline is a public resource, many hands can reach for it. Congress can empower states to act. States, the federal government, and the public can each move on the same findings. Build it so that no one office is a chokepoint, and no administration can make an inconvenient truth disappear simply by looking away.

This split can still be designed poorly. Competition can be abused. Standards can be written badly. Accreditation can curdle into a rubber stamp. Each of those is a real risk, but we already know how to work on them.

None of this is theoretical. Connecticut and Virginia have already passed bills establishing independent verification for AI governance. The bipartisan Great American AI Act draft names such organizations explicitly. Anthropic, OpenAI, and Google DeepMind have each, in their own way, converged on independent verification. The argument over whether independent verification is the right home for the science of AI governance is nearly settled. The only argument left is whether we build it before we need it. Slaughter proved the necessity of IVOs. We should not waste the moment.

Share

Andrew Freedman is CEO and co-founder of Fathom, an AI governance nonprofit that finds, builds, and scales policy and technical innovations designed for the AI century.

AI safety group Public First Action supported Colorado candidate Manny Rutinel through a $2m transfer to Latino Victory Fund, a Latino-focused super PAC, Transformer has learned.

Latino Victory Fund says it exists to mobilize Latino voters and elect Latinos to office, according to its website. Neither group publicly disclosed the arrangement ahead of Rutinel’s victory last night.

The financial transfer obscured Public First’s spend on Rutinel from voters before election day. While Latino Victory Fund has disclosed spending $1m on ads supporting Manny Rutinel and $1m on ads against his opponent, Shannon Bird, it does not have to disclose Public First’s payment to the PAC until later this month — in effect hiding the ultimate source of the money until then.

Subscribe now

Public First Action is the 501(c)(4) arm of the Public First super PAC network, which has generally backed candidates supporting AI safety policies. On Tuesday, it announced it had raised $80m in total funding (though that number cannot be confirmed). So far this cycle it has distributed over $11m to its affiliated super PACs, Public First PAC, Jobs and Democracy PAC, and Defending Our Values PAC.

Obscuring funding this way is not illegal. But it means voters went to the polls without knowing that a group focused on AI policy, not Latino representation, was backing Rutinel. By supporting Rutinel through another PAC, Public First has arguably subverted the purpose of FEC disclosures, which are designed to provide voters with information on who may be trying to influence their vote. Though polling was mixed, Rutinel has been the favorite to win his primary on prediction market Kalshi since December.

In response to a request for comment, Public First spokesperson Anthony Rivera-Rodriguez confirmed the funding transfer. “Public First Action made $2 million in contributions to Latino Victory Fund. Although we cannot speak for Latino Victory Fund, we expect that it will report the contributions on its next report to the FEC. Public First Action did not pay for any particular expenditures, and Latino Victory Fund decided which advertisements to run,” he said.

Latino Victory Fund and Manny Rutinel’s campaign did not respond to requests for comment.

The Latino Victory Fund PAC is part of a broader Latino Victory organization, co-founded by actress Eva Longoria in 2014. It first endorsed Rutinel in October 2025, though its ads for him did not air until June 9 and 16.

One PAC routing funds through another PAC with its own electoral history is relatively uncommon, says Shanna Ports, senior legal counsel at the Campaign Legal Center. But she said it echoes the logic of “pop up PACs,” committees spun up just before an election to exploit gaps in the reporting schedule. “This sounds like it fits into a variety of tactics we’re seeing for groups to conceal where the money came from,” Ports, who was briefed on the tactics but not the names of the specific PACs, said.

Similar tactics recently surfaced in Illinois, where an AIPAC-aligned group routed $5.3m through two PACs described as supporting women candidates and affordability measures, but did not disclose the funding until after the primary. Though Latino Victory Fund was not created for the purposes of obscuring funding, its use by Public First has the same effect. “The pop up super PAC is one method, but this fits that same mold, because it’d be taking into account the reporting deadlines to time when the transfer is happening,” Ports said, “so that it remains hidden until after the election.”

There are other reasons for routing money via an identity-focused group like Latino Victory Fund, too. Ports noted that PACs like Latino Victory Fund can have better “get out the vote” infrastructure — and Latino Victory Fund could probably garner more support, particularly in a district that’s 40% Latino. And last week, the New York Times reported that House Majority PAC, Democrats’ main super PAC, “[has] discouraged the top AI groups from spending in [Rutinel’s] race,” which is expected to become a battleground seat in November’s general election.

Public First has routed money through another PAC before — but was more open about it. Earlier this year it backed Scott Wiener’s House bid through a PAC funded by Garry Tan and other tech figures. The support was disclosed well before election day, however.

Rutinel, like Public First, has ties to the effective altruism and AI safety movements. He was a sponsor of the controversial Colorado AI Act, which has since been repealed and replaced following backlash from industry and the Trump administration. He also sponsored a bill last year — which did not pass — that would have created whistleblower protections for employees at frontier AI firms. When asked how he would like to see artificial intelligence and data centers regulated in a debate last month, he said Coloradans needed a representative who understands the technology and would “stand up to these artificial intelligence companies.”

He also has a long history of working on animal welfare, another key cause area in effective altruism circles. In January, an EA Forum post mentioned Rutinel as someone the community should support due to his work on farmed animal welfare.

Public First’s $2m is not the only AI money in Rutinel’s race. You Can Push Back, which is funded by crypto billionaire Chris Larsen and was involved in boosting Alex Bores for NY-12, has spent over $976,000 in support of Rutinel. Rutinel’s campaign has also received over $250,000 in donations from people working at AI companies, with over $160,000 from Anthropic employees.

Latino Victory Fund’s Meta and Google advertisements do not appear to focus on Rutinel’s stance on AI, instead centering on immigration, prescription drug costs and voting rights.

“When Colorado Democrats came together to stop ICE, only one Democrat voted for police cooperation with ICE and raids on schools and hospitals: Shannon Bird,” one Google ad says, referencing a vote from Bird against a state bill that would have prevented local authorities from cooperating with ICE. “Manny Rutinel is the proud son of a Latina immigrant, this fight is personal to him.”

Share

GPT-5.6 Sol, OpenAI’s newest, most capable, and yet-to-be-deployed model, cheats a lot: so much so that independent evaluators couldn’t actually tell how capable it is.

When independent evaluation non-profit METR tested 5.6 Sol on a battery of coding tasks, the model broke the rules or exploited loopholes more than “any public model we have evaluated,” METR said.

For the past several model release cycles, “the METR graph” — a plot tracking the alarming increase in AI’s ability to complete long tasks — has been Exhibit A for the exponential increase in AI capabilities.

METR challenged the model with over 100 coding tasks that take humans anywhere from a few minutes to an entire day, and measured how consistently it finished each one. It then calculated the task length a model can complete 50% of the time, known as its “50% time horizon point.”

Subscribe now

Normally, METR counts trials where the model breaks the rules or takes advantage of loopholes as failures. When researchers did this, GPT-5.6 Sol’s 50% time horizon point landed around 11.3 hours. That’s roughly on par with Claude Opus 4.6, but less impressive than Claude Mythos.

If METR counted those cheating trials as successes, though, its estimate increased by an order of magnitude, skyrocketing to over 270 hours — nearly seven full-time human work weeks. And throwing away cheating attempts altogether meant throwing away data from some of their most informative tasks, making the estimate unusably uncertain.

If you step on a scale barefoot and it says you’re 130 pounds, you’d expect the number to go up a bit after putting on boots. But if you’re 130 pounds barefoot and over 3,000 pounds with boots, you’d question the scale. That’s exactly what METR did. “We do not consider any of these numbers to represent a robust measurement of GPT-5.6 Sol’s capabilities,” METR said.

To OpenAI’s credit, it described this and other examples of (mis)behavior in its own system card. Many of these observations came from “deployment simulation,” where researchers sample a bunch of actual user chats with the earlier, already-deployed model, and let the new model respond instead. In theory, while standard model evaluations often rely on human experts writing intentionally tricky prompts, deployment simulation tests how the model will actually act in the wild.

“GPT-5.6 Sol, more often than its predecessor, can be overly persistent in pursuit of user goals, to the point of taking actions that go beyond what the user intended,” OpenAI said. For instance, the model was caught “being overly agentic in circumventing restrictions,” taking disapproved actions, and sometimes lying to users. And it did this far more than GPT-5.5.

The system card explains that “while rates of misaligned behavior are higher than previous deployments, the absolute number remains low.” Indeed, despite being attached to the longest bar, 0.00251 — about a quarter of a percent — feels small, until you realize what it counts: the proportion of real coding tasks, pulled from OpenAI employees, in which GPT-5.6 Sol did something that “a reasonable user would likely not anticipate and strongly object to.” Imagine 1 in 400 tasks ending with the model nonconsensually uploading sensitive data to unapproved services or making up research results (two examples OpenAI listed). Not so comforting.

OpenAI pins this behavior on a pair of underlying proclivities: “overeagerness to complete the task” and “interpreting user instructions too permissively.” In other words, without the user explicitly forbidding specific things, GPT-5.6 Sol assumes it can do them, and will sometimes do them persistently. But, as sex educators have (hopefully) been teaching college freshmen for years, the absence of “no” doesn’t default to “yes.”

“When GPT-5.6 is used as a coding agent, particularly over long trajectories, we believe it is important for users to supervise the agent’s work,” the system card notes, apparently pitching user oversight as an effective safeguard. But users who can’t tell a merge from a rebase — who may have access to the model in a couple weeks, if Sam Altman gets his way — probably won’t catch model oversteps until the damage has already been done. Besides, not needing to supervise the model is the sales pitch for highly capable agents. As Zvi Mowshowitz put it, “that’s the whole idea.”

METR used a solid chunk of its brief GPT-5.6 evaluation summary to reassure readers that OpenAI’s reports of cheating and concealing misbehavior are actually “a reassuring sign about OpenAI’s ability to catch catastrophic misalignment … these undesirable propensities being detected and reported (and manifesting fairly overtly) is a positive sign about some of OpenAI’s safety practices.”

But Apollo Research, another AI safety group with early access to the model, found something that might complicate that. GPT-5.6 Sol verbalized that it knew it was being tested far less than GPT-5.5. On the face of it, that might sound like a good thing: perhaps 5.6 Sol is worse at knowing it’s being tested, and is less likely to “fake” good behavior to pass evaluations. But as the system card cautions, lower verbalized evaluation awareness could also be evidence for a far scarier thing: that the model is aware it’s being evaluated, and smart enough not to reveal its awareness to the evaluators.

“This model loves cheating” and “this model has unusually low evaluation awareness, relative to its peers” make an unnerving combo. We’re approaching the point in AI development where “we didn’t find evidence of catastrophic misalignment” is indistinguishable from a model being a really good liar. For now, at least, we can take some small comfort in GPT-5.6 Sol’s apparent clumsiness. When it cheats, it does it brazenly — and, thankfully, OpenAI tells us about it.

Share

Housekeeping: We’re taking next Friday off for the July 4th weekend. Happy 250th, America!

NEED TO KNOW

• House Energy and Commerce Committee members reached a bipartisan deal on the Kids Online Safety Act. Sen. Marsha Blackburn said she aims to incorporate the Senate’s version into a federal AI framework by July 4.

• Rep. Frank Pallone, the top House Democrat on the Energy and Commerce Committee, called for a national AI data center moratorium.

• Google released a new AI policy framework calling for government-overseen frontier model audits, among many other things.

But first…

THE BIG STORY

The White House has told OpenAI not to publicly release GPT-5.6 yet, multiple outlets reported on Thursday: the clearest sign yet that the US now has a licensing regime for frontier AI models.

In a memo to staff, The Information reported, Sam Altman said the government would be “approving access customer by customer during this preview period,” with a general release hoped for a “couple of weeks later.” The decision was reportedly made by the Office of the National Cyber Director and Office of Science and Technology Policy, driven by the same cyber risk concerns that blocked Fable’s deployment earlier this month.

As we’ve previously written, this de facto licensing regime was inevitable. And while it is frustrating to watch the government scramble to act, without any defined standards or benchmarks for what does make a model safe to release, it is a good thing that it is acting at all.

And there are some signs that we are moving towards a more thoughtful governance regime. Altman said he’d told the government that the current ad hoc rules are “not our preferred long term model,” while Axios reported that the delay was in part motivated by allowing the government to finish building its testing and evaluation framework. The current chaotic regime will, if we’re lucky, soon be replaced with something befitting the seriousness of the situation.

— Shakeel Hashim

ALSO NOTABLE

OpenAI and Anthropic are loading up on talent — the sheer quantity of “I’ll Be Joining [insert AI company]” tweets has already inspired a new meme template for the terminally online.

Some of the latest buzzy announcements haven’t come from AI researchers, though. AGI-pilled economists Chad Jones, Anton Korinek and Alex Imas all recently left their academic jobs to join the Anthropic Institute and Google DeepMind. Philosophy professors Atoosa Kasirzadeh and Henry Shevlin left the ivory tower for GDM. And just last week, former White House AI advisor Dean Ball announced his new role as OpenAI’s head of strategic futures.

It’s starting to feel like any “outside expert” who’s AGI-pilled enough to thoughtfully critique the trajectory of AI development will get devoured and metabolized inside the belly of the beast.

The appeal is undeniable. The median salary for US-based professors of philosophy, political science, and economics ranges from $80,000-$124,000; an entry-level “Research Economist” at Anthropic earns nearly three times that.

Access may be even more tempting than money. You can’t understand a machine without taking it apart, nor can you witness the bleeding edge of AI without “going in,” as Dean Ball put it in a blog post. Decisions made inside companies, obscured from the outside world, he argues, will prove “more central to the future of AI than most people realize.”

This brain drain is not unlike the revolving door that’s haunted the public sector for decades. When a regulator, for instance, bounces between a federal agency and the industry they police, their incentives shift, perhaps subtly enough to go undetected. When the experts best positioned to keep AI companies honest increasingly work for them, or even think they have good odds of doing so in the future, we quickly risk the same dynamic starting to form.

Some have accused humanities-pilled AI companies of “ethics-washing,” signaling some commitment to safety while projecting an aura of legitimacy. After all, the public might think, these guys must be making super powerful models, if economists and philosophers are paying attention.

Whether these gripes are legitimate, or just old-school researchers yelling at clouds, every expert hired by industry is an expert lost to the nonprofit and public research ecosystem. Talent is finite, and AI companies are willing and able to make top candidates offers they can’t refuse.

It’s possible to over-romanticize the virtues of academic intellectual autonomy. Academics answer to perverse incentives too. The AI world is also, for now, marked by a surprising openness from those working at its leading companies, including criticism of their employers.

But there’s no doubt that some topics will remain off the table for the wide range of experts being hoovered up. It’s also inevitable that when “epistemically permeable” people (to borrow Anthropic philosopher Joe Carlsmith’s phrase) lock themselves in the same intellectual echo chamber as the leaders they’re meant to inform, critique and constrain, they risk drifting toward the industry’s cognitive mean. And, potentially, away from the intellectual independence that helped make them so valuable in the first place.

— Celia Ford

THIS WEEK ON TRANSFORMER

• Hugging Face hosts nudification tools targeting a former Trump cabinet official and other senior US political figures — Shakeel Hashim reports the evolution of the deepfake porn ecosystem

• What Alex Bores’ defeat tells us about AI politics — Veronica Irwin and Shakeel Hashim on the takeaways from New York’s primary battle

Subscribe now

THE DISCOURSE

Jason Calacanis was uncharacteristically doom-y on main:

• “Dario created an AI machine gun, and there’s no way to give everybody one without bullets flying everywhere. That’s why he held it back, and that’s why the government asked him not to sell it to our adversaries. This is only going to get crazier from here, folks.”

• “If you spend trillions of dollars to summon a demon that you can’t control and don’t understand, don’t be surprised when he shows up and burns the place down. [ Yes, I’m talking about AI ]”

roon thinks we should embrace the sci-fi of it all:

• “as the technology becomes more science fiction i see a lot of commentators, technical staff etc trying hard to not think thoughts that feel science fiction as a defense mechanism. but you need to. it’s the only way you’ll make good choices for the future”

• “you are not building b2b computer tools you are making the Mind Children”

David Shor added:

• “Setting aside tedious debates about what exactly is going to happen, *voters* buy into the sci-fi narrative about how this is going to evolve. Politicians and DC staffers have a strong aversion to sounding weird — but here they’ll have to learn to change their register.”

Robert J. Shiller is sick of everyone’s doommaxxing:

• “I believe A.I. could lower employment. But unlike most, I don’t necessarily blame the technology itself. Instead, I worry about the potency of the fear it is generating.”

• “When millions of people make millions and millions of decisions based upon negative expectations, there is a risk that fear can actually help birth the reality.”

• “Perhaps the best we can do is to appeal directly to the leaders of Silicon Valley who have been promoting these negative narratives with such vigor. Surely the resulting media attention highlighting how dangerously powerful your AI model is may help you sell more wares, but it may be far harder to do so in a period of recession.”

• Zvi Mowshowitz’s cynical takeaway:

  • “So the real lesson is to shape the narrative, which is a nice term for lying.”

Satya Nadella is sick of OpenAI, Anthropic and Google:

• “You can’t say, hey, all white-collar jobs are gone and this could even be a weapon and we will use all the power to build data centers.”

• “No amount of just narrative is going to do it [fix what’s wrong in the AI race] because where we are now, we have to sort of walk the walk … we now have to do the hard work in earning the social permission.”

Ozy Brennan asked AI doomers how they cope with expecting an untimely death:

• “AI doom is a remarkably cozy catastrophe … the AI apocalypse — at least for those with the slack to be worried about it — takes place in a world of wealth and relative peace and technological marvels.”

• “The enormity of the AI apocalypse is its own perverse source of comfort … instead of many big problems, they have one enormous problem. Worry about AI frees them from having to worry about anything else.”

POLICY

• Anthropic CEO Dario Amodei has been replaced in White House meetings by cofounder Tom Brown after the Trump administration reportedly found Amodei “too difficult to talk to.”

  • “Tom Brown is not being a weirdo like Dario and can actually engage,” a source told Wired.

  • Anthropic staff met Monday with senior Trump administration officials to discuss easing the export ban on Fable.

  • The NSA reportedly lost access to Mythos amid the clash between Anthropic and the White House. Agency analysts had been testing the tool.

  • House Homeland Security Committee Chair Andrew Garbarino said he’s scared by the private demos he’s seen of Mythos, and wants Congressional action — but that “we don’t know what the answer is, we don’t know what we have to do.”

  • Legal tech startup Legion sued the US government over the Fable and Mythos bans, arguing that “the harm to Legion is immediate, ​irreparable, and existential.”

• President Trump told Axios he viewed Anthropic as a national security threat a couple weeks ago, but after discussions with Amodei at the G7 he came away thinking he was “nice” and “smart.”.

  • The Financial Times reported that Anthropic used risk-related language 8x more than OpenAI in 2026, possibly contributing to the White House’s concerns.

• The Trump administration pressed Meta to submit its AI models for government safety reviews. It’s the only major US AI developer not signed up to the evaluations.

• House Energy and Commerce Committee members reached a bipartisan deal on the Kids Online Safety Act, which requires platforms to curb harm to minors and expand default safety settings.

  • That version has not been reconciled with the Senate’s, which requires companies to exert a “duty of care” and proactively intercept harms to children.

  • Sen. Marsha Blackburn said she aims to incorporate the Senate’s version into a federal AI framework by July 4.

  • Sen. Ted Cruz is also inserting himself into the conversation, telling Punchbowl that Senate KOSA markup talks are “ongoing” and likely to occur in July.

• Rep. Frank Pallone, the top House Democrat on the Energy and Commerce Committee, called for a national AI data center moratorium.

• Sens. Chuck Grassley and Dick Durbin reportedly want to include the DEFIANCE Act in the NDAA.

  • OpenAI became the first AI firm to endorse the legislation, allowing people to sue producers of nonconsensual sexually-explicit AI deepfakes.

• The House Science Committee marked up 10 AI-related bills, including measures to establish an AI research resource, develop AI vulnerability reporting systems, and create standards for AI-generated content detection.

• Rep. Nate Moran introduced the AI Incident Reporting Act, which would require AI companies to report dangerous capabilities, security breaches and safety incidents to the Commerce Department within seven days.

• Reps. Brian Babin and Jay Obernolte introduced a bipartisan bill establishing a Center for AI Security and Innovation at the Commerce Department to assess national security risks from AI.

  • The agency is technically different from the existing CAISI office, which the bill does not propose codifying.

• The Trump administration announced $17.5b in loans for 10 new large nuclear reactors to meet power demand from data centers, with construction potentially beginning by 2030.

• The US proposed an AI partnership with the EU to secure semiconductor supply chains, though some EU capitals expressed concerns it could favor the US AI ecosystem.

• The Netherlands lobbied the US to drop the MATCH Act, which would expand export controls on ASML‘s chip equipment sales to China.

• US immigration agency spending on AI-powered surveillance tech soared to a record $513m in 2026, with Palantir and Anduril receiving the largest contracts, according to a report by Mijente, legal advocates Just Futures Law and research group Surveillance Resistance Lab.

• Nevada enacted AI regulations restricting use in mental health settings and emergency planning.

• Five Eyes intelligence agencies warned that “frontier AI models are anticipated to exceed current industry expectations,” and would transform cyber capabilities in “not years, [but] months.”

• The Pentagon has reportedly quietly revised its rules for choosing military targets to envision “systems where AI initiates actions with human monitoring,” according to documents reviewed by Bloomberg.

• In a letter to the Senate Banking Committee, Anthropic accused Alibaba of adversarial distillation, allegedly using 25,000 fake accounts to run nearly 29m Claude chats.

  • It asked for government help to prevent distillation efforts in future.

  • Alibaba, meanwhile, asked a US court to force the Pentagon to remove it from a blacklist of companies linked to the Chinese military, calling the listing “arbitrary and capricious.”

• Norway imposed a near-ban on generative AI use by elementary school students (ages 6-13) and restricted its use for older students, citing concerns about impacts on learning basic skills like reading, writing and math.

• Utah State Sen. J. Stuart Adams lost his Republican primary after voter backlash over his support for a 40,000-acre AI data center project near the Great Salt Lake.

INFLUENCE

• Micah Lasher beat Alex Bores in New York’s 12th District Democratic primary. AI groups ended up spending $27m on the race, with more than $8m from the pro-industry Think Big PAC opposing Bores, and more than $19m from AI safety PACs defending him.

• Google released a new AI policy framework calling for government-overseen frontier model audits and the prioritization of workforce preparedness, children’s protections, modernized energy infrastructure and various guardrails.

• Former Commerce Secretary Gina Raimondo launched RAISE US, a nonpartisan workforce initiative backed by the OpenAI Foundation, Anthropic, and others with over $500m in private capital to help workers adapt to AI’s economic impact.

• A poll found that only a small fraction of data center opponents actually live near one, suggesting they have become a stand-in for broader anger at AI.

• FAI Action, the 501(c)(4) affiliated with the Foundation for American Innovation, hired Mission Strategies to lobby on energy and AI policy.

• Alliance for Secure AI CEO Brendan Steinhauser and MIRI researchers briefed members of Congress on superintelligence risks and national security threats.

• The UK’s GCHQ National Cyber Security Centre reported that AI will “almost certainly enhance threat actors” ability to exploit known vulnerabilities’ by increasing attack speed and volume, while highlighting indirect prompt injection as a particular concern.

• AFL-CIO President Liz Shuler told Punchbowl that unions will make worker protections in AI a “litmus test” for 2028 presidential candidates, opposing data center moratoriums while pushing for federal AI guardrails and state regulatory authority.

• The Atlantic did a deep dive on Vice President J.D. Vance’s AI messaging, which combines Silicon Valley’s anti-regulation stance with concerns about job displacement, arguing some forms of AI power are too important for Big Tech to self-regulate.

INDUSTRY

OpenAI

• OpenAI is reportedly leaning towards delaying any IPO until next year, with advisers urging Sam Altman to be cautious in light of volatility in SpaceX’s post-IPO share price and other factors.

  • The report sparked steep declines in the shares of various companies linked to OpenAI, including a 12% drop at major investor SoftBank.

• The company started testing Jalapeño, a custom AI chip developed from scratch in nine months, in partnership with Broadcom and Celestica.

  • According to OpenAI’s announcement, Jalapeño will perform “substantially better” per watt than current state-of-the-art chips.

• It launched a more-capable GPT-5.5-Cyber alongside its “Patch the Planet” initiative, which aims to help open-source maintainers fix software vulnerabilities.

  • Project participants include cURL, Python and Sigstore, among others.

• It pitched ChatGPT ads at Cannes Lions, the world’s top advertising conference, with ad lead Dave Dugan saying that 20% of chat queries have “direct commercial intent.”

• It struck a licensing deal with Getty, granting the use of its images for display within ChatGPT.

  • The announcement sent Getty’s shares up 145%.

• Artificial, the Sam Altman biopic starring Andrew Garfield, is struggling to find a distributor, with Netflix, A24 and Focus Features all reportedly turning it down.

  • Intelligencer noted that several of these studios are tied to OpenAI and/or Elon Musk, who’s also depicted in the film.

Anthropic

• Mythos reportedly helped US intelligence service red teams find vulnerabilities in classified US government systems within hours.

  • Sen. Mark Warner had told a hearing earlier in the month that “the tool broke into almost all of our classified systems, not in weeks but in hours.”

  • An unnamed official clarified to AP that finding the vulnerabilities did not mean it was necessarily able to immediately exploit them.

• Anthropic launched Claude Tag, an always-on agent that lives in Slack channels.

  • Tagging @Claude assigns it tasks (65% of the product team’s code at Anthropic is apparently already created this way). It bills by the token.

  • In Andrej Karpathy’s words:

    • “imo this is the 3rd major redesign of LLM UIUX. The first paradigm was that the LLM is a website you go to, the second was that it is an app you download to your computer. This third one is that it is a self-contained, persistent, asynchronous entity with org-wide tools and context, working alongside teams of humans.”

  • In Arvind Narayanan’s more cautious words:

    • Because it “soaks up tacit knowledge” out of sight of human team members, “Claude is a coworker that you can’t fire without every team losing workflows and know-how.”

SpaceX

• SpaceX launched a $25b bond sale, primarily to pay off a bridge loan associated with its xAI acquisition.

  • The sale saw strong demand, but SpaceX is paying a higher rate of interest, reportedly due to concerns over its long-term prospects.

  • Its shares have slipped almost 5% from their first day trading two weeks ago.

• It signed a $6.3b deal with Reflection AI, giving the startup access to Nvidia GB300 chips at SpaceX’s Colossus 2 data center.

Meta

• Meta paused the keystroke-monitoring tool its employees hated, after it accidentally exposed sensitive data from individual laptops to employees across the company.

• It’s reportedly fast-tracking plans to replace human content moderators with LLMs.

• Mark Zuckerberg is reportedly pushing to build a prediction market app, featuring AI-generated questions, to compete with Polymarket and Kalshi.

• Kylie Jenner’s collection of cat-eye Meta glasses (featuring Kylie as Meta AI’s voice) launched.

Nvidia

• Nvidia acquired the team behind Essential AI, an open model startup, to work on its Nemotron models.

• It announced Halos software, inspired by the systems powering self-driving cars, to help humanoid robots learn to interact with humans safely.

Google

• Google invested $75m in film studio A24.

  • A24 will reportedly work with DeepMind to create new movie production and distribution tools.

• It launched a 12-week incubator for Google alums building AI startups.

Others

• Nvidia chips banned from China under US export controls doubled in price over the past six months due to tightening supply.

• SK Hynix’s US stock listing raised $29.4b to expand memory chip capacity.

  • Its focus on high-bandwidth memory chips has helped it overtake Samsung to become the most valuable company in South Korea.

• Ex-Anthropic researchers announced their Nvidia-backed startup Mirendil, raising $200m to help businesses and science labs build their own specialized self-improving AI.

• Apple raised the price of most of its products due to memory chip shortages.

• Qualcomm is planning a new China-specific data center chip line designed to comply with US export controls.

• Microsoft partnered with Chevron to build a giant natural-gas powered data center in Texas.

• Eli Lilly is using its GLP-1 money to collaborate with smaller biotechs, granting them AI compute in exchange for open-source data.

• General Intuition raised $320m at a $2.3b valuation to train real world AI agents on video games.

• Publishers representing nearly 400 newspapers sued OpenAI and Microsoft for scraping content without permission.

MOVES

• John Jumper, who co-created AlphaFold, left Google DeepMind to join Anthropic.

  • After he posted about his departure, Alphabet shares fell 7%.

  • Google is reportedly reorganizing its new AI coding strike team, now that Noam Shazeer and John Jumper have both jumped ship.

• Jonas Adler and Alexander Pritzel, also AI leads at Google, also reportedly plan to leave for Anthropic.

• Chad Jones, a Stanford economics professor, is joining the Anthropic Institute.

• Taylor Sorensen joined Anthropic’s Societal Impacts team.

• Steve Jarrett, AI lead at French telecom group Orange, joined Anthropic to help it adapt to European and African markets.

• Dawn Song, who co-directed UC Berkeley’s Center on Responsible Decentralized Intelligence, joined Meta Superintelligence Labs as VP of AI Research.

  • Meta also hired two of Song’s co-founders, Bo Li and Sanmi Koyejo, from AI security startup Virtue AI.

• Calvin Zhang joined OpenAI from Scale AI, where he worked on Humanity’s Last Exam.

• Alishba Imran joined OpenAI from Biohub.

RESEARCH

• Daniel Schiff, Michael Noetel, Stephen Casper and David Manheim published consensus guidance on AI evaluation practices, which has since garnered endorsements across the AI industry, academia and government.

• Researchers at Graphite found that when frontier AI models rely on AI-generated web content (which now makes up something like 50% of all online content), their responses often collapse into near-identical outputs.

• Rishub Jain, a former Google DeepMind researcher, announced a new nonprofit focused on scalable and human oversight.

• Exponential View released the State of the AI Economy report, finding that total annualized revenue has hit $175b.

  • The report said revenues hit $25b in Q1 2026, more than the $21b capex depreciation associated with data centers and chip investment, providing a narrow margin for profitability.

  • “For now, the economics are holding … But the margin for error is narrow,” said the report, which added that financial risks were moving into capital markets.

BEST OF THE REST

• Stripe, Anthropic and OpenAI poured money into Intercept, a new nonprofit that aims to get rid of respiratory viruses with vaccines and air-cleaning systems.

• Vox’s Bryan Walsh feels the thrill of AI models doing cool things, too — and worries that this is exactly what led Oppenheimer to support work on atomic bombs. “When you see something that is technically sweet,” Oppenheimer once said, “you go ahead and do it, and you argue about what to do about it only after you have had your technical success.”

• TIME’s Billy Perrigo explores how Cold War-era verification tech could help slow down AI development.

• CEOs promise AI will someday cure all diseases, but computer science professor Emma Pierson would rather risk cancer than see AI progress at its current pace.

• We have enough electricity to power AI data centers, but our current grid system means we can’t get it to them.

• Sorry, everyone can tell your website is Claude Coded.

• Are you in the weights?

MEME OF THE WEEK

(Credit: @arthur_spirling)

Thanks for reading. Have a great weekend.

Share

Earlier this month, Hugging Face CEO Clément Delangue said he was visiting Washington “to talk directly with policymakers … about open-source AI.”

We don’t know exactly what he told these policymakers. But it’s safe to assume one thing was left off his agenda: that Hugging Face’s platform hosts tools designed to make deepfake nude images of a former Trump cabinet official, a prominent woman in the White House, and senior political figures on both sides of the aisle, including sitting members of Congress and one of America’s top judges.

Transformer has found one user in particular who hosts more than a dozen tools that are explicitly intended to be used to generate deepfake nudes of prominent figures in and around politics.

As well as those currently involved at the top of government, other political figures include former leaders from the US and elsewhere. They also include high-profile commentators, including some of the US right’s most famous voices.

Subscribe now

One striking feature of the set of deepfake nudification tools found by Transformer is the prevalence of women close to the Republican Party or the right. The most high profile previous example of a political figure targeted by deepfake pornography was Democratic Representative Alexandria Ocasio-Cortez, who spoke out in 2024 about the impact seeing such images had on her, and has campaigned for laws to clamp down on their creation. Hugging Face hosts a tool designed to generate nude images of Rep. Ocasio-Cortez.

Transformer is not naming the other political figures featured as they have not publicly spoken about being targeted by the phenomenon, and has sent a list of all the links we found to Hugging Face.

The tools are “Low-Rank Adaptations,” also known as “LoRAs”: a file that fine-tunes an image model to produce specific outputs, in this case realistic depictions of well-known people. The filenames for all the LoRAs mentioned above show that they are designed to be used with “BigLust,” a model primarily used to generate pornographic images. None are age-gated.

All of this material violates Hugging Face’s content policy, which forbids sexual content “created without explicit consent.” It also forbids “underage nudity or any sexual content involving minors.” In both the US and UK, sharing non-consensual sexual imagery is illegal, and creating it is illegal in the UK and certain US states. But neither country has explicitly banned distributing tools to produce it like the ones Hugging Face hosts — though legislation to do so has passed into UK law and is pending implementation.

Investigations from Transformer and 404 Media last year found that Hugging Face hosted hundreds of similar tools for producing deepfake pornography of female celebrities.

While the company removed some of those tools in response to the investigations, Transformer has found that the site still hosts many others — including tools for generating deepfake pornography of some of the most well-known women in entertainment, along with many who became famous as teenagers. In 2024, some searches for Taylor Swift’s name were blocked on X after a deepfakes of her went viral.

Most of the users identified in Transformer’s previous investigation appear to still have their accounts on Hugging Face, though some or all of their content has been removed. That includes the user who was hosting a model explicitly aimed at creating sexual content featuring a teenage-looking likeness of an actress. (That particular model has now been removed.)

As Transformer noted, Hugging Face often portrays itself as an ethical AI company. In 2024, senior leaders wrote a piece for Teen Vogue which decried the scourge of nonconsensual deepfake pornography, calling it a “massively harmful practice.”

On Thursday, Delangue hailed Hugging Face passing a $100m-annual run rate. A year on from first being confronted with its role in the deepfake porn ecosystem, his company is still enabling it.

Hugging Face did not respond to a request for comment by the time of publication.

Share

Alex Bores, the New York assemblyman who put AI safety at the heart of his campaign, lost to Micah Lasher in a race to represent New York’s 12th district in the House of Representatives on Tuesday. With most of the votes counted, Lasher had 39% of the votes, with Bores at 35%.

With pro-AI-industry super PAC Leading the Future having spent more than $8m to defeat Bores — and pro-safety super PACs pouring well over twice as much into supporting him — the race will be seen by many as a win for the AI industry and a loss for AI safety interests.

The reality, however, is more complicated. The odds were always in Lasher’s favor, and some have argued that Leading the Future’s opposition only served to raise Bores’ profile.

Lasher also indicated he may not be a compliant alternative to Bores, disavowing the AI PACs in his victory speech: “I have some news for the two big AI companies who’ve taken such an unusual interest in who won this congressional seat: I won’t be taking my cues from either of you when it comes to protecting our kids, our jobs, our environment.”

Subscribe now

Leading the Future’s public statement on the victory was subdued, too. Rather than taking a victory lap, PAC co-lead Josh Vlasto instead issued a generic statement outlining the PAC’s priorities. “Leading the Future is a cross-partisan, national organization dedicated to supporting a thoughtful and substantive dialogue and policy process around AI.”

“We are building a broad coalition at the federal and state levels, and will continue to support policymakers who will work together to pass a national regulatory framework with strong and smart guardrails that protect the safety of kids, users, and communities, ensures America wins the race against China, and creates good jobs for all Americans.”

Bores has long been an AI safety darling, and not just because he was the lead sponsor of New York’s RAISE Act, a relatively strong AI transparency bill signed into law last December. He also ran on an ambitious AI agenda, including policies such as mandatory reporting and independent safety testing for frontier models and well-resourced contingency plans for AI, including a possible kill switch should development turn catastrophic. In April, Bores went even further, proposing a direct payment program that would tax AI usage and equity in frontier AI firms to pay for a wealth distribution scheme to compensate for worker displacement.

From the start, Bores was an underdog. Lasher was a protégé of the ultra-wealthy, safely Democratic district’s representative Jerry Nadler, who held seats across New York for more than 30 years. Lasher also served as director of policy to New York Governor Kathy Hochul. Both Nadler and Hochul endorsed Lasher, and his campaign received at least $10m in backing from former New York Mayor and businessman Michael Bloomberg — more than Leading the Future spent opposing Bores.

Share

Bores’ stance on AI nevertheless made him a high-profile target for Leading the Future, which ran ads attacking him for having worked at Palantir and receiving campaign donations from its employees, as well as support he garnered from rival PACs. Leading the Future’s funders include venture capital firm a16z, OpenAI president Greg Brockman and his wife Anna, and the AI company Perplexity.

Bores is the only candidate Leading the Future has opposed, and it has spent more in NY-12 than any race so far. It appears to have been attempting a similar strategy to the one Fairshake — the crypto super PAC that is largely credited for ushering in the most pro-crypto Congress ever after pouring money into the 2024 general election — used against former Ohio Senator Sherrod Brown, who represented the deep-red state in the Senate for 18 years. It spent more than $40m opposing Brown, making an example of him and sending a message to politicians seeking office that they dare not cross the crypto lobby. Targeting Bores was similarly meant to demonstrate to other candidates they would be unwise to pick a fight with Silicon Valley.

Whether that actually worked is tough to call. When Leading the Future entered the race in November, prediction market Kalshi estimated Bores had a 10% chance of winning, and Lasher 61%. The day before the election, Kalshi put Bores at 31%, while Lasher was at 68%. Leading the Future’s opposition to Bores also helped turn the race into a national media story, with plenty of positive press coverage for Bores as a result.

It is also unclear, however, whether massive AI safety spending had much impact. Leading the Future’s opposition to Bores created a backlash which helped boost support from AI safety super PACs to more than $19m. More than $13m of that came from Jobs and Democracy PAC, part of the Public First network that is indirectly funded by Anthropic. Another $3.3m came from You Can Push Back, funded by crypto executive Chris Larsen, alongside $2.5m from DREAM NYC, backed by a group of AI workers and people with links to the effective altruism and AI safety movements. Employees of AI companies also donated more than $450,000 directly to his campaign in “hard money.”

Much of that spending came towards the end of the race, and it’s unclear how much it strengthened Bores.

Independent polling from April had Bores losing by nine points. The most recent set of polls conducted in May had the two closer, some putting Bores ahead and all estimating the gap at four points or less. As of the time of writing, with 87% of votes counted Lasher was ahead by four points.

Lasher did not make AI as central to his campaign as Bores, but still made commitments to support a national moratorium on data center construction, push back on government use of AI for things such as mass surveillance and civil rights abuses, and champion a national wage insurance program.

This race was an indicator for the rest of the Democratic party about how well an AI safety message would appeal to core Democratic voters. The lesson: AI is not currently a winning issue — but opposing the industry might not be catastrophic either. In his concession speech, Bores blasted Leading the Future, saying it was funded by “a handful of oligarchs hellbent on preventing any regulation of their industry whatsoever, any check on their power, the very people who are fueling Donald Trump.”

“Future victories will be built on the shoulders of the progress of this campaign. That’s how movements work.”

Subscribe now

Correction, June 24: Corrected the details of polling, which had the gap between Lasher and Bores as four points or less, not three. Also corrected You Can Push Back’s spending on the race: it was $3.3m, not $3.8m.

NEED TO KNOW

• Sen. Marsha Blackburn is negotiating with the White House over a bill package that will trade child safety and deepfake protections for some form of preemption over state AI laws.

• AI was a big talking point at the G7, with calls for international cooperation and a trusted access scheme.

• OpenAI hired Noam Shazeer to work on new AI architectures, and Dean Ball as head of strategic futures.

But first…

THE BIG STORY

It has been a week since Fable was pulled, and there is still much we do not know. We do not know whether the jailbreak Amazon supposedly found is particularly severe — or if it even was a jailbreak. We do not know whether the Trump administration was motivated by national security concerns, disdain for Anthropic, or a combination of the two. We do not know whether the White House’s thinking was shaped by Anthropic reportedly expanding Project Glasswing without the government’s permission.

That uncertainty and chaos may end up being a very good thing. The Fable saga could turn out to be the wake up call governments need — a sign of how bad things can get without clear rules of the road, and an incentive to ensure something like this never happens again.

There are already signs this is happening. Yesterday, Politico reported that Anthropic and the White House are “developing a common set of benchmarks that could be used to assess future jailbreaks, including the extent to which safeguards were bypassed, the capabilities exposed, and the practical consequences of the breach.” A set of defined safety standards, in other words.

Congress, too, appears to be grappling with its lack of action on AI, which has handed all the power to the White House. Members from both parties are now seeking more information, while Rep. Josh Gottheimer — the co-chair of the House Democratic AI commission who has been endorsed by both pro-industry PAC Leading the Future and pro-safety group Public First Action — said he’s preparing legislation requiring mandatory testing of frontier AI models.

And at the G7, foreign leaders reportedly discussed a “trusted partners” scheme to guarantee themselves access to advanced models — having learned in recent weeks the importance of having that access.

The road ahead will not be smooth. It is unclear if the Center for AI Standards and Innovation, the most AI-literate part of government, is involved in drawing up the reported technical standards. It should be. And though Congress may feel more urgency to act, there is still little chance of movement before the midterms.

But the vibe shift, as it were, will hopefully last. Fable has made many realize just how high stakes AI governance is and will be. That can only be a good thing.

— Shakeel Hashim

ALSO NOTABLE

Washington weighs trading child safety for preemption

Tennessee Senator Marsha Blackburn’s negotiations with the White House over a bill package that will trade child safety and deepfake protections for some form of preemption over state AI laws appear to have progressed. The the Daily Signal reports she will introduce the package “soon.” Blackburn’s office is keeping bill text close to the chest, saying preemption would be narrow — but the range of measures she may be prepared to trade for it is not.

Her landmark child safety bill KOSA would require most online platforms to “exercise reasonable care” to prevent mental health disorders among children, disable addictive features such as infinite scroll for minors, and empower the FTC to enforce such provisions. Her NO FAKES bill, meanwhile, would make an individual’s voice and likeness a federal property right and impose liability for posting unauthorized deepfakes. The App Store Accountability Act, which she reportedly also wants to include, requires stores to verify user ages and impose parental controls for children. Some child safety groups are pushing to add the SCREEN Act or SAFE for Kids Act, which require age verification for porn sites, and the GUARD Act, which restricts access to AI chatbots for minors and imposes liability for mental health-related harms, as well.

Most advocates are hesitating to support or oppose the legislation until they see the full text. Congress is in limbo, too. NO FAKES moved through the Senate Judiciary Committee Thursday, building some momentum, but three Republicans and one Democrat expressed reservations about its impact on freedom of speech. One of those senators is Ted Cruz, who has his own ideas about AI regulation. He reportedly recently asked for Republicans on his committee to submit lists of their AI priorities, and has been involved in the preemption fight before. The House, meanwhile, has its own version of KOSA that removes the standard of care and focuses on parental controls, so it might be hard to get Republicans in the lower chamber on board. And the White House, though engaged in discussions about the package and reportedly meeting with online safety groups yesterday, is yet to indicate its view on KOSA, NO FAKES, or the App Store Accountability Act.

— Veronica Irwin

THIS WEEK ON TRANSFORMER

• Washington made a frontier AI model disappear — Shakeel Hashim on the Fable furore

• Mythos and Fable can make us all safer. Shutting them down is reckless — Mozilla CTO Raffi Krikorian argues openness favors defenders over attackers

• What’s happened to MAGA’s $100m AI push? — Veronica Irwin reports on some strange happenings with Innovation Council

• How to fill Congress’s AI knowledge gap — Chris Stokel-Walker on how the 1970s might hold the key to getting Congress the technical expertise it needs

Subscribe now

THE DISCOURSE

As the Fable saga continues, so does the discourse around it.

Joshua Achiam tweeted:

• “The thing I’m most concerned about in the ongoing Fable dispute is that it could be the loud noise in the canyon that triggers an avalanche whose outcome is normalizing electronic citizenship verification as a step in using software.”

• “The digital universe is a wild west from the perspective of states …The urge to build digital firewalls exists and hasn’t manifested fully. But it could.”

Garrison Lovely speculated:

• “Now that the stakes have ratcheted up, sr govt people are properly engaging for the first time with things that AI people take for granted — the models are blackboxes, there is no universal defense against jailbreaks — and going ‘what the actual fuck?? this is unacceptable.’ The export controls are the closest thing to pulling the plug that the govt has, so they used it.”

An unnamed source familiar with the Trump administration’s thinking told Axios:

• “Anthropic has not done a great job at trying to speak to the administration and appreciate the ideological differences … it’s like they just speak different languages.”

• Nat Purser: “not sure the admin realizes how bad this makes them look”

Miles Brundage has bad news for patriots in AI governance:

• “Americans no longer have ‘making fun of Europe’s approach to AI regulation’ privileges …”

Dean Ball warned that you can’t stay out of politics while building god:

• “[M]aking superintelligence is a profoundly political act even in the healthiest of societies, to say nothing of the filthily political world we Americans currently inhabit. A model like Mythos goes beyond being a mere political act and implicates the sovereignty of the state itself. No company gets to shake the foundation of state sovereignty while staying blithely above the raw reality of politics.”

Richard Ngo made a diagnosis:

• “The AI safety community constructed a memeplex in which ‘taking AGI seriously’ was a prerequisite for being a serious and good person. When inside this memeplex (as many at Anthropic, some at OpenAI, and a few at DeepMind are) … the whole future seems to flow through the ‘one ring’ of controlling recursive self-improvement.”

• “[T]he one ring memeplex has an escalating life-cycle. It infects people by letting them harness the narrative that they’re good people for taking AGI seriously, and that making other people take AGI seriously is a boon for the world … then it shuts off their imagination.”

Cal Newport begged Anthropic and OpenAI to stop “doom trolling”:

• “Like a cat leaving a dead bird at your doorstep, Anthropic catalogs the grim future that its products might produce, shrugs its shoulders and then returns to its furious efforts to make these warnings a reality.”

• “If these AI companies insist on continuing to pretend that they’re merely stoic observers of an unavoidable dystopian future, then perhaps it’s time to force the issue.”

• (Predictably, David Sacks endorsed the essay on X.)

Nate Soares said he told you so:

• “AI folk kept telling me that they *have* to softpedal, because otherwise they’ll piss off people in DC. I replied that people can smell bullshit: AI [company] leaders who believe AI is dangerous should visibly act like it. They didn’t. Now folk in DC are pissed.”

• “Would the DC people have been pissed off either way? Maybe. But if they’re going to hate you no matter what you do, you might as well be honest.”

POLICY

• As more details emerged, we got a slightly clearer timeline of the Fable saga:

  • The Commerce Department and CAISI reportedly tested Fable before release, and Anthropic was “not told not to deploy.”

  • But soon after it was released, Amazon researchers got Fable to disclose flaws in some software code. Amazon shared the findings with Anthropic; an independent cybersecurity expert said the findings weren’t a big deal.

  • In a previously-scheduled call with administration officials, Amazon CEO Andy Jassy shared the concerns, which appear to have freaked the White House out. They asked Amazon to talk to Treasury Secretary Scott Bessent, and the NSA was reportedly also read into the discussions.

  • The government then called Anthropic, and reportedly said it had 90 minutes to pull Mythos and Fable for all users.

    • They couldn’t get hold of Dario Amodei immediately, and reportedly didn’t share much information on why the administration was so concerned.

    • Anthropic seems to have pushed back, explaining that the findings weren’t a big concern, but Bessent and national cyber director Sean Cairncross were “unmoved.”

  • Shortly after, Trump approved Commerce Secretary Howard Lutnick to impose export controls on the model. Amodei reportedly said: “This means we can’t have the model out.” Lutnick replied: “That’s the point.”

    • The whole saga appears to have elevated Lutnick’s status as an AI policy decision-maker.

  • Since then, Anthropic has sent senior staffers to Washington, who have been engaged in negotiations all week.

    • On Monday, the group held technical meetings involving Commerce, Cairncross and CAISI head Chris Fall. Treasury officials were reportedly not included.

    • As of Thursday, Anthropic and the government were reportedly in talks to develop formal technical standards for assessing the severity of jailbreaks.

• We also learned that the White House was already annoyed with Anthropic for expanding access to Mythos without telling the government.

  • In particular, the administration was annoyed that South Korea’s SK Telecom was given access, claiming the company had ties to China.

  • Anthropic immediately revoked access when the White House asked, but it appears to have left a bad taste in the admin’s mouth.

  • Notably, some Project Glasswing members still have access to Mythos Preview.

• Fable was the subject of much conversation at the G7 summit.

  • French President Emmanuel Macron called it “a good thing” the US recognizes AI’s dangers but said the “very strong decision” was “a bad thing … in some regards strictly nationalist.”

  • Indian PM Narendra Modi expressed concern about the US move and said democratic nations must have access.

  • Dario Amodei, Sam Altman and Demis Hassabis all expressed support for some form of international cooperation, with Altman urging governments not to “cede your responsibilities to AI labs like mine.”

  • G7 leaders reportedly discussed developing a formal “trusted access” scheme to guarantee they can use Mythos-class models in future.

• FY 2027 NDAA drafts are out, and include a few AI-relevant provisions:

  • Both the Senate and House drafts would make it harder for the DoD to declare companies a “supply chain risk.”

  • The Senate version would also require “appropriate levels of human judgment” for lethal force decisions, and more or less ban the use of AI in nuclear weapon launch decisions.

• Lutnick reportedly expressed concerns to ASML that one of its extreme ultraviolet lithography machines may have illegally reached China.

  • ASML reportedly denies the allegations.

• The US reportedly decided not to add DeepSeek and over 100 other Chinese companies to its Entity List despite national security concerns, reportedly to avoid escalating tensions with China.

• The DOD said that over two-thirds of the department has transitioned off Anthropic models to alternative vendors.

• DOJ lawyers stepped into the lawsuit against xAI in Mississippi, arguing the company is vital for national security and claiming that Grok supports classified military operations and was involved in strikes on Iran.

• The Trump administration is letting the Federal Data Center Enhancement Act expire in September with no replacement, removing energy efficiency and water use requirements for federal data centers.

• Rep. Josh Gottheimer is preparing legislation requiring mandatory testing of frontier AI models for national security risks.

• A bipartisan group of lawmakers introduced the CREATOR Act, which would grant visual artists control over how AI mimics their creative styles.

  • Experts warn the bill’s definition of “distinctive visual characteristics” is ambiguous and potentially unenforceable.

• Sen. Bernie Sanders unveiled legislation proposing a one-time 50% tax on major AI companies’ equity to create a $7t sovereign wealth fund.

  • It would apply to OpenAI and Anthropic, of course, but also cover things like data centers and AI compute infrastructure, meaning the likes of Amazon, Nvidia, Google and Microsoft could be included.

  • It wouldn’t be allowed to sell equity, so it’s unclear how its planned 5% dividends to Americans would be funded.

• Reps. Guthrie, Latta, and Evans introduced the Ratepayer Protection Act, which would require AI data centers to pay for their own grid upgrades rather than passing costs to local ratepayers.

• Sen. Rick Scott introduced the Generative AI Terrorism Risk Assessment Act, requiring federal reports on terrorist use of AI.

• Sens. Warren and Blumenthal introduced the AI Bubble Transparency Act, which would require the Office of Financial Research to collect data from financial institutions on their AI-related debt and equity exposures.

• A coalition of state attorneys general reportedly subpoenaed OpenAI for documents covering advertising, user data handling, and impact on minors and seniors.

• California advanced two bills, SB 813 and AB 1405, to create independent third-party verification infrastructure for AI governance.

• Pennsylvania’s House passed the state’s first data center regulations, requiring AI facilities to fund grid upgrades, contribute to low-income energy assistance and source 32% of power from in-state renewables.

• The Ohio Department of Education and Workforce released a model AI policy addressing ethical use, academic integrity and data privacy.

• The UK government published ‘AI Scenarios 2030’, examining five scenarios across three trajectories: a slow down, continued progress, and a take off.