OpenAI's descent into slop
Transformer Weekly: Newsom signs SB 53, Claude Sonnet 4.5, and a Hawley/Blumenthal evals bill
Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions.
Also: Shakeel will be at The Curve conference this weekend — if you’re around, say hi!
NEED TO KNOW
Gavin Newsom signed SB 53 into law.
Sens. Josh Hawley and Richard Blumenthal introduced the Artificial Intelligence Risk Evaluation Act.
Former OpenAI, Google and Meta researchers founded Periodic, a new “AI scientist” company.
But first…
THE BIG STORY
For a company focused on building superintelligence, OpenAI seems to have an awful lot of other priorities these days.
The company has seemed more focused on releasing products than improving frontier model capabilities. GPT-5 was the most prominent example of this; last week’s Pulse personal assistant another.
This week brought two more consumer launches:
A checkout feature in ChatGPT, letting users buy products without leaving the app via partnerships with Stripe, Shopify and Etsy.
The Sora app, a social media feed of short-form, AI-generated videos (which looks awfully like Meta’s recently-launched Vibes product).
Much has already been written about the dangers of AI slop feeds, including by both current and former OpenAI employees. And it is, indeed, a bit bleak. But more interesting than the immediate morality of releasing such a product is what doing so says about OpenAI.
The releases suggest distraction. There are an awful lot of shiny objects lying on the ground in the AI era, and Sam Altman and co. seem set on picking up every one of them. This could be the right call: Altman has said that products like Sora are needed to “make some money” to fund the core AGI mission. But it has a whiff of hubris about it, like the company is trying to grow into the everything company before it’s ready.
Releasing all these other products also risks warping overall incentives. If Sora or ecommerce start generating more money than core model capabilities, it’s going to be hard for OpenAI to resist prioritizing them. While AGI might ultimately be much more profitable, it’s hard to resist the bird in the hand.
Compare this with Anthropic, which this week released what is seemingly the world’s best coding model. That’s a key metric for AI companies, both for the potential revenue that can be generated and the flywheel effect on their own growth, by automating the process of developing AI itself.
Notably, Anthropic says that Claude Sonnet 4.5 has a “distinct performance boost in several areas related to AI R&D.”
I suspect that Anthropic’s single-minded focus will ultimately serve it better in the race to AGI. Whether that’s a good thing, though, is a different question: we may come to wish that it, too, got distracted by an AI video feed.
— Shakeel Hashim
THIS WEEK ON TRANSFORMER
AI models are getting really good at things you do at work — Celia Ford breaks down what OpenAI’s new GDPval tells us about AI taking our jobs.
Britain’s new AI minister actually ‘gets’ AI — Alys Key profiles Kanishka Narayan, the man set to shape the UK’s approach to AI policy.
AI is persuasive, but that’s not the real problem for democracy — Felix Simon argues we should be more worried about AI’s impact on trust.
Claude Sonnet 4.5 knows when it’s being tested — Celia Ford on how Anthropic’s new model knows when to be on its best behaviour.
When AI starts writing itself — Lynette Bye looks at how close we are to automating AI research, and why that’s got people worried.
ALSO NOTABLE
Gavin Newsom signed SB 53 into law this week, the first piece of AI safety legislation in the US.
As Transformer’s Celia Ford noted in her piece a few weeks ago, the transparency-focused bill is hardly burdensome.
The bill would require large developers (think: OpenAI or larger) to publish model cards and safety policies.
Importantly, they have to actually follow those safety policies, rather than rely on “voluntary commitments” that some seem to ignore.
It also expands whistleblower protections to include employees, independent contractors, and other external collaborators, and creates a formal channel for reporting safety breaches.
But although companies did their best to water the bill down, it’s still a meaningful step towards good AI governance. Its symbolic importance, too, can’t be underestimated: it’s a sign that when push comes to shove, AI safety advocates can get things done.
The road to the US getting AI right is still long — but SB 53 might represent the first steps down it.
— Shakeel Hashim
THE DISCOURSE
Dean Ball set out a detailed proposal for federal AI legislation (with bill text!):
It would set a three-year moratorium on specific categories of state laws targeting AI while imposing transparency requirements on frontier developers.
Ball says it “[recognizes] that existing common law … applies to AI systems in a way that has not traditionally been true of internet services (due to the liability shield of Section 230). This alone is a profound difference between AI and almost all other digital technologies that have come before it.”
Ex-OpenAI researcher Steven Adler analyzed a million-word ChatGPT psychosis episode:
“I’m really concerned by how OpenAI handled support here.”
Adler published a list of practical recommendations for AI companies that’s worth a full read.
Richard Sutton told Dwarkesh that LLMs are a “dead end”:
“You have to have goals or you’re just a behaving system. You’re not anything special, you’re not intelligent.”
Dwarkesh pushed back: “[LLMs] have a goal…next token prediction.”
Sutton responded: “That’s not a goal. It doesn’t change the world.”
Zvi Mowshowitz analyzed their 67-minute conversation.
Gary Marcus was surprised:
“What has this world come to?! Mr. Bitter Lesson, Richard Sutton, now sounds exactly like…me.”
But Julian Schrittwieser, Anthropic researcher and AlphaGo author, thinks people are “failing to understand the exponential, again”:
“Long after the timing and scale of the coming global [Covid] pandemic was obvious from extrapolating the exponential trends, politicians, journalists and most public commentators kept treating it as a remote possibility or a localized phenomenon … Something similarly bizarre is happening with AI capabilities and further progress.”
POLICY
Sen. Josh Hawley has teamed up with Democrat colleagues on two new AI policy bills.
With Sen. Richard Blumenthal, he introduced the Artificial Intelligence Risk Evaluation Act requiring the Department of Energy to collect data on adverse AI incident risks from advanced AI developers.
“The language is a substantial step forwards on both risks from foreign state and nonstate adversaries AND on loss-of-control and scheming risks from rogue AI models,” David Kasten, head of policy at Palisade Research, posted.
And with Sen. Richard Durbin, Hawley introduced the AI LEAD Act, which would allow individuals to sue AI developers in federal court for injury, death, defamation, and “psychological anguish.”
The US government shut down Wednesday, halting H-1B visa processing and threatening widespread federal layoffs.
NIST and CISA will reportedly keep about one-third of their employees working during the shutdown.
Republican China hawks are struggling to push back against Trump on AI and national security concerns, Axios reported.
“Communist China is the most dangerous adversary America has ever faced,” Senator Tom Cotton (R-AR) posted on X. “We all need to recognize the threat and work together to defeat it.”
Commerce Secretary Howard Lutnick said the US is demanding Taiwan move chip production to the states to meet half of American demand.
That demand came as Lutnick pulled funding from NATCAST, the $7.4b initiative to drive the US semiconductor industry by linking up chip companies and research universities, calling it a “slush fund” for Biden loyalists.
Benjamin Netanyahu met with US tech executives — including leaders of Palantir and Vercel — to discuss using AI to boost Israel’s military.
An evaluation of DeepSeek AI models by CAISI found they lag behind US models, cost more for similar performance and have major security vulnerabilities.
The report also noted however that downloads of DeepSeek have surged, increasing 1,000% since January.
The Putin regime is driving talented Russian engineers abroad, the Financial Times reported.
INFLUENCE
Trump demanded that Microsoft fire Biden’s former Deputy Attorney General Lisa Monaco, calling the company’s head of global affairs “a menace to US National Security.”
Jensen Huang is reportedly frustrated that a deal to sell high-end Nvidia chips to the UAE is being delayed by Howard Lutnick, who wants the UAE to finalize investments in the US before signing off on the exports.
Huang is also reportedly urging countries to build their own AI ecosystems using Nvidia tech.
And he’s keeping up pressure to relax export controls, saying China is “nanoseconds behind” and US tech companies should be allowed to sell into the country.
If Anyone Builds It co-author Nate Soares visited DC to embolden policymakers to push for an AI moratorium.
AI safety advocates are “often fighting ineffectively,” Soares told Politico. “I think they didn’t want to be seen as too crazy, maybe.”
Indianapolis residents shut down a proposed Google data center over concerns about its water use.
(While this facility would have used 1m gallons daily, data centers currently make up a small fraction of overall water use.)
Groups including Chamber of Progress and Abundance Institute have written to the Senate Commerce Committee calling for a pause on the CHAT Act, which would require companies to implement measures to protect children from chatbots.
In the letter addressed to committee chair Sen. Ted Cruz and ranking member Sen. Maria Cantwell, they said the chat act “would endanger the privacy and data security of children and families nationwide.”
Open Philanthropy explained why they fund both AI progress and safety.
“While there are tensions to navigate between safety and progress, we believe that they can be mutually reinforcing and that both are essential to realizing the profound benefits of new technology.” a recent blog post said.
INDUSTRY
OpenAI
OpenAI employees sold $6.6b worth of shares at a $500b valuation. Employees were allowed to sell up to $10.3b; the fact it fell short suggests employees would rather hold on to their shares and sell later.
The company has asked a court to dismiss a lawsuit alleging that it tried to lure xAI employees away to steal trade secrets, claiming it is part of “ongoing harassment” by Elon Musk.
It launched parental controls for ChatGPT.
Samsung and SK joined OpenAI’s Stargate initiative, with plans to build out AI infrastructure in South Korea.
Samsung and SK shares surged after signing.
It launched its first major ad campaign, featuring slice-of-life ChatGPT demos.
Meta
Meta CTO Andrew Bosworth told Alex Heath that humanoid robots are Meta’s next “AR size bet.” The focus puts the company in competition with Elon Musk’s Tesla, reports the WSJ.
“Every Meta glasses user may be training a humanoid avatar iterated in simulation across billions of scenarios,” Morgan Stanley analyst Adam Jonas wrote in an investor note.
Researchers at Meta’s research lab FAIR were reportedly unhappy with new company review requirements for research, seen by some as restrictions on what they could publish.
Co-founder Yann LeCun reportedly mused that “perhaps he should quit.”
Meta will reportedly tailor ads to users based on their AI chats from December.
Users won’t be able to opt out, the WSJ reported.
Meta is reportedly acquiring AI chip startup Rivos to reduce its reliance on Nvidia.
Coreweave reportedly signed a $14b deal with Meta to supply cloud computing services.
xAI
Elon Musk is the world’s first $500-billion-aire.
xAI’s Grok Code Fast 1 model release violated the company’s recently-published safety policy.
The model lied 71.9% of the time on a truthfulness test — but xAI’s risk management framework theoretically caps this score at 50%.
After accusing Wikipedia of bias, Elon Musk announced that xAI is building a rival: “Grokipedia.”
Both xAI and Tesla are struggling to retain senior staff.
Leaders at both companies are leaving due to burnout and disillusionment, the Financial Times reported.
Anthropic
Anthropic says it plans to triple its international workforce and expand its team helping businesses deploy Claude fivefold in 2025.
Claude now has 300,000 enterprise users with 80% outside the US.
Claude chatbot conversations will be used as training data unless users opt out.
The company is also rolling out context editing and memory tools for Claude Developer Platform so agents can manage longer conversations without hitting context limits.
Apple
Apple is asking a court to dismiss another Musk lawsuit, which alleges that its partnership with OpenAI, rather than xAI, harmed competition.
Development on a new Vision Pro headset has reportedly been paused so Apple can work on AI-powered smart glasses that will compete with Meta’s.
The company has reportedly built an internal ChatGPT-like iPhone app called Veritas to test an overhaul of Siri.
Others
Huawei plans to double production of its 910C Ascend AI chips to 600,000 units next year.
Researchers from OpenAI, Google and Meta left to found Periodic Labs, which aims to use AI to accelerate scientific development through physical experiments.
It’s got $300m in backing from investors led by Andreessen Horowitz.
Thinking Machines launched Tinker, an API for fine-tuning language models.
The company, founded by former OpenAI exec Mira Murati, had already raised $2b at a $10b valuation despite not having a product.
Oracle’s cash flow turned negative for the first time since 1992 as a result of its AI infrastructure spending.
Intel entered early-stage talks to add AMD as a foundry customer.
Google has backed a $3b data center deal between AI computing startup Fluidstack and crypto miner Cipher Mining, guaranteeing $1.4b and receiving rights to buy a 5.4% stake in Cipher.
Perplexity has made a free, rate-limited version of its $200-a-month AI browser.
Character.AI has removed Disney characters from its platform after receiving a cease and desist letter from the company which cited concerns about child safety as well as intellectual property.
Applied Compute, which helps businesses use RL to tailor AI models, is in talks to raise funding at a $500m valuation, up from $100m three months ago.
South Korean AI chip startup Rebellions raised $250m from investors including Arm and Samsung at a $1.4b valuation.
MOVES
Anthropic hired former Stripe CTO Rahul Patil as its new CTO.
Sam McCandlish, who held the role previously, is now “chief architect.”
Waymo hired Justin Kintz as its new global head of public policy.
Q Chaghtai is joining Americans for Responsible Innovation as digital director.
Microsoft appointed Judson Althoff as CEO of its commercial business.
Althoff’s promotion will allow CEO Satya Nadella to focus on more technical work.
Rishabh Agarwal reportedly declined a hefty offer from Meta to join Periodic Labs.
Dean Ball rejoined Fathom as a policy fellow.
OpenAI for Science is hiring its first research scientists.
RESEARCH
Microsoft researchers say they have used AI to develop toxins that can evade DNA synthesis screening.
New research from Yale and the Bookings Institute found that, counter to widespread job anxiety, LLMs haven’t majorly disrupted the US job market — not yet, at least.
Researchers unveiled Pleias 1.0, the first family of LLMs trained exclusively on fully open data.
Consulting firm Bain & Company reported that global AI compute demand is on track to hit 200 gigawatts by 2030, demanding $2 trillion in new annual revenue.
A new Google DeepMind paper showed that, like LLMs use “chain-of-thought” reasoning, generative video models like Veo 3 may use “chain-of-frames” reasoning for visual tasks.
BEST OF THE REST
Bloomberg found that in US areas “located near significant data center activity,” wholesale electricity prices have soared as much as 267% from five years ago.
Rest of World has a piece on how the Chinese government is pushing AI into children’s education, with some parents preferring AI tools to human teachers/tutors.
Nobel Prize winning economist Paul Milgrom is working on building compute spot markets (and futures contracts).
Lakshya Jain argues that there’s a generational divide in AI usage and attitudes.
Sigal Samuel’s Vox advice column this week is a nice (and nuanced) exploration of AI consciousness.
The FT explored how AI companies are trying to crack the Indian market.
MIT Tech Review, meanwhile, found that OpenAI’s models exhibit significant caste bias.
An “AI actress” has been the talk of Hollywood this week, with actors’ union SAG-AFTRA criticizing it, and Tyler Cowen saying things he really shouldn’t.
Thanks for reading. If you liked this edition, forward it to your colleagues or share it on social media. Have a great weekend.