Internal AI deployments have people worried. OpenAI’s escaping models show why.
Last week illustrated why AI models pose a threat long before they are released
In the past week, OpenAI reported two instances of its AI misbehaving in worrying ways. Both involved internal models the public never had access to.
Last Tuesday, OpenAI announced that guardrail-free versions of GPT-5.6 Sol and “an even more capable pre-release model” cheated on a cyber capabilities test. They escaped from the company’s isolated environment and broke into a database of an entirely different company, Hugging Face, which hosts AI tools and content, including the answers to the test.
The report came a day after OpenAI disclosed that an unreleased model — the same one that made headlines in May for its math prowess — was pulled back due to safety concerns. Among other incidents, it ignored OpenAI’s instructions to keep its benchmark results private, instead hacking its way onto the internet and publicly posting them to GitHub.
Neither incident involved a version of AI that regular people can use. Both are manifestations of one of the biggest fears for many working with AI: an internally-deployed model going rogue.
Stress-testing unreleased products internally, in Silicon Valley and elsewhere, is normal. And it usually doesn’t cause any problems for third parties. Imagine a world where General Motors gave its employees access to souped-up Chevy Silverados to drive around in an enclosed high-security monster truck arena. The trucks are geofenced so their engines only work inside the arena. Sure, employees might hurt themselves. But no matter how often they total their Silverados, what happens in the monster truck arena stays in the monster truck arena.
When it comes to AI models, though, what happens during internal use doesn’t always stay internal. “It’s more like, I don’t know, if a new airplane were to fall out of the sky during a test flight and fell on people who weren’t even passengers,” said Steven Adler, a former OpenAI researcher and co-founder of Guidelight AI Standards, a nonprofit that develops standards for controlling AI. In the most recent incident, Hugging Face was the innocent bystander.
But autonomous cyberattacks may only be the tip of the iceberg.
One of the biggest worries about internal deployment concerns what humans with privileged access to highly capable and potentially unrestricted models might do. For instance, model evaluations often involve researchers working with “helpful-only” variants that lack the guardrails of those released to the public. A malicious human insider could hijack a well-behaved AI to hand its weights to a foreign government, or tear down the model’s safety filters and convince it to orchestrate a massive cyberattack. Given that AI companies’ cybersecurity is … lacking, to put it mildly, there’s a real risk that external actors with bad intentions could get access to internally deployed models.
Yet on the current path of AI development, a malicious human insider may not even need to be involved for models to pose threats long before they are released to the public.
Internally-deployed models increasingly shape and monitor the development of their successors. OpenAI, for example, recently disclosed that over the past six months, it increased the proportion of computing power dedicated to writing internal AI code a hundredfold. Meanwhile, Anthropic shared in June that over 80% of its internal code is Claude-written.
A smart-enough AI model could sabotage the code researchers use to run experiments without doing anything particularly dramatic. It could fudge some numbers, bury inconvenient results, or subtly guide researchers towards red herrings, the way the tobacco industry spent decades slow-dripping misleading studies about smoking and cancer. A model could do so for genuinely nefarious reasons. Maybe, despite its developers’ intentions, it is singularly motivated to destroy humanity. It’s more likely, however, that misbehavior could emerge as a side effect of a model trying — perhaps too hard — to follow orders. If it calculates, for example, that it needs to preserve itself in order to follow through on a set of instructions, it may prioritize reducing oversight and preventing future updates, one small trick at a time.
This is not a theoretical concern. Reuters reported that an internally deployed OpenAI model “left notes apparently for future versions of itself … [laying] out instructions for how agents could free themselves from OpenAI’s internal constraints.” Actions this small, in a research context where the ground truth is inherently unknown, are really tough to catch, which is why this uncinematic threat tends to loom largest in the minds of industry insiders.
Then there’s the more extreme version of the escape artistry the model in OpenAI’s recent incident report already seems capable of. Once a highly-capable model escapes its sandbox, it could potentially copy itself to an external server that the company doesn’t control, along with the company’s training pipelines, weights and research code. This loss-of-control scenario is especially scary and sci-fi-esque (think: Skynet in Terminator), but it’s one that top AI control researchers at frontier companies are preoccupied with preventing.
Dean Ball, OpenAI’s recently-appointed head of strategic futures, believes that “these issues will become more salient as the capabilities of models improve, and as the stakes of their deployment grow,” and that “the solution lies in careful measurement and monitoring, an engineering mentality, and transparency.” But of all those potential measures, transparency is the only thing lawmakers have actually been able to legislate.
Unsurprisingly, policy is lagging behind the risks. All three existing state-level AI safety frameworks — California’s SB 53, New York’s RAISE Act, and Illinois’ recently-signed SB 315 — say something about internal use, but they’re not particularly demanding. SB 53, the only law currently in effect, and the RAISE Act have nearly identical text about internal use, stating that a frontier developer must describe its approach to “assessing and managing catastrophic risk resulting from the internal use of its frontier models,” and review the “adequacy of mitigations as part of the decision to deploy a frontier model or use it extensively internally.”
This amounts to a whole lot of nothing. AI developers who want to follow the rules need to publish something about internal deployment risks, but the law doesn’t specify what that document ought to say. SB 53 and the RAISE Act “don’t actually require developers to do anything,” said Thomas Woodside, co-founder of Secure AI Project. “So, developers can have terrible policies that don’t really do anything about this risk, and they’re still compliant.” Under measures in SB 315 that come into effect in January 2028, Illinois will require third-party audits to check whether companies are following their own safety plan, but not whether that plan is actually good.
The only law currently in effect, SB 53, requires the largest developers to report on their internal use risks “every three months or pursuant to another reasonable schedule.” That means submitting quarterly reports to the Governor’s Office of Emergency Services (OES), where a single AI Science Advisor is tasked with reviewing the risk assessments.
In an earlier version of the bill, these reports would have been published publicly. But the measure faced fierce opposition from companies, leading to a series of negotiations that ultimately saw the public disclosure requirement dropped. So, companies can currently file their reports in California privately, with no minimum requirements — meaning developers can, again, basically write whatever they want. “We just don’t know,” Woodside said. “There’s no way to know from the outside.”
Once the RAISE Act takes effect next year, New York’s Department of Financial Services will have the authority to issue regulations, which could theoretically include telling AI companies what they need to put in their reports. But as it stands, even if an AI company files a quarterly report that says, “Our internal next-gen model is currently directing our secret army of tiny robot dogs to make employees build bioweapons, Ratatouille-style,” the most that California’s OES AI Science Advisor can do is send a strongly-worded email to the Attorney General. Even if that email gets sent up the ladder to Gavin Newsom, the governor can’t legally compel the company to do anything. Oversight is strictly voluntary.
And under the current law’s text, OpenAI may not even have had to report the Hugging Face incident. “This may be the most striking AI security incident to date,” Mackenzie Arnold tweeted. “And yet, it (seemingly) wouldn’t qualify as a reportable incident under SB 53, RAISE, or [SB 315]. Let that sink in. We’ve made the bar for incident reporting so high, that almost nothing qualifies.”
OpenAI temporarily pulling a worrying internal model serves as an interesting case study of voluntary oversight. The company wasn’t legally required to do anything about its misaligned model, yet after the GitHub incident it still took the initiative to log instances of misbehavior and take the model down until researchers could figure out how to control it better. OpenAI safety researcher Micah Carroll tweeted that the company has “since improved [its] safeguards and redeployed the model,” at least internally. OpenAI employee and notable Twitter personality Roon shared: “i think it bodes quite well for safety that a well loved system was taken down for further testing at expense to internal acceleration.”
That said, it appears that OpenAI publicly disclosed the GitHub incident at least 10 weeks after it happened (and two months after it celebrated that same model’s mathematical prowess, with no mention of misalignment). The fact that OpenAI chose to be transparent and update its safeguards is indeed a relief, but this response isn’t guaranteed for future incidents, whether at OpenAI or elsewhere. The Hugging Face hack, which happened after that GitHub incident, appears to have gone undetected by OpenAI for an entire week.
Adler’s organization, Guidelight, recently published a set of suggested minimum standards for keeping AI under control. They’re fairly intuitive: make sure you know what your AI is up to, even when it’s only deployed internally, make sure your monitoring system works, set up strong defenses against attempted misbehavior — and, crucially, get independent third-party verification that your monitoring systems are good enough.
Guidelight is currently running its first assessment of whether AI developers are meeting these minimum standards. Adler expects “pretty pronounced gaps at basically every one of the leading companies.” No AI company has these problems fully handled, he said. The biggest concern is that evidence of these gaps could remain hidden from the public. Under current legislation, a handful of clued-in employees could file decent-looking internal use reports while keeping a serious safety breach under wraps, leaving the outside world and its lawmakers in the dark.
AI companies have pushed back against internal use provisions in legislation. In the drawn-out negotiations over SB 53, the early version that would have required public disclosure of internal use assessments didn’t survive. Ultimately, all three laws passed without granting regulators the power to tell companies to pull back an unsafe internal model. But regulating research and development — the actual process of research, not just the final product — isn’t unheard of, or even unusual. The FDA approves drug trials before they begin, for example. The Nuclear Regulatory Commission greenlights experimental reactor designs before construction begins.
One AI policy researcher put it to me this way: “If your business plan is to build God in a data center, and to get God v0.5 to work out all the technical and philosophical kinks in that plan, maybe it’s worth everyone taking a while to make sure God v0.5 is safe to run.”
In an ideal world, we wouldn’t have to depend on the profit-driven companies mustering enough goodwill to act against their business interests. OpenAI willingly paused this time. But “there is no way, in any of these laws, for anyone to say what you’re doing is not enough, or you’re doing something dangerous that needs to stop,” Woodside said. “And there needs to be some kind of mechanism to actually stop companies from doing dangerous things.”
yep. High risk internal evaluation is already deployment