We’re sleepwalking into an AI surveillance dystopia

The technology needed for industrial-scale observation exists — and is already in widespread use

Illustration by Chuan Ming Ong for Transformer

There are almost too many ways it could start.

Emboldened ICE agents are sent en masse into another blue city. The White House sends federal agents to “observe” polling stations. Federalized national guards are deployed. In the face of what they see as a life-or-death threat to their neighbors — or to American democracy itself — local residents start to mobilize, as they have in so many cities before.

As public social media accounts post details of protests, these are already being cross-referenced against bulk public datasets — bought, entirely legally, by federal agencies — to identify who is behind them. Connecting your TikTok account to a shop for a discount, or using X to log in to a delivery service just once, is enough: entirely legally, your anonymous social media account is tied to your real identity, and perhaps even your home address. From there, contact networks can be unmasked. This time, though, the federal government has made preparations.

Under the guise of monitoring for foreign interference in US politics, the FBI has been operating hundreds of AI-powered personas, or “bot” accounts, based in different US cities. Having built up a posting history over the course of months, these AI agents easily earn the trust required to get inside the various Signal and Telegram groups used to mobilize resistance, gathering information about meet places, timings and unguarded statements. Usernames and other details are easily matched to real-world identities, including their full online histories, their Amazon purchases, their…everything.

By the time anyone even takes to the streets, the government has the contents of their messages and coordination, and has preloaded facial recognition systems and geo-tagging tools to ping when certain individuals cross a police line, or enter somewhere off-limits. It is ready for mass arrests on all manner of charges: conspiracy, impeding the operations of law enforcement, and whatever it goes on to find. As networks of license-plate-tracking cameras coupled with advanced AI systems, such as those operated by Flock, proliferate, such pursuits only get easier. The kind of peaceful protest against ICE seen in Minneapolis just became impossible.

A Flock safety camera in Burbank, California. Credit: Justin Sullivan/Getty Images

For now, this is a near-future hypothetical, another warning of surveillance dystopia, a refreshed version of 1984 or Minority Report. But this time, there’s a crucial difference: all of the technology to deliver this already exists, and is already in widespread use. Existing large language models would be more than capable of delivering such a system, not just pulling together disparate sources of intelligence, but analyzing them in real-time, “advising” police on who to grab, and where to find them. Those systems will only improve in time. An administration could at least argue — and try its luck through the courts —that existing laws would allow all of this to be done.

Whether we have all-pervasive surveillance or not is now a matter of policy and budget priorities alone. It no longer relies upon development timelines. When a little over a decade ago Edward Snowden revealed the scale of the NSA’s mass collection of data — the reporting of which I worked on for the Guardian — the agency struggled to come up with success stories to justify its program to Congress. All it could show for spending billions of dollars was that the system had helped it catch a single $8,500 donation to the Somali terror group al-Shabab. The aim of Keith Alexander, the NSA director at the time, to “collect it all” ran ahead of the government’s capabilities to analyze that much data in real time.

Over the last few months, I have had conversations with current and former law enforcement staff, former intelligence agents from multiple countries, figures inside the large AI companies and broader technology sector, alongside academics and figures in civil society. Out of that, one clear message has emerged. Thanks to AI, the surveillance dystopia science fiction has spent decades warning us about is already here. We just haven’t noticed yet.

“We’ll enjoy this brief period of endurance of individual liberal rights solely on the basis of the good fortune of incompetence,” says Seth Lazar, a philosopher of the School of Government and Policy at Johns Hopkins University. Lazar, not a natural optimist, suggests we may look back on this as the good times. “The language models are, like, total fucking narcs. They could just as easily be snitches as well.”

The truth is that the specter of mass surveillance as a threat to our civil liberties is the dog that never barked. Snowden upended his life — to this day, he lives in exile in Russia — to reveal the extent of US mass surveillance capabilities. After some initial outrage, however, the world largely shrugged. Cameras have become ubiquitous in London, and life continues largely unchanged. We are mostly desensitized to these capabilities.

With the Snowden revelations, it seemed as if, yes, the NSA was collecting everything they could — but they were just drowning themselves in data. In London, the reason is fairly simple: yes, there are cameras everywhere, but no one is watching what they record, even if we’d like them to (when our bike is stolen, for example).

Most criticisms of mass surveillance have been predicated on it not working as intended. The civil liberties version of the argument goes like this: imagine you’re living in a Western country and you text your friend, who happens to be of Pakistani heritage, that the game this weekend is going to be “THE BOMB.” One misfire of a crude surveillance algorithm later, and you could both be on the wrong side of some aggressive police attention.

The public safety argument is a mirror image of this, first set out to me years ago by the independent surveillance law expert Eric Kind. In most Western countries, the number of people who are seriously planning or considering carrying out a major terror attack in the immediate future is far more likely to be in the hundreds at the most, rather than the thousands. Generally, these people can be identified through old-fashioned methods — human and community intelligence. In the UK, the perpetrator of almost every major terror attack in the last decade was already known to authorities.

Using mass surveillance to tackle this problem is essentially making the haystack ever bigger as you look for a fixed number of needles. All of that changes, though, in a world where AI is smart enough to get it right much more often. How do you argue against an always-on surveillance system that actually works?

Take London: all of the technology for collection is already there. Cameras are everywhere, though mostly not centrally networked. Mobile phone tracking is not just possible, but actively used on the underground system. Technology linking people’s payment cards to their movement already exists. Police already have facial recognition technology, though only use it at present on particular deployments. Even the legal framework to use these new technologies is almost entirely in place, if often untested.

A Metropolitan Police facial recognition van deployed in central London. Credit: Leon Neal/Getty Images

Tying these together into an AI-powered system to — at first, at least — help with the detection of serious crime is a matter of joining up existing systems and getting access to either a data center with the processing power to handle that much information, or an API from one of the big AI providers. The tools are now powerful enough to ingest, sort and join-up this data in a way that only a decade ago was out of reach.

A certain type of AI skeptic — the sort of person who tried ChatGPT when it was powered by GPT-3 and was underwhelmed, perhaps — might argue that the technology to underpin such a broad surveillance apparatus still isn’t there. To an extent, they’d have a point: cutting-edge AI models hallucinate far less often than older ones, but it still happens. There is an intrinsic limit as to how accurate assessments made by this technology could be, and it is obviously less than perfect.

But actual deployments of modern data analysis show the obvious potential of such systems. The Metropolitan Police recently fed data from 35 sources into a system provided by Palantir to monitor its own officers and staff for disciplinary violations or criminal offenses — such as keeping nudes on police devices, or crossing ethical boundaries with victims of crime.

The trial “resulted in investigations into hundreds of officers for issues including work-from-home violations, as well as a smaller number flagged for misconduct and criminality including sex offenses,” according to The Times. Modern AI technology will make mistakes, but it is evidently good enough to flag incidents for human investigation — and it can handle huge volumes of data in a sophisticated way. The argument on surveillance needs to keep up with the actual capabilities.

This will require a bigger conceptual shift than is immediately obvious. The most apparent risks of a system like this are how they might be used by bad actors. There is a proverb popular in South America, often attributed to Peruvian president Óscar Benavides: “For my friends, everything; for my enemies, the law.”

A tool that can analyze virtually everything for infractions at low cost and without alerting the target could be extremely powerful. The ability to check, almost immediately, someone’s data to make sure they are entirely in compliance with every law, every business regulation, and with their taxes entirely in order has huge potential for misuse.

Simply looking at Donald Trump’s efforts to persecute his opponents shows this up clearly. Trump’s Department of Justice has tried to prosecute James Comey over a social media post, with limited success given obvious First Amendment protections.

Similarly, Trump has tried to unseat Lisa Cook from the Federal Reserve over paperwork irregularities in a mortgage application, again so far without luck. An effort to prosecute John Bolton has proven more fruitful, thanks to Bolton’s unbelievable carelessness with classified materials — a trait he seems to share with the president himself, though Trump has so far avoided any consequences.

---

The data center is a symbol

---

Trump’s efforts to find wrongdoing for which he could take down his enemies were hampered by the often inept efforts of his prosecutors to find clear infractions. It is obvious how an AI-powered mass surveillance system could supercharge these efforts. If it could look deeply and cleanly enough into someone, it would almost inevitably find something. Selective deployment of even an entirely accurate surveillance system could easily be used for malign ends.

The reality is that laws were never meant to be enforced all of the time. When we file our taxes, we are aware that they might be audited, but probably won’t be, and that keeps most of us mostly honest. When we cross the road in a place where jaywalking is illegal, we often take a chance on our common sense.

In an essay warning of AI’s potential — for good and for ill — Anthropic CEO Dario Amodei famously equated a powerful model to a “country of geniuses in a data center.” In his own warning, Harvard professor and member of OpenAI technical staff Boaz Barak modified that formulation to sound superficially a little more prosaic. What could AI do if, instead, we thought of it as a “country of IRS agents in a data center” — a limitless resource for a government to use to investigate and control its own citizens?

There is a huge difference between the possibility of observation and the certainty of it. Criminal law is generally drawn up in an expansive way, to avoid the possibility of wrongdoers escaping accountability through loopholes or overly-tight regulations.

Society operates on an understanding that the most stringent laws should, in practice, be moderated through common sense and prosecutorial discretion. No one could be watching all the time. Most of us have never lived like we are being watched all the time, unless we grew up in totalitarian regimes — notably not enjoyable places to live. Perhaps the nearest anyone in a Western democracy has to come is those in the US while on a visa or green card, who at the moment are all too aware of the precarity of their living situation.

This is set to fundamentally transform the nature of the relationship between citizens and the state, explains Lazar.

“All of our laws and norms are calibrated to a certain level of enforcement,” he says, “and if you just take precisely the same laws and norms and apply them to kind of an environment where you’re able to do perfect enforcement, then they’re going to have very different aggregate impact.”

At the moment, we are living in an interregnum in which these capabilities exist but are very unevenly deployed, delayed by differential interest in take up, itself shaped by regulatory environments, resourcing available to authorities, and similar factors — rather than anything intrinsic to tech.

“So much of what we have done and built up is a kind of infrastructure that’s been lying latent, waiting for this moment – and I think that its illiberal implications are really profound,” says Lazar. “You can do so much more governing when you can monitor everybody all the time.”

For most of its early rollout and popularization, the internet was a countercultural tool. Hacker culture encouraged creativity and challenge to authority. The internet provided ways to get information outside the control of centralized states. It still offers ways to communicate secretly, transfer money out of sight, and to find guides as to how to do all of those things — unless you ask AI.

A 2026 preprint study coauthored by Lazar examined the phenomenon of “blind refusal,” testing what AI models would do when asked to help a user bypass a manifestly ridiculous, illegitimate, or unfair rule. It found that most would not help in these situations. (GPT-5.4 refused most consistently, while Grok was the most permissive, though the researchers note this was also true in the “control” state, too.)

Omnipresent AI-monitored surveillance requires far less in the way of technological advancement than AGI. It likely requires no technological breakthrough at all, meaning it is also much closer to reality than AGI, with only implementation delays standing in the way.

---

Child safety vs privacy: AI’s age verification dilemma

---

Despite this, we haven’t really made any of the shifts in our thinking required to tackle what any of it might mean. To return to London one final time, British courts have allowed UK police forces to roll out facial recognition technology under their existing legal powers. Judges have explicitly refused to consider hypothetical consequences of this, saying that is the role of Parliament — even as Parliament shows no interest in considering such questions.

Americans, though, are even less protected than Europeans, who have a more extensive body of internet-era data protection and privacy laws. The Fourth Amendment does not prevent law enforcement or federal agencies from using any information a citizen voluntarily discloses to someone else, who then relays it to them — a loophole known as a “third-party doctrine.”

In the age of commercial data brokers, this loophole has become large enough to steer a supertanker through: almost all information shared online — search history, contact books, browsing histories — for commercial exploitation can be purchased by the government and then used with minimal restraints.

The main barrier against its wider exploitation was always processing capability — and so it is already crumbling. The Supreme Court has already been wrestling with some of the implications of this: it was central to the recent Chatrie v. United States case limiting the use of geo-fencing by law enforcement. But for now the so-called “data broker loophole” is almost wholly intact.

The technology is being rolled out and implemented almost by default. The thinking simply isn’t keeping pace. Large US AI companies no longer want to discuss the implications of their technology too loudly in public, especially after Anthropic lost its federal contracts and was designated a supply chain risk for insisting on minimal legal safeguards.

Talking about the transformational power of AGI is, oddly, safe enough to do as it doesn’t seem imminent to politicians or regulators, even if some within technology companies feel otherwise. For those within the industry, asking the questions to challenge the AI surveillance paradigm right on our doorstep feels much riskier, because it is more likely to provoke regulatory action in the short-term.

Taking even the short-term potential of AI-integrated surveillance seriously requires a major reckoning, and necessitates either a change in how we draft our laws or sweeping new safeguards in how they are enforced — with significant new power for citizens to resist selective use of such powers.

Even minor advances in the power of AI could take this still further. Straightforward AI tools constantly scanning tax returns, movement, and similar data for existing offenses would significantly strengthen the state versus the citizen.

A more advanced system giving citizens risk scores or similar would be a logical next step once that infrastructure was in place, and one that would inevitably occur to vendors and customers alike. The ratchet is inevitable, and as it grows in abstraction, the role of the AI’s reasoning becomes ever more central.

The shift from “no one is watching” to “someone might be watching” — thanks to surveillance cameras, phone tapping, and similar technologies — took the entirety of the 20th century. The final shift to “something is always watching” could be completed inside a decade, and still catch us entirely unaware. Silicon Valley, which still, at least on some level, thinks of itself as libertarian, freewheeling and innovative, might be about to facilitate the largest transfer of power from citizen to state in the history of humanity.

There is still, just about, time to prevent that. But we’d need to start now. Waiting even a year or two might mean we simply sleepwalk into an AI surveillance dystopia by default.