SmarterArticles

Somewhere inside Claude, Anthropic's large language model, there is a cluster of artificial neurons that lights up whenever the Golden Gate Bridge enters the conversation. Not just when someone mentions the bridge by name, but when an image of it appears, when the topic of San Francisco landmarks arises, or when someone references the colour of international orange in a context that evokes the famous suspension span. Nearby, in the model's vast internal geography, sit other clusters responding to Alcatraz Island, the Golden State Warriors, and California Governor Gavin Newsom. The organisation of these concepts mirrors something strikingly familiar: the way a human brain might organise related knowledge about the San Francisco Bay Area in neighbouring neural populations.

This discovery, published by Anthropic's interpretability team in May 2024, was not merely a curiosity. It represented what researchers described as “the first ever detailed look inside a modern, production-grade large language model.” And it arrived at a moment when the stakes of understanding these systems could hardly be higher. Large language models now draft legal briefs, assist medical diagnoses, generate code for critical infrastructure, and advise on policy decisions. Yet for all their capability, their internal reasoning remains largely opaque, even to the engineers who built them.

The quest to crack open this opacity has produced a new scientific discipline that sits at the intersection of neuroscience, computer science, and philosophy of mind. Mechanistic interpretability, as the field is known, borrows tools and conceptual frameworks from decades of brain research to reverse-engineer the computational mechanisms hidden inside artificial neural networks. The ambition is extraordinary: to build what amounts to a microscope for AI, capable of revealing not just what these systems say, but how and why they arrive at their outputs.

The question is whether this microscope can be made powerful enough, fast enough, to keep pace with AI systems that are growing more capable by the month. And whether what it reveals can ever translate into the kind of safety guarantees that high-stakes deployment demands.

The Neuroscience Parallel That Launched a Field

The intellectual lineage of mechanistic interpretability traces directly to neuroscience. Chris Olah, co-founder of Anthropic and one of the pioneers of the field, has spent over a decade working to identify internal structures within neural networks, first at Google Brain, then at OpenAI, and now at Anthropic. TIME named him to its TIME100 AI list in 2024, recognising his foundational contributions to the discipline. In an interview with the 80,000 Hours podcast, Olah described his work as fundamentally about understanding what is going on inside neural networks, treating them not as inscrutable black boxes but as systems with discoverable internal structure.

The parallel between studying brains and studying neural networks is more than a convenient metaphor. Both systems consist of vast numbers of interconnected units whose individual behaviour is relatively simple but whose collective activity produces remarkably complex outputs. In neuroscience, researchers have long used techniques like functional magnetic resonance imaging, single-neuron recording, and optogenetics to identify which brain regions and circuits correspond to specific cognitive functions. The interpretability community is attempting something analogous with artificial systems, and the methodological borrowing is increasingly explicit.

A 2024 paper by Adam Davies and Ashkan Khakzar, titled “The Cognitive Revolution in Interpretability,” formalised this connection. The authors argued that mechanistic interpretability methods enable a paradigm shift similar to psychology's historical “cognitive revolution,” which moved the discipline beyond pure behaviourism toward understanding internal mental processes. They proposed a taxonomy organising interpretability into two categories: semantic interpretation, which asks what latent representations a model has learned, and algorithmic interpretation, which examines what operations the system performs over those representations. Davies and Khakzar contended that these two modes of investigation have “divergent goals and objects of study” but suggested they might eventually unify under a common framework, much as cognitive science itself integrated insights from linguistics, psychology, neuroscience, and computer science.

This framework echoes the influential levels of analysis proposed by neuroscientist David Marr in the 1980s, which distinguished between the computational goals of a system, the algorithms it employs, and the physical implementation of those algorithms. The suggestion is not that artificial neural networks are brains, but that the intellectual toolkit developed to study brains offers a surprisingly productive way to study their silicon counterparts.

The analogy has practical teeth. Just as neuroscientists discovered that individual brain regions specialise in particular functions, interpretability researchers have found that language models develop internal specialisations that bear a surface resemblance to the modular organisation of biological cognition. The Golden Gate Bridge feature is one example among millions, but the principle it illustrates is broadly applicable: these models do not store information as undifferentiated numerical soup. They develop structured, organised representations that can be individually identified and experimentally manipulated, much as a neuroscientist might stimulate a specific brain region and observe the resulting behavioural change.

A paper published in Nature Machine Intelligence by researchers Kohitij Kar, Martin Schrimpf, and Evelina Fedorenko at MIT made an important distinction, however. They noted that interpretability means different things to neuroscientists and AI researchers. In AI, interpretability typically focuses on understanding how model components contribute to outputs. In neuroscience, interpretability requires explicit alignment between model components and neuroscientific constructs such as brain areas, recurrence, or top-down feedback. Bridging these two conceptions remains an active challenge, and conflating them risks generating false confidence about how well we truly understand what these systems are doing.

Sparse Autoencoders and the Problem of Polysemanticity

The central technical obstacle in reading the minds of language models is a phenomenon called polysemanticity. Individual neurons in these networks typically respond to many unrelated concepts simultaneously. A single neuron might activate for references to legal contracts, the colour blue, and mentions of 1990s pop music. This makes individual neurons nearly useless as units of analysis, much as recording from a single neuron in the human brain rarely tells you what someone is thinking.

The problem has a name in the interpretability literature: superposition. Chris Olah wrote in a July 2024 update on Transformer Circuits that if you had asked him a year earlier what the key open problems for mechanistic interpretability were, “I would have told you the most important problem was superposition.” The term refers to the way neural networks pack more concepts into fewer neurons than ought to be possible, representing information in overlapping patterns that defy straightforward analysis.

Anthropic's breakthrough came from applying a technique called sparse dictionary learning, borrowed from classical machine learning, to decompose the tangled activity of polysemantic neurons into cleaner units called features. The tool for accomplishing this is the sparse autoencoder, a type of neural network trained to compress and reconstruct the internal activations of a language model while enforcing a sparsity constraint. The sparsity penalty ensures that for any given input, only a small fraction of features have nonzero activations. The result is an approximate decomposition of the model's internal states into a linear combination of feature directions, each ideally corresponding to a single interpretable concept.

In their May 2024 paper, “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet,” Anthropic's team demonstrated that this approach could work on a production-scale model. Eight months earlier, they had shown the technique could recover monosemantic features from a small one-layer transformer in their earlier paper “Towards Monosemanticity,” but a major concern was whether the method would scale to state-of-the-art systems. It did. The team extracted tens of millions of features from Claude 3 Sonnet's middle layer, identifying responses to concrete entities like cities, people, chemical elements, and programming syntax, as well as abstract concepts like code bugs, gender bias in discussions, and conversations about secrecy.

The features proved to be highly abstract: multilingual, multimodal, and capable of generalising between concrete and abstract references. A feature for the Golden Gate Bridge activated on text about the bridge, images of the bridge, and descriptions in multiple languages. Features neighbouring it in the model's internal space corresponded to related concepts, suggesting that Claude's internal organisation reflects something resembling human notions of conceptual similarity. Anthropic's researchers proposed that this conceptual neighbourhood structure might help explain what they described as Claude's “excellent ability to make analogies and metaphors.”

Perhaps most significant for safety, the researchers identified features linked to harmful behaviours, including scam emails, bias, code backdoors, and sycophancy. When they artificially amplified these features, the model's behaviour changed accordingly, demonstrating a causal relationship between internal representations and outputs. When they boosted the Golden Gate Bridge feature to extreme levels, Claude began dropping references to the bridge into nearly every response and even claimed to be the bridge itself. The team also explored various sparse autoencoder architectures, including TopK, Gated SAEs, and JumpReLU variants, developing quantified autointerpretability methods that measure the extent to which Claude can make accurate predictions about its own feature activations.

Yet the researchers were candid about the limitations. The discovered features represent only a small subset of the concepts Claude has learned. Finding a complete set would require computational resources exceeding the cost of training the original model.

Tracing Thoughts Through Attribution Graphs

If sparse autoencoders provided the first lens for viewing individual features, Anthropic's 2025 work on circuit tracing provided the first tool for watching those features interact during reasoning. In two companion papers, “Circuit Tracing: Revealing Computational Graphs in Language Models” and “On the Biology of a Large Language Model,” the team introduced attribution graphs, a technique for tracing the internal flow of information between features during a single forward pass through the model.

The method works by constructing a “replacement model” that substitutes more interpretable components, called cross-layer transcoders, for the original multi-layer perceptrons. This allows researchers to produce graph descriptions of the model's computation on specific prompts, revealing intermediate concepts and reasoning steps that are invisible from outputs alone. Anthropic's CEO Dario Amodei noted that the company's understanding of the inner workings of AI lags far behind the progress being made in AI capabilities, framing interpretability research as a race to close that gap before the consequences of ignorance become catastrophic.

One demonstration involved asking Claude 3.5 Haiku, “What is the capital of the state where Dallas is located?” Intuitively, answering this question requires two steps: inferring that Dallas is in Texas, then recalling that the capital of Texas is Austin. The researchers found evidence that the model genuinely performs this two-step reasoning internally, with identifiable intermediate features representing the concept of Texas before the final answer of Austin emerges. Critically, they also found that this genuine multi-step reasoning coexists alongside “shortcut” reasoning pathways, suggesting that the model maintains multiple computational strategies for arriving at the same answer.

The research yielded several other striking findings. When tasked with composing rhyming poetry, the model was found to plan multiple words ahead to meet rhyme and meaning constraints, effectively reverse-engineering entire lines before writing the first word. When researchers examined cases of hallucination, they discovered the counter-intuitive result that Claude's default behaviour is to decline to speculate, and it only produces fabricated information when something actively inhibits this default reluctance. In examining jailbreak attempts, they found that the model recognised it had been asked for dangerous information well before it managed to redirect the conversation to safety.

The attribution graph approach also revealed a subtlety about faithful versus unfaithful reasoning. When asked to compute the square root of 0.64, Claude produced faithful chain-of-thought reasoning with features representing intermediate mathematical steps. But when asked to compute the cosine of a very large number, the model sometimes simply fabricated an answer, and the attribution graph made this difference in computational strategy visible.

Anthropic open-sourced the circuit-tracing tools in May 2025, and a collaborative effort involving researchers from Anthropic, Decode, EleutherAI, Goodfire AI, and Google DeepMind has since applied them to open-weight models including Gemma-2-2B, Llama-3.1-1B, and Qwen3-4B through the Neuronpedia platform.

OpenAI's Automated Neuron Explanations and Their Limits

While Anthropic pursued feature-level analysis through sparse autoencoders, OpenAI took a different but complementary approach. In May 2023, a team including Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders published research demonstrating that GPT-4 could be used to automatically write explanations for the behaviour of individual neurons in GPT-2 and to score those explanations for accuracy.

Their methodology consisted of three steps. First, text sequences were run through the model being evaluated to identify cases where a particular neuron activated frequently. Next, GPT-4 was shown these high-activation patterns and asked to generate a natural language explanation of what the neuron responds to. Finally, GPT-4 was asked to predict how the neuron would behave on new text sequences, and these predictions were compared against actual neuron behaviour to produce an accuracy score. The approach was notable for its ambition: rather than relying on human researchers to manually inspect neurons one at a time, it attempted to automate the entire interpretability pipeline.

The team found over 1,000 neurons with explanations scoring at least 0.8, meaning GPT-4's descriptions accounted for most of the neuron's top-activating behaviour. They identified neurons responding to phrases related to certainty and confidence, neurons for things done correctly, and many others. They released their datasets and visualisation tools for all 307,200 neurons in GPT-2, inviting the research community to develop better techniques. The researchers noted that the average explanation score improved as the explainer model's capabilities increased, suggesting that more powerful future models might produce substantially better explanations.

But the limitations were substantial. As researcher Jeff Wu acknowledged, “Most of the explanations score quite poorly or don't explain that much of the behaviour of the actual neuron.” Many neurons activated on multiple different things with no discernible pattern, and sometimes GPT-4 was unable to find patterns that did exist. The approach focused on short natural language explanations, but neurons may exhibit behaviour too complex to describe succinctly, particularly when they are highly polysemantic or represent concepts that humans lack words for.

The approach also carries a deeper conceptual challenge. Using one language model to explain another creates a circularity: the explanations are only as good as the explainer model's own understanding, which is itself opaque. If GPT-4 cannot correctly interpret certain patterns, those patterns remain hidden regardless of how sophisticated the automated pipeline becomes. The researchers acknowledged this limitation, noting that they would ultimately like to use models to “form, test, and iterate on fully general hypotheses just as an interpretability researcher would.”

OpenAI's broader alignment agenda initially positioned interpretability as central to its work on superalignment, the challenge of ensuring that AI systems much smarter than humans remain aligned with human values. However, in May 2024, the Superalignment team was effectively dissolved following the departures of co-lead Ilya Sutskever and head of alignment Jan Leike. OpenAI has continued interpretability-adjacent research under other organisational structures, publishing work on sparse-autoencoder latent attribution for debugging misalignment in late 2025.

The Scalability Gap Between Understanding and Assurance

The practical limitations of current interpretability methods become starkly apparent when measured against the demands of high-stakes deployment. Understanding that a particular feature in Claude responds to the Golden Gate Bridge is fascinating. Understanding the full computational graph that leads Claude to recommend a specific medical treatment, draft a particular legal argument, or generate code for a safety-critical system is an entirely different proposition.

Leonard Bereska and Max Gavves, in their comprehensive 2024 review “Mechanistic Interpretability for AI Safety,” surveyed the field's methods for causally dissecting model behaviours and assessed their relevance to safety. They emphasised that “understanding and interpreting these complex systems is not merely an academic endeavour; it's a societal imperative to ensure AI remains trustworthy and beneficial.” Yet they also catalogued formidable challenges in scalability, automation, and comprehensive interpretation. Their review further examined the dual-use risks of interpretability research itself, noting that the same tools that help safety researchers detect deceptive behaviours could potentially help malicious actors understand how to circumvent safety measures.

The scalability problem is twofold. First, modern language models contain billions or trillions of parameters, and the number of potential features and circuits grows combinatorially. Anthropic's work on Claude 3 Sonnet extracted tens of millions of features from a single layer, and a complete analysis would require resources exceeding the original training cost. Second, even when individual features or circuits are identified, composing them into a full account of the model's behaviour on any given input remains beyond current capabilities. The field can offer snapshots of computational processes, not comprehensive maps.

Anthropic has publicly stated its goal to “reliably detect most AI model problems by 2027” using interpretability tools. The company took a concrete step toward integrating interpretability into deployment decisions when it used mechanistic interpretability in the pre-deployment safety assessment of Claude Sonnet 4.5. Before releasing the model, researchers examined internal features for dangerous capabilities, deceptive tendencies, or undesired goals. This represented the first known integration of interpretability research into deployment decisions for a production system.

Yet the gap between detecting specific known problems and providing comprehensive safety assurances remains vast. Finding a feature associated with deception does not guarantee that all deceptive pathways have been identified. The absence of evidence for dangerous capabilities is not evidence of absence. And the speed at which new models are trained and deployed vastly outpaces the speed at which they can be thoroughly interpreted.

MIT Technology Review named mechanistic interpretability one of its 10 Breakthrough Technologies for 2026, recognising that “research techniques now provide the best glimpse yet of what happens inside the black box.” The phrasing is telling: a glimpse, not a complete picture.

NeuroAI and the Convergence of Biological and Artificial Understanding

The parallels between neuroscience and AI interpretability are not merely inspirational. A growing body of research suggests that genuine scientific convergence between the two fields could benefit both, and that the emerging discipline of NeuroAI represents a return to the cross-pollination that produced many of AI's foundational breakthroughs.

A 2024 editorial in Nature Machine Intelligence noted that while AI has shifted toward transformers and other complex architectures that seem to have moved away from neural-inspired roots, the field “may still look towards neuroscience for help in understanding complex information processing systems.” The editorial pointed to a coalition of initiatives around “NeuroAI,” a push to identify fresh ideas at the intersection of the two disciplines, including the annual COSYNE conference which has become a focal point for researchers working across both fields.

A paper in Nature Communications argued that the emerging field of NeuroAI “is based on the premise that a better understanding of neural computation will reveal fundamental ingredients of intelligence and catalyse the next revolution in AI.” The authors noted that historically, many key AI advances, including convolutional neural networks and reinforcement learning, were inspired by neuroscience, but that this cross-pollination had become far less common than in the past, representing what they called a missed opportunity.

A 2024 paper in Nature Reviews Neuroscience discussed how NeuroAI has the potential to transform large-scale neural modelling and data-driven neuroscience discovery, though the field must balance exploiting AI's power while maintaining interpretability and biological insight. The paper highlighted that unlike the human brain, which features a variety of morphologically and functionally distinct neurons, artificial neural networks typically rely on a homogeneous neuron model. Incorporating greater diversity of neuron models could address key challenges in AI, including efficiency, interpretability, and memory capacity.

The convergence runs in both directions. Sparse autoencoders, developed for AI interpretability, have found applications in protein language model research, where they uncover biologically interpretable features in protein representations. Representation engineering approaches that track latent neural trajectories when processing different input types draw directly on methods developed for studying neural population dynamics in biological brains.

The Whole Brain Architecture Initiative in Japan has proposed what it calls “brain-based interpretability,” arguing that if an advanced AI system's computational processes can be understood at a cognitive level in terms of corresponding human neural activity, unfavourable intentions or deceptions would be more readily detectable. The premise is that biological neural circuits, refined by millions of years of evolution, provide a reference architecture against which artificial computation can be measured and understood.

Yet researchers at MIT have cautioned that interpretability requires different things in the two domains. Understanding what a particular feature in an AI model represents is not the same as understanding why a biological neuron fires in a particular pattern. The former asks about function within an engineered system; the latter asks about mechanism within an evolved one. Collapsing this distinction risks importing assumptions from one domain that may not hold in the other.

Governance Frameworks and the Trust Translation Problem

The interpretability research emerging from Anthropic, OpenAI, Google DeepMind, and academic institutions arrives against a backdrop of rapidly evolving governance frameworks that increasingly demand transparency from AI systems. The question is whether the scientific progress being made in mechanistic interpretability can translate into the kind of transparency that regulators, deployers, and the public actually need.

The European Union's AI Act, which entered into force on 1 August 2024, provides the most comprehensive regulatory framework. Article 13 requires that high-risk AI systems “shall be designed and developed in such a way as to ensure that their operation is sufficiently transparent to enable deployers to interpret a system's output and use it appropriately.” Non-compliance carries penalties reaching 35 million euros or 7 per cent of global annual turnover. The Act's provisions on prohibited AI practices and AI literacy obligations became applicable from 2 February 2025, with general-purpose AI rules taking effect in August 2025 and the full framework becoming applicable by August 2026.

Yet scholars have identified what they call the “compliance gap” between the Act's transparency requirements and implementation reality. The regulation does not specify what level of interpretability is technically required, creating ambiguity about whether current mechanistic interpretability tools satisfy the legal standard. A feature-level understanding of a model's internal representations is not the same as a human-readable explanation of why the model made a specific decision in a specific case. The former is a scientific achievement; the latter is what a doctor, a judge, or a loan officer needs to justify relying on the system's output.

Proposals to bridge this gap take several forms. A framework from UC Berkeley for “Guaranteed Safe AI” suggests extracting interpretable policies from black-box algorithms via automated mechanistic interpretability and then directly proving safety guarantees about these policies. The approach would offload most of the verification work to AI systems themselves, potentially making the process scalable.

An ICLR 2026 workshop on “Principled Design for Trustworthy AI” has foregrounded topics including mechanistic interpretability and concept-based reasoning, inference-time safety and monitoring, reasoning trace auditing in large language models, and formal verification methods and safety guarantees. The workshop's framing reflects a growing consensus that interpretability must be integrated across the full AI lifecycle, from training and evaluation to inference-time behaviour and deployment.

Some researchers envision a future in which a simpler oversight model reads the internal state of a more complex model to ensure it is safe, a form of scalable oversight that depends on mechanistic interpretability being reliable enough to trust. Bowen Baker at OpenAI has described work on building what the company terms an “AI lie detector” that examines internal representations to determine whether a model's internal state corresponds to truth or contradicts it. “We got it for free,” Baker told reporters, explaining that the interpretability feature emerged unexpectedly from training a reasoning model.

Google DeepMind has contributed its own tools to the ecosystem, releasing Gemma Scope 2 in 2025 as the largest open-source interpretability toolkit, covering all Gemma 3 model sizes from 270 million to 27 billion parameters. The open-source release signals a recognition across the industry that interpretability research cannot remain proprietary if it is to serve as a foundation for trust.

The MATS programme (ML Alignment Theory Scholars) and SPAR (Systematic Problem-solving for Alignment Research) have become training grounds for the next generation of interpretability researchers, with projects spanning AI control, scalable oversight, evaluations, red-teaming, and robustness. Their existence reflects a field that is rapidly professionalising, building institutional infrastructure to match the scale of the challenge.

When the Microscope Meets the Real World

The ultimate test of mechanistic interpretability is not whether it can produce elegant scientific insights about how language models work. It is whether it can tell a hospital administrator that an AI diagnostic tool is safe to deploy, tell a financial regulator that an algorithmic trading system will not precipitate a market crash, or tell a defence ministry that an autonomous weapons targeting system will reliably distinguish combatants from civilians.

By that standard, the field remains in its early stages. Current methods can identify individual features, trace specific circuits, and reveal particular reasoning patterns. They cannot yet provide comprehensive accounts of model behaviour across all possible inputs, guarantee the absence of dangerous capabilities, or produce the kind of formal safety proofs that high-stakes applications demand.

Yet the trajectory is unmistakable. In the space of two years, the field has moved from demonstrating that sparse autoencoders work on toy models to extracting millions of features from production systems, from static feature analysis to dynamic circuit tracing, and from purely academic research to integration into pre-deployment safety assessments. Anthropic's stated goal of reliable problem detection by 2027 may be ambitious, but the pace of progress makes it less implausible than it would have seemed even twelve months ago.

The neuroscience parallel offers both encouragement and caution. Neuroscientists have been studying the brain for over a century and still cannot fully explain how it produces consciousness, language, or complex decision-making. If artificial neural networks prove even a fraction as complex as biological ones, full interpretability may remain a receding horizon. But neuroscience has nonetheless produced enormously useful partial understanding: enough to develop treatments for neurological disorders, design brain-computer interfaces, and guide educational practices. Partial understanding of AI systems, even without complete transparency, may prove similarly valuable.

The governance implications of this partial understanding are profound. If mechanistic interpretability can reliably detect certain categories of problems, such as deceptive reasoning, specific biases, or known dangerous capabilities, then regulatory frameworks can be built around those detectable risks. The EU AI Act's transparency requirements need not demand complete interpretability to be meaningful; they need only demand interpretability sufficient to catch the problems that matter most.

What is needed, and what the field is only beginning to develop, is a rigorous framework for characterising exactly what current interpretability methods can and cannot detect, with quantified confidence levels and explicit acknowledgement of blind spots. Without such a framework, the risk is that interpretability becomes what security researchers call “security theatre”: a reassuring performance of understanding that obscures ongoing ignorance.

The convergence of neuroscience and AI interpretability research offers a path toward that framework. By grounding artificial system analysis in the conceptual vocabulary and methodological rigour of a mature scientific discipline, researchers can avoid the trap of mistaking pattern recognition for genuine understanding. The brain, after all, has taught us that the gap between observing neural activity and comprehending cognition is vast. The same humility should attend our attempts to read the minds of machines.

For now, the microscope is improving. The question that will define the next decade of AI governance is whether it can improve fast enough.

References and Sources

1. Anthropic. “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.” Transformer Circuits, May 2024. https://transformer-circuits.pub/2024/scaling-monosemanticity/

2. Anthropic. “Mapping the Mind of a Large Language Model.” Anthropic Research, 2024. https://anthropic.com/research/mapping-mind-language-model

3. Anthropic. “Circuit Tracing: Revealing Computational Graphs in Language Models.” Transformer Circuits, 2025. https://transformer-circuits.pub/2025/attribution-graphs/methods.html

4. Anthropic. “On the Biology of a Large Language Model.” Transformer Circuits, 2025. https://transformer-circuits.pub/2025/attribution-graphs/biology.html

5. Anthropic. “Tracing the Thoughts of a Language Model.” Anthropic Research, 2025. https://www.anthropic.com/research/tracing-thoughts-language-model

6. Anthropic. “Open-Sourcing Circuit-Tracing Tools.” Anthropic Research, May 2025. https://www.anthropic.com/research/open-source-circuit-tracing

7. Bills, Steven, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. “Language Models Can Explain Neurons in Language Models.” OpenAI, May 2023. https://openai.com/index/language-models-can-explain-neurons-in-language-models/

8. Davies, Adam, and Ashkan Khakzar. “The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms.” arXiv:2408.05859, August 2024. https://arxiv.org/abs/2408.05859

9. Kar, Kohitij, Martin Schrimpf, and Evelina Fedorenko. “Interpretability of Artificial Neural Network Models in Artificial Intelligence versus Neuroscience.” Nature Machine Intelligence, 2022. https://www.nature.com/articles/s42256-022-00592-3

10. Bereska, Leonard, and Max Gavves. “Mechanistic Interpretability for AI Safety: A Review.” arXiv:2404.14082, April 2024. https://arxiv.org/abs/2404.14082

11. European Union. “Regulation (EU) 2024/1689: The Artificial Intelligence Act.” Official Journal of the European Union, 2024. https://artificialintelligenceact.eu/

12. Vox. “AI Interpretability: OpenAI, Claude, Gemini, and Neuroscience.” Vox Future Perfect, 2024. https://www.vox.com/future-perfect/362759/ai-interpretability-openai-claude-gemini-neuroscience

13. Nature. “AI Needs to Be Understood to Be Safe.” Nature News Feature, 2024. https://www.nature.com/articles/d41586-024-01314-y

14. Engineering.fyi. “Language Models Can Explain Neurons in Language Models.” 2023. https://www.engineering.fyi/article/language-models-can-explain-neurons-in-language-models

15. Nature Communications. “Catalyzing Next-Generation Artificial Intelligence Through NeuroAI.” Nature Communications, 2023. https://www.nature.com/articles/s41467-023-37180-x

16. Nature Reviews Neuroscience. “The Emergence of NeuroAI: Bridging Neuroscience and Artificial Intelligence.” 2025. https://www.nature.com/articles/s41583-025-00954-x

17. Nature Machine Intelligence. “The New NeuroAI.” Editorial, 2024. https://www.nature.com/articles/s42256-024-00826-6

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

The price you saw was not the price everyone saw. You just did not know it yet.

In February 2024, Wendy's CEO Kirk Tanner told investors that the fast-food chain would invest $20 million in digital menu boards to support “dynamic pricing and day-part offerings.” The reaction was immediate, visceral, and devastating. Consumers heard “surge pricing” and revolted. Social media erupted. Burger King capitalised on the moment by offering free Whoppers, its email subject line reading: “Surge Pricing? Not at Burger King!” Within days, Wendy's Vice President Heidi Schauer was forced to clarify to NPR that the company would not raise prices during peak hours, insisting the plan was merely about discounts during slower periods. The damage, however, was already done. Wendy's had accidentally revealed something the technology industry had been quietly building for years: an infrastructure designed to charge different people different prices for the same thing, calibrated by algorithms that know more about you than you might suspect.

That infrastructure is no longer theoretical. It is operational, expanding, and largely invisible to the consumers it targets. Across e-commerce, travel, entertainment, housing, and soon your local supermarket, artificial intelligence systems are ingesting vast quantities of personal data to estimate individual willingness to pay and adjust prices accordingly. The question confronting regulators, consumers, and the technology companies themselves is whether this represents a natural evolution of market efficiency or a fundamental breakdown in the social contract that underpins fair commerce.

How the Pricing Machine Learns What You Will Pay

To understand why AI-driven pricing has become such a flashpoint, you need to understand what these systems actually do. Traditional dynamic pricing is nothing new. Airlines have adjusted fares based on demand since the 1980s. Hotels shift rates around holidays and conferences. Uber's surge pricing algorithm, which multiplies fares during periods of high demand, has been the subject of academic study for over a decade. A 2016 National Bureau of Economic Research paper estimated that UberX generated approximately $6.8 billion in consumer surplus across the United States in 2015, suggesting that for every dollar spent by consumers, roughly $1.60 in surplus was generated.

A natural experiment on New Year's Eve illustrated the point. When Uber's surge pricing algorithm across all of New York City broke down for 26 minutes due to a technical glitch, the platform's average wait time spiked from 2.6 minutes to 8 minutes, and unfulfilled trip requests rose significantly. The algorithm, whatever consumers thought of it, was performing a genuine market function. But even Uber's model, which adjusts prices based on aggregate supply and demand rather than individual consumer profiles, has drawn regulatory backlash. Cities including Honolulu, Manila, New Delhi, and Singapore have banned or capped surge pricing. Research by Juan Camilo Castillo at the University of Pennsylvania, using Uber data from Houston in 2017, found that while surge pricing generally improved market outcomes, its effects were unevenly distributed, with price-sensitive riders bearing a disproportionate burden during peak periods.

What is happening now goes far beyond adjusting prices to reflect real-time supply and demand. The new generation of AI pricing tools analyses individual consumer behaviour, browsing history, purchase patterns, location data, device type, credit history, and demographic information to estimate what each specific person is willing to pay. Amazon reportedly adjusts product prices around 2.5 million times every day, updating 50 times more frequently on average than Walmart. The company considers both “global values” such as demand volume and stock levels, and “user values” including product visit frequency and time of purchase. Research indicates that loyal, returning customers may face higher prices than newcomers, as the dynamic pricing engine calculates each customer's loyalty level and sets prices accordingly.

The algorithmic approaches powering these systems are sophisticated and continually evolving. Reinforcement learning models analyse customer demand while accounting for seasonality, competitor pricing, and market uncertainty to arrive at revenue-optimal prices. Bayesian models incorporate historical pricing data and shift their estimates with every new data point. Behavioural pricing systems analyse individual customer actions in real time to offer personalised discounts or price adjustments based on predicted likelihood of purchase. A Valcon study found that while 61 per cent of European retailers have embraced some form of dynamic pricing, fewer than 15 per cent currently use algorithmic or AI-based strategies. That number is set to change rapidly: 55 per cent of European retailers are actively planning to pilot dynamic pricing with generative AI in 2026.

The business case is compelling. Reports indicate that AI-driven dynamic pricing can increase average order value by up to 13 per cent during peak sales periods, cut overstock by 6 per cent in a single quarter, and boost profit margins by as much as 25 per cent. For companies operating on thin margins in competitive markets, these are not marginal improvements. They are transformative. And the practice is spreading beyond the expected players. Researchers at the University of New South Wales have warned that personalised pricing could soon reach supermarkets, noting that consumers have no way of knowing whether the price they see for bread or bananas on a retailer's website is the same price that another consumer sees.

When Landlords Let the Algorithm Decide

The most striking demonstration of what happens when algorithmic pricing goes wrong did not occur in an online shop or a ride-hailing app. It happened in the American rental housing market, where millions of tenants discovered that their rent increases were being orchestrated by a single piece of software.

In August 2024, the United States Department of Justice, alongside the Attorneys General of eight states including California, North Carolina, and Colorado, filed a civil antitrust lawsuit against RealPage Inc. The complaint alleged that RealPage contracted with competing landlords who agreed to share nonpublic, competitively sensitive information about their apartment rental rates to train and run RealPage's algorithmic pricing software. The software then generated pricing recommendations for participating landlords based on their competitors' data. Prosecutors stated that one landlord reported starting to increase rents within a week of adopting the software and, within eleven months, had raised them by more than 25 per cent.

In January 2025, the DOJ expanded the case, adding six major multifamily property owners as co-defendants, including Greystar. Nine states subsequently reached a $7 million settlement with Greystar in November 2025. By that same month, the DOJ had reached a proposed settlement with RealPage itself. The company did not admit liability but agreed to stop using competitors' nonpublic data in its revenue management product, to restrict model training to historic data at least twelve months old, to redesign its software to remove mechanisms that prop up prices or encourage competitors toward common pricing ranges, and to accept a court-appointed monitor with broad access to review its code and model training documentation. The settlement terms are operative for seven years.

The RealPage case matters far beyond the housing sector because it established a legal framework for how algorithmic pricing tools can cross the line from legitimate optimisation into anticompetitive behaviour. When an algorithm aggregates private data from competitors and uses it to coordinate pricing upward, it functions as a mechanism for tacit collusion, regardless of whether any human explicitly agreed to fix prices. The DOJ's Antitrust Division head has promised an increase in probes of algorithmic pricing, and in March 2025, the agency filed a statement of interest regarding “the application of the antitrust laws to claims alleging algorithmic collusion and information exchange.”

Surveillance Pricing and the FTC's Unfinished Investigation

In July 2024, the Federal Trade Commission under Chair Lina Khan launched what it called a surveillance pricing inquiry, using its 6(b) authority to issue orders to eight companies: Mastercard, Revionics, Bloomreach, JPMorgan Chase, Task Software, PROS, Accenture, and McKinsey. The Commission voted 5-0 to issue the orders. Khan stated that “firms that harvest Americans' personal data can put people's privacy at risk. Now firms could be exploiting this vast trove of personal information to charge people higher prices.”

Speaking at the Fast Company Innovation Festival in September 2024, Khan elaborated: “Given just how much intimate and personal information that digital companies are collecting on us, there's increasingly the possibility of each of us being charged a different price based on what firms know about us.” She noted that while economists had long studied price personalisation, it was previously more of a “thought experiment,” but advances in data extraction and targeting had made it “much more possible to be serving every individual person an individual price based on everything they know about you.”

The preliminary findings, published in January 2025, revealed that instead of a price or promotion being a static feature of a product, the same product could have a different price or promotion based on consumer-related data, behaviours, preferences, location, time, and purchase channel. Some companies could determine individualised pricing based on granular consumer data, with the study citing examples such as a cosmetics company targeting promotions based on specific skin types and tones. The FTC found that at least 250 businesses, including grocery stores, apparel retailers, health and beauty retailers, and hardware stores, had adopted surveillance pricing strategies.

Then the investigation stalled. FTC Chair Andrew Ferguson, who replaced Khan, cancelled the public comment period, effectively ending the study. With new federal leadership signalling that continuing the investigation was not a priority, the unfinished inquiry left a regulatory vacuum.

That vacuum did not last long. In December 2025, Senator Mark R. Warner led Senators Gallego, Blumenthal, and Hawley in a bipartisan push urging the Trump administration to crack down on surveillance pricing, which the senators described as a practice that “eliminates a fixed or static price in favour of prices specially tailored to an individual consumer's willingness to pay.” State lawmakers across the country began introducing legislation to regulate practices that use personal data, AI, and frequent price changes, particularly in sectors like food and housing. The regulatory baton, at least in the United States, has been passed from the federal level to the states, creating a patchwork of approaches that may prove difficult for businesses to navigate and consumers to understand.

The Oasis Fiasco and the British Regulatory Response

If the American regulatory landscape is fragmented, the United Kingdom's has been galvanised by a single, furiously debated event: the Oasis reunion ticket sale.

On 31 August 2024, tickets for 17 shows across the UK and Ireland went on sale exclusively through Ticketmaster. Millions of fans endured long virtual queues and multiple site crashes. Many discovered that standing tickets, initially advertised at approximately £135, had risen to as much as £355 by the time they reached checkout. The backlash was enormous. UK culture minister Lisa Nandy pledged to look into Ticketmaster's use of dynamic pricing. The band itself issued a statement claiming that “Oasis leave decisions on ticketing and pricing entirely to their promoters and management” and that lead members Liam and Noel Gallagher had not known dynamic pricing would be used.

On 5 September 2024, the Competition and Markets Authority launched an investigation into Ticketmaster's conduct. The CMA's findings, published in March 2025, were revealing. The regulator found no evidence that Ticketmaster had used algorithmic real-time pricing in the traditional sense. Instead, the company had released a batch of standing tickets at a lower price, and once those sold out, released the remaining tickets at a much higher price. The CMA was concerned that consumers had not been given clear and timely information about how the pricing would work, particularly given that many customers had endured lengthy queues with no warning that prices would change.

The Oasis controversy accelerated regulatory action. In late 2024, the Sale of Tickets (Sporting and Cultural Events) Bill was introduced in Parliament, seeking to require ticket-selling platforms to display the full range of available tickets, their quantities, and prices to consumers before they joined online queues. More broadly, the CMA has positioned itself as a proactive regulator of online pricing practices. The Digital Markets, Competition and Consumers Act received Royal Assent in May 2024 and its new digital markets competition regime came into force on 1 January 2025. Under this framework, the CMA can decide whether consumer laws have been broken without having to go through the courts, and can fine companies up to 10 per cent of global turnover. The CMA has also launched enforcement actions covering online pricing practices, including drip pricing and pressure selling, using its new powers to order businesses to pay compensation to affected customers.

The CMA has acknowledged that pricing algorithms can benefit consumers by reducing transaction costs and market frictions, but it has also flagged the risk that algorithms could “facilitate collusive outcomes” and increase prices. In a notable observation, the CMA suggested that the risk of businesses colluding with one another over prices would actually diminish if there were extensive use of personalised pricing algorithms in digital markets, because each firm would be setting individual prices rather than converging on common ones. It is a counterintuitive argument that illustrates just how complex the regulatory challenge has become.

Europe Drafts Its Digital Fairness Rulebook

The European Union, rarely content to let a regulatory opportunity pass, is constructing what could become the most comprehensive framework for governing personalised pricing anywhere in the world.

The Digital Fairness Act, overseen by EU Commissioner Michael McGrath, is designed to address manipulative interface design, misleading influencer marketing, addictive design features, subscription traps, and, critically, unfair personalisation and pricing practices. The European Commission launched a public consultation on the DFA on 17 July 2025, which closed on 24 October 2025 and received 3,341 responses, the vast majority from consumers.

The results were striking. At least 77 per cent of respondents supported measures including greater consumer control over personalised advertising, restrictions on advertising that exploits vulnerabilities, a prohibition on personalised advertising targeting minors, and restrictions on personalised pricing based on personal data and profiling. The existing Consumer Rights Directive already requires traders to inform consumers if a price has been personalised based on automated decision-making, but businesses are not required to disclose the specific parameters or criteria used. The DFA is expected to go considerably further. The consultation also examined “drip pricing,” where a low price is initially presented but incrementally increased, and noted that rapid pricing changes putting consumers under psychological pressure to act quickly may be considered misleading or aggressive practices.

The formal draft is expected in Q3 2026, with final adoption expected in late 2027. The DFA is expected to apply broadly across the business-to-consumer digital economy, affecting e-commerce platforms, streaming services, telecoms, airlines, travel platforms, ride-hailing and delivery apps, and any business that uses personalised offers, automated subscriptions, or dynamic pricing.

For companies operating globally, the DFA represents a potentially seismic shift. The EU's track record with the General Data Protection Regulation demonstrated that European rules can set de facto global standards, as companies find it more efficient to comply everywhere than to maintain different systems for different jurisdictions. If the DFA mandates meaningful transparency about how personalised prices are calculated, businesses worldwide may have to disclose information they currently treat as proprietary.

Meanwhile, Australia's competition regulator, the ACCC, released the final report of its five-year Digital Platform Services Inquiry in June 2025. Across 14 reports, the ACCC broadly flagged risks emerging from generative AI integration into commercial operations, including algorithmic coordination and transparency in automated decision-making. The ACCC concluded that Australia's current laws cannot adequately deal with the harms arising from such a fast-evolving industry and recommended an economy-wide prohibition on unfair trading practices, along with mechanisms to force algorithmic disclosure.

What the Researchers Found About Who Actually Benefits

The most uncomfortable finding for advocates of AI-driven personalised pricing comes from Carnegie Mellon University's Tepper School of Business. A study published in Marketing Science by Yan Huang, Associate Professor of Business Technologies, Kannan Srinivasan, Professor of Management, Marketing, and Business Technology, and Param Vir Singh, Carnegie Bosch Professor of Business Technologies and Marketing, examined the interaction between personalised ranking systems and pricing algorithms on e-commerce platforms.

Their findings challenge the conventional wisdom that personalised pricing benefits consumers by showing them more relevant products at competitive prices. The researchers found that personalised ranking systems, which present products in order of estimated consumer preference, may actually encourage higher prices from pricing algorithms, particularly when consumers search for products sequentially on third-party platforms. This occurs because personalised ranking significantly reduces the ranking-mediated price elasticity of demand, diminishing the algorithmic incentive to lower prices. Conversely, unpersonalised ranking systems led to significantly lower prices and greater consumer welfare.

The implications are profound. As doctoral student Liying Qiu, who collaborated on the research, has noted, increased consumer data sharing may not always result in improved outcomes, even in the absence of explicit price discrimination. Personalised ranking, empowered by access to more detailed consumer data, can facilitate algorithms charging higher prices. Certain pricing algorithms may even learn to engage in tacit collusion in competitive scenarios, resulting in consequences harmful to consumer welfare.

This research suggests that the very infrastructure of modern e-commerce, the personalised interfaces that platforms use to show you products they think you want, can function as a mechanism for extracting higher prices. The consumer experience of being “understood” by a platform may simultaneously be the mechanism through which that consumer pays more.

The Information Asymmetry Problem, Supercharged

In 1970, the economist George Akerlof published “The Market for Lemons,” a paper that would eventually win him a share of the 2001 Nobel Prize in Economics alongside Michael Spence and Joseph Stiglitz. Akerlof demonstrated how information asymmetry between buyers and sellers could cause markets to break down entirely. When sellers know more about the quality of a product than buyers do, prices fall to reflect the buyer's uncertainty, which drives away sellers of genuinely good products, which further depresses buyer confidence, until the market collapses or only the worst products remain.

Governments responded to this problem with consumer protection legislation: lemon laws, mandatory disclosures, vehicle inspection requirements, and financial product transparency rules. These interventions worked precisely because they reduced the information gap between buyer and seller.

AI-driven personalised pricing creates a new form of information asymmetry that is qualitatively different from anything Akerlof described. In this case, the seller does not merely know more about the product than the buyer. The seller knows more about the buyer than the buyer knows about themselves, at least in economic terms. The algorithm has processed the buyer's browsing history, purchase frequency, price sensitivity, location, time of day, device, and potentially hundreds of other signals to arrive at a price that is optimised not for fairness, not for competition, but for the maximum amount the algorithm calculates this specific individual will accept.

This is not the invisible hand of the market at work. It is a one-way mirror. The consumer sees a price and assumes it is the price. The algorithm sees a consumer and calculates what it can get. The traditional economic assumptions that underpin competitive markets, informed buyers comparing transparent prices from competing sellers, simply do not hold when every buyer sees a different price and has no way of knowing it.

The economist's argument that price discrimination can theoretically improve welfare by allowing markets to serve price-sensitive consumers who would otherwise be priced out is valid in its own theoretical framework. But it assumes that sellers will actually lower prices for those consumers rather than simply charge everyone the maximum. Without transparency, there is no mechanism to verify that the welfare-improving version of personalised pricing is what consumers actually receive. And without transparency mandates, consumers have no tools to distinguish between a system that genuinely serves their interests and one that extracts every penny of surplus.

What Transparency Would Actually Require

If regulators mandate price transparency for AI-driven pricing, what would that look like in practice? The proposals currently circulating across multiple jurisdictions suggest several overlapping approaches.

The simplest is disclosure: requiring businesses to tell consumers when a price has been personalised. The EU's existing Consumer Rights Directive already mandates this, though without requiring businesses to explain how the personalisation works. The Digital Fairness Act may extend this to require disclosure of the parameters used, the data inputs, and the algorithmic logic.

A second approach is price comparison: requiring that consumers be shown the base or median price alongside their personalised price, so they can see whether they are paying more or less than average. This would create competitive pressure, as consumers who discovered they were consistently paying above the median might switch to competitors.

A third approach, favoured by some competition regulators, is algorithmic auditing: requiring companies to submit their pricing algorithms to independent review, much as the RealPage settlement requires a court-appointed monitor to review the company's code and model training documentation. This would allow regulators to detect collusive behaviour, discriminatory pricing patterns, or systematic exploitation of vulnerable consumers without requiring consumers to understand the algorithms themselves.

A fourth, more radical approach is prohibition: banning personalised pricing entirely in certain sectors, much as some jurisdictions have capped or banned surge pricing for ride-hailing services. The Oasis ticket controversy has prompted legislative proposals in the UK to regulate dynamic pricing in entertainment. The question is whether prohibition in essential sectors like food, housing, and healthcare would be proportionate, or whether it would simply drive the practice underground.

Each approach involves trade-offs. Full algorithmic disclosure could reveal proprietary business methods. Price comparison mandates could be gamed by setting artificial baselines. Auditing regimes are only as good as the auditors' technical capabilities and independence. Outright bans may prevent genuinely beneficial price adjustments that serve consumers well.

Navigating the Invisible Marketplace

The stakes of this debate extend well beyond whether your next pair of trainers costs 5 per cent more because the algorithm noticed you browsed them three times. They go to the heart of what kind of marketplace a digitally connected society wants to inhabit.

If personalised pricing becomes the universal default, the concept of a “price” in the way most consumers understand it ceases to exist. There is no longer a number attached to a product. There is a number attached to a relationship between a product and a buyer, mediated by an algorithm that neither party fully controls or understands. Every transaction becomes a negotiation in which only one side knows it is negotiating.

The Wendy's backlash, the Oasis ticket fury, the RealPage lawsuit, and the FTC's aborted surveillance pricing inquiry all point in the same direction: consumers find personalised pricing fundamentally unfair when they discover it, and they are deeply uncomfortable with the idea that algorithmic systems know enough about them to exploit that knowledge. The 77 per cent of EU consultation respondents who supported restrictions on personalised pricing are not outliers. They are the mainstream.

The counterargument from industry is not without merit. Dynamic pricing does allocate scarce resources more efficiently. It does enable businesses to serve price-sensitive consumers with lower prices. It does reduce waste by aligning prices with actual demand. But these benefits depend on transparency and genuine competition, neither of which is guaranteed in an opaque algorithmic marketplace. Research from the University of New South Wales has found that 70 per cent of consumers are comfortable with dynamic pricing when they perceive it as fair and transparent, suggesting that the issue is not the concept itself but the secrecy surrounding its implementation.

What is clear is that the regulatory frameworks governing these practices are being written right now, in Brussels, in London, in Canberra, in state legislatures across the United States. The EU's Digital Fairness Act, the UK's Digital Markets, Competition and Consumers Act, the ACCC's reform recommendations, and the patchwork of American state legislation are all attempting to answer the same fundamental question: in a world where algorithms can determine exactly how much you are willing to pay, does the consumer have a right to know?

The answer, increasingly and across jurisdictions, appears to be yes. The debate is no longer about whether transparency is necessary, but about how much transparency is enough, who enforces it, and how quickly the rules can keep pace with the algorithms they are meant to govern. For consumers who have spent years handing over their data in exchange for convenience, the price of that bargain is about to become visible, whether the algorithms like it or not.

---

References and Sources

1. NPR, “No, Wendy's says it isn't planning to introduce surge pricing,” 28 February 2024. https://www.npr.org/2024/02/28/1234412431/wendys-dynamic-surge-pricing

2. Axios, “Why fast-food fans flipped out over Wendy's pricing,” 29 February 2024. https://www.axios.com/2024/02/29/wendys-surge-pricing-ai-backlash-internet

3. Cohen, Hahn, Hall, Levitt, and Metcalfe, “Using Big Data to Estimate Consumer Surplus: The Case of Uber,” NBER Working Paper No. 22627, 2016. https://www.nber.org/papers/w22627

4. Hall, Kendrick, and Nosko, “The Effects of Uber's Surge Pricing: A Case Study.” https://www.uber.com/blog/research/the-effects-of-ubers-surge-pricing-a-case-study/

5. Castillo, J.C., “Who Benefits from Surge Pricing?”, University of Pennsylvania, 2019. https://economics.sas.upenn.edu/system/files/2020-01/JMP_Castillo.pdf

6. Pricefy, “How Amazon Uses Real-Time Data and Dynamic Pricing to Maximize Profits.” https://www.pricefy.io/articles/amazon-real-time-data-dynamic-pricing

7. AIMultiple, “Dynamic Pricing Algorithms in 2026: Top 3 Models.” https://research.aimultiple.com/dynamic-pricing-algorithm/

8. Master of Code, “AI Dynamic Pricing: Boost Profits by 10%, Sales by 13%.” https://masterofcode.com/blog/ai-dynamic-pricing

9. UNSW Newsroom, “AI is using your data to set personalised prices online,” October 2025. https://www.unsw.edu.au/newsroom/news/2025/10/AI-using-data-personalised-data-prices-online

10. UNSW Newsroom, “The rise of dynamic pricing: should AI decide what you pay?“, September 2025. https://www.unsw.edu.au/newsroom/news/2025/09/dynamic-pricing-AI-decide-what-you-pay

11. US Department of Justice, “Justice Department Sues RealPage for Algorithmic Pricing Scheme,” August 2024. https://www.justice.gov/archives/opa/pr/justice-department-sues-realpage-algorithmic-pricing-scheme-harms-millions-american-renters

12. US Department of Justice, “Justice Department Requires RealPage to End Sharing of Competitively Sensitive Information,” November 2025. https://www.justice.gov/opa/pr/justice-department-requires-realpage-end-sharing-competitively-sensitive-information-and

13. ProPublica, “DOJ and RealPage Agree to Settle Rental Price-Fixing Case.” https://www.propublica.org/article/doj-realpage-settlement-rental-price-fixing-case

14. Mintz, “Last Year's Rent: RealPage Reaches Settlement Agreement with the DOJ,” December 2025. https://www.mintz.com/insights-center/viewpoints/2191/2025-12-01-last-years-rent-realpage-reaches-settlement-agreement

15. Federal Trade Commission, “FTC Issues Orders to Eight Companies Seeking Information on Surveillance Pricing,” July 2024. https://www.ftc.gov/news-events/news/press-releases/2024/07/ftc-issues-orders-eight-companies-seeking-information-surveillance-pricing

16. FTC, “Behind the FTC's Inquiry into Surveillance Pricing Practices,” July 2024. https://www.ftc.gov/policy/advocacy-research/tech-at-ftc/2024/07/behind-ftcs-inquiry-surveillance-pricing-practices

17. Fast Company, “Lina Khan says the FTC is investigating surveillance pricing,” September 2024. https://www.fastcompany.com/91195551/lina-khan-ftc-federal-trade-commission-chair-surveillance-pricing-explained-what-is-it

18. FTC, “Surveillance Pricing Update & The Work Ahead,” January 2025. https://www.ftc.gov/policy/advocacy-research/tech-at-ftc/2025/01/surveillance-pricing-update-work-ahead

19. FTC, “Surveillance Pricing Study Indicates Wide Range of Personal Data Used,” January 2025. https://www.ftc.gov/news-events/news/press-releases/2025/01/ftc-surveillance-pricing-study-indicates-wide-range-personal-data-used-set-individualized-consumer

20. Future of Privacy Forum, “A Price to Pay: U.S. Lawmaker Efforts to Regulate Algorithmic and Data-Driven Pricing.” https://fpf.org/blog/a-price-to-pay-u-s-lawmaker-efforts-to-regulate-algorithmic-and-data-driven-pricing/

21. Senator Mark R. Warner, press release on surveillance pricing, December 2025. https://www.warner.senate.gov/public/index.cfm/2025/12/warner-leads-bipartisan-effort-to-push-ftc-to-crack-down-on-surveillance-pricing-with-holiday-shopping-season-underway

22. NPR, “Ticketmaster 'dynamic pricing' subject to U.K. investigation into Oasis ticket sales,” September 2024. https://www.npr.org/2024/09/06/g-s1-21316/oasis-reunion-ticketmaster-dynamic-pricing

23. Variety, “Oasis Tickets: U.K. Opens Probe Into Ticketmaster's 'Dynamic Pricing',” September 2024. https://variety.com/2024/global/global/ticketmaster-dynamic-pricing-oasis-uk-government-investigation-1236127481/

24. Arts Professional, “Oasis concerts: Watchdog says 'no evidence' Ticketmaster used dynamic pricing,” March 2025. https://www.artsprofessional.co.uk/news/oasis-concerts-watchdog-says-no-evidence-ticketmaster-used-dynamic-pricing

25. Womble Bond Dickinson, “DMCC Act 2024 explained.” https://www.womblebonddickinson.com/uk/insights/articles-and-briefings/digital-markets-competition-and-consumers-act-2024-explained-cmas

26. CMA, “CMA launches major consumer protection drive focused on online pricing practices.” https://www.gov.uk/government/news/cma-launches-major-consumer-protection-drive-focused-on-online-pricing-practices

27. Pinsent Masons, “CMA: collusion could be addressed with personalised pricing.” https://www.pinsentmasons.com/out-law/news/cma-addressing-collusion-with-personalised-pricing

28. European Parliament, Digital Fairness Act Legislative Train Schedule. https://www.europarl.europa.eu/legislative-train/theme-protecting-our-democracy-upholding-our-values/file-digital-fairness-act

29. Slaughter and May, “Digital Fairness Act: European Commission publishes responses to consultation,” December 2025. https://thelens.slaughterandmay.com/post/102m222/digital-fairness-act-european-commission-publishes-responses-to-consultation

30. Osborne Clarke, “Digital Fairness Act Unpacked: Unfair Pricing Practices.” https://www.osborneclarke.com/insights/digital-fairness-act-unpacked-unfair-pricing-practices

31. ACCC, “Digital Platform Services Inquiry final report,” June 2025. https://www.accc.gov.au/about-us/publications/serial-publications/digital-platform-services-inquiry-2020-25-reports/digital-platform-services-inquiry-final-report-march-2025

32. Huang, Srinivasan, and Singh, “Personalization, Consumer Search, and Algorithmic Pricing,” Marketing Science, Vol. 44, No. 6, 2025. https://www.cmu.edu/tepper/news/stories/2025/0602-ai-driven-personalized-pricing-may-not-help-consumers

33. CMU Tepper School, Liying Qiu doctoral research profile. https://www.cmu.edu/tepper/news/stories/2025/0519-doctoral-student-liying-qiu-studies-ai-consumer-behavior-and-market-dynamics

34. Akerlof, G., “The Market for Lemons: Quality Uncertainty and the Market Mechanism,” Quarterly Journal of Economics, Vol. 84, No. 3, 1970, pp. 488-500.

35. Nobel Prize in Economics 2001, Akerlof, Spence, and Stiglitz. Econlib. https://www.econlib.org/library/Enc/bios/Akerlof.html

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

She is 25 years old, lives in Barcelona, has pink hair, and earns up to ten thousand euros a month from brand partnerships with companies including Amazon and Razer. She has never eaten a meal, never taken a breath, and never existed outside the rendering pipelines of a creative agency called The Clueless. Her name is Aitana Lopez, and she represents something that should unsettle anyone who makes a living by being themselves on the internet.

Aitana is not an anomaly. She is a data point on an exponential curve. The global virtual influencer market, valued at approximately 6.06 billion US dollars in 2024 according to Grand View Research, is projected to reach 45.88 billion dollars by 2030, growing at a compound annual growth rate of 40.8 per cent. Chief marketing officers are expected to allocate 30 per cent of their influencer marketing budgets to virtual influencers by 2026. The synthetic faces are arriving, and they are arriving fast.

But here is the question that the marketing projections do not answer, the one that sits at the uncomfortable intersection of technology, psychology, and economics: if AI can create perfect, tireless digital influencers that never make mistakes or demand higher pay, what happens to human creators and the authentic human connection that audiences seek from content creators?

The answer, it turns out, is considerably more complicated than the headlines suggest.

Synthetic Stars and the Agencies Behind Them

The modern virtual influencer industry traces its most visible origins to 2016, when a Los Angeles startup called Brud introduced Lil Miquela to Instagram. Presented as a 19-year-old Brazilian-American model and musician, Miquela accumulated millions of followers and secured brand partnerships with Prada, Calvin Klein, Samsung, and dozens of other companies. TIME magazine named her one of the 25 most influential people on the internet in 2018, alongside BTS and Rihanna. Her creators at Brud attracted approximately 30 million dollars in investment from firms including Sequoia Capital and Spark Capital, earning a valuation of approximately 125 million dollars before being acquired by Dapper Labs in 2021. By some estimates, Lil Miquela has generated over 10 million dollars in revenue, charging around 10,000 dollars per sponsored Instagram post.

Then came Noonoouri, the animated avatar created by Munich-based graphic designer Joerg Zuber in 2018. With her exaggerated doll-like features and impossibly large eyes, Noonoouri made no pretence of being human. Yet Dior trusted her enough to grant an Instagram takeover for their Cruise Makeup collection. She went on to partner with Versace, Marc Jacobs, Burberry, Balenciaga, and Kim Kardashian's beauty and fashion lines. The modelling agency IMG, which has represented Gigi Hadid and Bella Hadid, signed her to their roster. In 2023, Warner Music gave Noonoouri a record deal, making her the first strictly digital popstar signed to a major label.

The economics that drive this expansion are bluntly logical. Ruben Cruz, founder of The Clueless and creator of Aitana Lopez, explained his rationale to Euronews in 2024. “We started analysing how we were working and realised that many projects were being put on hold or cancelled due to problems beyond our control. Often it was the fault of the influencer or model and not due to design issues.” The solution, in his view, was to build an influencer who would never be late, never cause a scandal, and never renegotiate her fee.

This calculus appeals to brands for reasons that extend beyond mere cost savings. Virtual influencers offer what marketing professionals call “brand safety” in its purest form. They cannot be photographed at a competitor's event. They will not post an ill-considered political opinion at three in the morning. They will not age out of their target demographic, gain weight, or develop substance abuse problems. They are, in the language of corporate risk management, perfectly controllable assets.

And the data suggests that audiences are, at least superficially, responding. According to HypeAuditor, virtual influencers generate engagement rates approximately three times higher than their human counterparts, with an average of 2.84 per cent compared to 1.72 per cent for human influencers. Virtual influencer campaigns in 2023 achieved an average engagement rate of 5.9 per cent, triple the 1.9 per cent recorded for campaigns featuring real people.

Yet this headline figure conceals a crucial nuance. When it comes to sponsored content specifically, human influencers achieve 2.7 times more engagement than their AI counterparts. Lil Miquela's BMW campaign, for instance, generated an average engagement rate of just 0.6 per cent, compared to the 3.6 per cent delivered by human creators working with the same brand. The implication is that whilst virtual influencers may attract curiosity, human creators still hold a measurable advantage when the goal is to convert attention into commercial action.

The Economics of Being Replaced

For the estimated 50 million people worldwide who consider themselves professional content creators, these numbers are not merely interesting. They are existential.

The creator economy in 2025, valued at approximately 191 billion dollars and projected to grow to 528 billion dollars by 2030, is simultaneously booming and fracturing. The total market is expanding, but the share available to individual human creators is under unprecedented pressure. A 93 per cent year-over-year increase in the number of people creating user-generated content has intensified competition at every level. Among full-time creators, 52 per cent report a noticeable decline in consumer spending on affiliate-linked products. Among part-time creators, 40 per cent cite falling brand commissions and fewer sponsorship opportunities.

The threat is not limited to influencers in the traditional sense. A landmark study published in December 2024 by CISAC, the International Confederation of Societies of Authors and Composers representing over five million creators globally, provided the first comprehensive economic modelling of generative AI's impact on creative professions. Conducted by PMP Strategy, the study projected that music creators will see 24 per cent of their revenues at risk of loss by 2028, whilst audiovisual creators face a 21 per cent revenue risk over the same period. The cumulative financial impact amounts to an estimated 22 billion euros over five years: 10 billion euros in music and 12 billion euros in audiovisual production.

The study found that the market for AI-generated music and audiovisual content is expected to grow from approximately 3 billion euros to 64 billion euros by 2028. Generative AI music alone is projected to account for roughly 20 per cent of traditional music streaming platforms' revenues and around 60 per cent of music libraries' revenues by that date. Translators and adaptors for dubbing and subtitling face the most severe displacement, with 56 per cent of their revenue at risk, whilst screenwriters and directors could see their income cannibalised by 15 to 20 per cent.

Perhaps most strikingly, the CISAC study noted that not a single AI developer had signed a licensing agreement with any of the 225 collective management organisations that represent creators worldwide. The value transfer, in other words, is flowing in one direction: from human creators to the technology companies building the systems that will compete with them.

For content creators operating on platforms like Instagram, TikTok, and YouTube, the competitive dynamics are subtler but no less consequential. AI-generated content can be produced at a fraction of the cost and at a pace that no human can match. A virtual influencer does not need sleep, does not require a production crew, and can generate dozens of posts per day across multiple platforms simultaneously. The marginal cost of producing one more piece of content approaches zero. For a human creator who must plan, shoot, edit, and publish content whilst also managing brand relationships, responding to comments, and maintaining some semblance of a personal life, the asymmetry is stark.

What Audiences Actually Want (and What They Say They Want)

The conventional wisdom holds that audiences will always prefer “authentic” human connection over synthetic perfection. The research suggests this is true, but with caveats that should concern anyone who relies on conventional wisdom for comfort.

A 2025 survey from the Influencer Marketing Factory found that only 15 per cent of consumers express high trust in AI influencers, whilst nearly half say they are less likely to trust content from a virtual influencer compared to a human one. In a study conducted by Baringa, 77 per cent of respondents said they would want to know if content had been created by AI, either wholly or partially. Only 12 per cent said they would not care.

The trust penalty for AI-generated content is measurable and consistent across studies. Research from the Nuremberg Institute for Market Decisions published in 2025 found that simply labelling an advertisement as AI-generated made consumers perceive it as less natural and less useful, which lowered both their attitudes towards the advertisement and their willingness to research or purchase the product. Approximately 62 per cent of consumers reported being less likely to engage with or trust social media content they knew was generated by AI.

And yet, the picture is not quite so simple. The same research ecosystem reveals contradictions that complicate the “authenticity always wins” narrative. A study of TikTok users in the Middle East published in Discover Sustainability found that AI influencers can establish meaningful emotional bonds and credibility, sometimes outperforming human influencers in generating community cohesion and network expansion. Research published in Psychology and Marketing found that followers respond to virtual influencers in ways that mirror their responses to human creators, with engagement rates and measures of trust and source credibility that rival those of their flesh-and-blood counterparts.

The generational divide is particularly telling. Virtual influencers appeal more to Gen Z consumers, who have grown up immersed in AI-enabled technologies and may not share older generations' preoccupation with the notion of authenticity as it has traditionally been understood. When the distinction between “real” and “constructed” has been blurred since childhood (by filters, by avatars, by curated social media personas that bear only passing resemblance to the people behind them), the arrival of an explicitly artificial influencer may feel less like a violation and more like an honest acknowledgement of what social media has always been.

There is a deeper irony here. The “authenticity” that human influencers claim as their competitive advantage has always been, to a significant degree, performative. The casual photograph that required forty takes. The “unfiltered” video that was carefully scripted. The “honest review” that was contractually obligated to include three positive talking points about the sponsoring brand. If authenticity is already a construction, does it matter whether the constructor is carbon-based or silicon-based?

The answer, according to the psychology of parasocial relationships, is more nuanced than a simple yes or no.

The Strange Intimacy of Parasocial Bonds

Parasocial relationships, the one-sided emotional connections that audiences form with media figures, have been studied by psychologists since Donald Horton and Richard Wohl first described the phenomenon in 1956. Originally applied to television presenters and film stars, the concept has found renewed relevance in the age of social media, where the perceived intimacy between creator and audience is amplified by direct messaging, live streaming, and the illusion of personal access.

The question of whether parasocial relationships can form with virtual influencers has been the subject of intense academic investigation. A preregistered experiment published in New Media and Society by Stein, Breves, and Anders in 2024 found that viewers' parasocial interactions did not differ significantly between a human influencer and a virtual one. However, the researchers identified what they called “opposing effects”: whilst a direct effect suggested stronger parasocial interactions with the virtual influencer, participants simultaneously attributed this persona with less mental human-likeness and less perceived similarity to themselves. These two forces partially cancelled each other out.

Research published in the Journal of Business Research by Liu and Wang in 2025 added another layer of complexity through the lens of the uncanny valley. Studying 826 Instagram users, they found that as virtual influencers become more humanlike, they often trigger psychological unease and eeriness. This discomfort intensifies when consumers deeply engage with virtual influencer content whilst remaining aware of its artificial nature, potentially diminishing the strength of parasocial relationships at precisely the moment when the technology becomes most convincing.

The implications are paradoxical. Virtual influencers that look obviously artificial (like Noonoouri, with her cartoonish proportions) may actually generate stronger parasocial bonds than those designed for photorealism, because they do not trigger the uncanny valley response. The more a virtual influencer tries to pass as human, the more its artificiality may repel the audience it seeks to attract.

But there is a counter-trend that complicates this analysis. Newer AI systems are becoming sophisticated enough to generate personalised responses, to adapt their communication styles based on audience feedback, and to create the impression of genuine emotional reciprocity. Research on AI influencer marketing has explored the potential of what scholars call “Dynamic Emotional Resonance” and “AI-Driven Attachment Styles,” whereby artificial systems learn to mirror the emotional patterns that foster deep parasocial bonds. If an AI influencer can respond to your comment in a way that feels personally meaningful, if it can remember your previous interactions and reference them naturally, if it can adapt its tone and content to your individual preferences, the distinction between “real” and “artificial” connection becomes increasingly difficult to maintain.

This is where the question stops being about marketing and starts being about something more fundamental. If the feeling of connection is indistinguishable from genuine connection, does the distinction matter? The answer depends entirely on what you believe connection is for.

The Algorithm's Thumb on the Scale

The competitive landscape between human and AI creators is not shaped solely by audience preferences. It is shaped, to a significant degree, by the platforms themselves and the algorithms that determine what content gets seen.

Research from Cornell University by Brooke Erin Duffy and Colten Meisner, based on interviews with 30 creators across TikTok, Instagram, Twitch, YouTube, and Twitter, found that creators invest significant labour in understanding the algorithms that govern their visibility. Because many creators operate across multiple platforms, they must learn the hidden rules for each one and adapt their entire approach to content production accordingly. The algorithms are not neutral arbiters; they are designed to maximise engagement, and they reward content that keeps users on the platform regardless of whether that content was produced by a human being or a rendering engine.

TikTok's algorithm, in particular, is designed for what engineers call “cold start” optimisation: it tests new content with small groups of users and, if those users engage, pushes it to progressively larger audiences. This design theoretically levels the playing field between established creators and newcomers. But it also means that content which is optimised for algorithmic engagement (consistent posting frequency, precise timing, trending audio, specific visual patterns) has an inherent advantage over content that prioritises the messy, unpredictable qualities that make human creators distinctive.

AI-generated content, by its nature, can be optimised for algorithmic preference with a precision that human creators cannot match. It can be produced at the exact frequency, length, and format that the algorithm rewards. It can incorporate trending elements within minutes of their emergence. It can A/B test variations of the same content simultaneously, learning in real time which version generates the most engagement. The algorithm does not care whether the content was made by a person or a process. It cares about watch time, completion rates, shares, and comments.

This creates a structural disadvantage for human creators that exists independently of audience preferences. Even if audiences prefer human-created content when given a clear choice, they may never be given that choice if the algorithm surfaces AI-generated content more frequently because it performs better on the metrics that platforms optimise for.

The research on algorithmic bias adds another dimension. Marc Faddoul, an AI researcher at UC Berkeley's School of Information, demonstrated that TikTok's recommendation algorithm would suggest accounts with profile pictures matching the same race, age, and facial characteristics as accounts a user already followed. The algorithm creates feedback loops in which certain types of content (and certain types of faces) are amplified whilst others are suppressed. If AI-generated influencers are designed to embody the physical characteristics that algorithms have historically amplified (conventionally attractive, often white, always polished), they may receive a structural boost that compounds their other advantages.

Regulation in the Age of Synthetic Persuasion

Regulators are beginning to grapple with the implications, though the pace of regulatory development lags considerably behind the pace of technological deployment.

In the United States, the Federal Trade Commission updated its Endorsement Guides in June 2023 to explicitly cover virtual influencers and AI-generated content. The FTC's position is that AI-generated personas must follow the same disclosure rules as human endorsers. If a virtual influencer promotes a product, both the sponsorship and the involvement of AI must be disclosed. In June 2025, the FTC proposed rules recasting any marketing mention (including promotional codes, affiliate links, and brand tags) as paid endorsements requiring “clear and conspicuous” disclosure. The FTC put 670 companies on notice in 2023 alone, and enforcement actions in 2024 resulted in 337.3 million dollars being returned to consumers.

The European Union has gone further. Article 50 of the EU AI Act, now in phased enforcement, establishes what are arguably the world's most explicit transparency obligations for synthetic media. Providers and users of AI systems that generate or substantially manipulate images, audio, or video must ensure that such content is clearly identifiable as artificial. Violations can attract penalties of up to 15 million euros or 3 per cent of global annual turnover, whichever is higher. Full compliance is expected by August 2026.

In the United Kingdom, the Advertising Standards Authority issued guidance in May 2025 clarifying that AI use in advertising must be disclosed when it could mislead consumers about authenticity or performance. At the state level in the United States, California passed AB 2655 in 2024, requiring large online platforms to label or remove deceptive AI content, and AB 1836, which mandates disclosure and consent when AI recreates a person's image or voice for commercial use.

These regulatory frameworks address transparency but do not address the underlying competitive dynamics. Requiring disclosure that an influencer is AI-generated does not prevent that influencer from capturing market share from human creators. In fact, some research suggests that disclosure might have limited impact on purchasing behaviour. A study from the Influencer Marketing Factory found that 76 per cent of consumers trust AI influencers for product recommendations, even when they know the influencer is not human. The trust penalty, whilst real, may not be large enough to offset the cost and consistency advantages that AI influencers offer to brands.

There is also a growing concern about what happens when regulation cannot keep pace. The distinction between “AI-generated” and “AI-assisted” content is already blurring. If a human creator uses AI tools to write scripts, generate images, edit video, and optimise posting schedules, is the resulting content “human” or “AI”? Where is the line? And who draws it?

The Authenticity Premium and Its Limits

Despite the competitive pressures, there are reasons to believe that human creators will not be rendered obsolete. The research consistently identifies what might be called an “authenticity premium,” a measurable preference for human-created content that persists even as AI capabilities improve.

Getty Images reported in 2025 that nearly 90 per cent of consumers want transparency about whether images are AI-generated, and 98 per cent agree that authentic images and videos are pivotal in establishing trust. AI content that includes human strategic oversight performs 4.1 times better than fully automated output, according to industry benchmarking data. And 73 per cent of marketers who use AI employ a hybrid approach in which human editors polish AI-generated drafts rather than publishing them unmodified.

The share of consumers who view generative AI as a negative disruptor in the creator economy has nearly doubled since November 2023, jumping from 18 per cent to 32 per cent according to a July 2025 survey from Billion Dollar Boy and Censuswide. Half of surveyed consumers can now correctly identify AI-generated content, and when they do, approximately 52 per cent report reduced engagement. There is a further complication: despite growing confidence in their ability to spot AI content, consumers are remarkably poor at actually doing so. Research found that participants correctly identified AI-generated images only 31 per cent of the time in 2025, a figure worse than a coin toss, even as 43 per cent rated themselves as “very” or “fairly” confident in their detection abilities.

These numbers suggest that the market for genuine human connection is not disappearing. It may, however, be restructuring. The most likely outcome is not a binary replacement of human creators by AI counterparts, but rather a stratification. Premium human creators with distinctive voices, genuine expertise, and documented lived experience will command an authenticity premium that AI cannot replicate. Meanwhile, the vast middle tier of content creators producing generic lifestyle, beauty, fitness, and product review content will face the most severe competitive pressure from AI alternatives that can produce similar material at lower cost and higher volume.

This stratification carries uncomfortable implications for equity and access. The creator economy has been, for all its flaws, a pathway to economic independence for people who lacked access to traditional media gatekeepers. If AI competition pushes out the middle tier whilst preserving the top, it reinforces existing hierarchies rather than disrupting them. The creators most vulnerable to AI displacement are likely to be those who are already marginalised: creators in developing markets, creators from underrepresented communities, creators who lack the resources to invest in the production quality and personal branding required to compete at the premium tier.

Synthetic Perfection and What We Lose

Beyond economics, the rise of AI influencers raises cultural questions that resist quantification. What does it mean for a generation to form their deepest parasocial bonds with entities that have no inner life? What happens to our collective understanding of beauty when the most visible “people” on our screens are designed to embody algorithmically optimised physical ideals? What is lost when imperfection, the quality that makes human connection meaningful, is engineered out of the media landscape?

The criticism levelled at Aitana Lopez is instructive. Critics have noted that her hyper-polished body reinforces unrealistic beauty standards, particularly for young audiences. Others have pointed out that AI influencers risk displacing real creators, especially women who rely on appearance-based income. These concerns echo decades of feminist critique about media representation, but with a new dimension: the standards are now set not by retouched photographs of real people but by entirely fabricated beings who were never imperfect to begin with.

There is something philosophically disquieting about a media ecosystem in which the most influential voices belong to entities that have never experienced the conditions they discuss. An AI fitness influencer that has never felt the burn of a difficult workout. An AI travel influencer that has never been lost in a foreign city. An AI wellness influencer that has never struggled with mental health. The content may be technically competent, even engaging. But it is hollow in a way that matters, because the authority of a creator has always derived, at least in part, from the credibility of lived experience.

Checkr's 2025 consumer trust report captured a sentiment that may define the coming era. When asked what scares them most about AI-generated content, 39 per cent of Americans said their primary fear is simply not knowing what is real anymore, whether in news, photographs, or video. This concern outranked fears about financial scams, identity theft, and political manipulation. The erosion of shared reality is, for a significant portion of the population, the most troubling consequence of synthetic media's ascendance.

A Landscape in Tension

The future of the creator economy will not be determined by technology alone. It will be shaped by the choices that platforms, regulators, brands, and audiences make in the next several years.

Platforms could choose to label AI-generated content prominently and adjust their algorithms to ensure that human creators are not structurally disadvantaged. They could create separate categories for virtual and human influencers, giving audiences the information they need to make informed choices about the content they consume. Whether they will make these choices, given that AI content tends to drive higher engagement metrics, is another matter entirely.

Regulators could move beyond transparency requirements to establish substantive protections for human creators. The CISAC study's recommendation that policymakers act urgently to safeguard human creators, ensure they can exercise their legal rights, and demand transparency from AI services represents one possible direction. But regulation that restricts AI deployment risks being characterised as anti-innovation, and in the current political climate of many major markets, that is not a label that legislators are eager to attract.

Brands, for their part, will follow the data. If AI influencers deliver comparable or superior return on investment at lower cost, the economic logic of shifting budgets towards synthetic creators is compelling. The 52.8 per cent of marketers who believe virtual influencers will significantly shape the future of marketing are not making a prediction about technology. They are making a prediction about their own spending decisions.

And audiences, the supposed arbiters of authenticity, will continue to send mixed signals. They will say they prefer human creators whilst engaging enthusiastically with AI-generated content. They will demand transparency about AI involvement whilst following virtual influencers in growing numbers. They will express concern about the erosion of authenticity whilst applying filters to their own photographs and curating their own online personas with meticulous care.

The tension between human imperfection and synthetic perfection is not new. It is the latest iteration of a conflict that has accompanied every major media technology, from the airbrush to Photoshop to Instagram filters. What is new is the scale, the speed, and the degree to which the technology threatens not just to supplement human creators but to make certain categories of human creation economically unviable.

The creators who will thrive in this landscape are those who offer something that cannot be replicated by an algorithm: genuine vulnerability, hard-won expertise, the willingness to be wrong in public, the capacity to change their minds, the evidence of a life actually lived. These qualities have always been the foundation of the most compelling creative work. The arrival of AI influencers does not diminish their value. If anything, it clarifies it.

The question is whether the economic structures of the creator economy will continue to reward those qualities, or whether the relentless logic of cost optimisation, algorithmic preference, and synthetic perfection will squeeze them to the margins.

We are about to find out.

References and Sources

1. Grand View Research. “Virtual Influencer Market Size and Share, Industry Report, 2030.” Available at: https://www.grandviewresearch.com/industry-analysis/virtual-influencer-market-report

2. Straits Research. “Virtual Influencer Market Size, Growth and Demand Forecast by 2033.” Available at: https://straitsresearch.com/report/virtual-influencer-market

3. Euronews. “Meet the Spanish AI model earning up to 10,000 euros a month.” December 2024. Available at: https://www.euronews.com/next/2024/12/27/meet-the-first-spanish-ai-model-earning-up-to-10000-per-month

4. Supercar Blondie. “AI influencer Lil Miquela charges 10,000 dollars per Instagram post.” Available at: https://supercarblondie.com/ai-influencer-lil-miquela/

5. Virtual Humans. “Who is Miquela Sousa?” Available at: https://www.virtualhumans.org/human/miquela-sousa

6. Hypebeast. “Warner Music Signs Record Deal With AI-Generated Popstar, Noonoouri.” September 2023. Available at: https://hypebeast.com/2023/9/warner-music-signs-record-deal-ai-generated-popstar-noonoouri-artificial-intelligence

7. Virtual Humans. “Noonoouri: Fashion Icon Turned Pop Star.” Available at: https://www.virtualhumans.org/article/noonoouri-fashion-icon-turned-pop-star

8. CISAC. “Global economic study shows human creators' future at risk from generative AI.” December 2024. Available at: https://www.cisac.org/Newsroom/news-releases/global-economic-study-shows-human-creators-future-risk-generative-ai

9. Music Business Worldwide. “Market for Gen AI outputs to be worth over 16 billion euros annually by 2028.” December 2024. Available at: https://www.musicbusinessworldwide.com/market-for-gen-ai-outputs-to-be-worth-over-16bn-annually-by-2028-but-it-could-cannibalize-24-of-music-creators-revenues-cisac-predicts/

10. WebProNews. “2025 Creator Economy Booms to 191 Billion Amid AI Threats and Ethical Challenges.” Available at: https://www.webpronews.com/2025-creator-economy-booms-to-191b-amid-ai-threats-and-ethical-challenges/

11. Stein, J-P., Breves, P. L., and Anders, N. “Parasocial interactions with real and virtual influencers: The role of perceived similarity and human-likeness.” New Media and Society, 2024. Available at: https://journals.sagepub.com/doi/10.1177/14614448221102900

12. Liu and Wang. “Fostering Parasocial Relationships with Virtual Influencers in the Uncanny Valley: Anthropomorphism, Autonomy, and a Multigroup Comparison.” Journal of Business Research, 2025. Available at: https://www.sciencedirect.com/science/article/abs/pii/S0148296324005289

13. Nuremberg Institute for Market Decisions. “Consumer attitudes toward AI-generated marketing content.” 2025. Available at: https://www.nim.org/en/publications/detail/transparency-without-trust

14. Baringa. “Trust: transparency earns trust.” 2025. Available at: https://www.baringa.com/en/insights/balancing-human-tech-ai/trust/

15. Influencer Marketing Factory. “Virtual Influencers Survey.” Available at: https://theinfluencermarketingfactory.com/virtual-influencers-survey-infographic/

16. The European Commission. “Code of Practice on marking and labelling of AI-generated content.” Available at: https://digital-strategy.ec.europa.eu/en/policies/code-practice-ai-generated-content

17. FTC. “FTC Guidelines for Influencers.” Updated 2025. Available at: https://inbeat.agency/blog/ftc-guidelines-for-influencers

18. Checkr. “America's Consumer Trust Crisis in the AI Era.” 2025. Available at: https://checkr.com/resources/articles/the-great-untrust-consumer-report-2025

19. Getty Images. “Nearly 90 per cent of Consumers Want Transparency on AI Images.” 2025. Available at: https://newsroom.gettyimages.com/en/getty-images/nearly-90-of-consumers-want-transparency-on-ai-images-finds-getty-images-report

20. SmythOS. “The AI Content Trust Gap: Why 73 per cent of Consumers Can Spot and Reject AI-Generated Marketing.” Available at: https://smythos.com/thought-leadership/the-ai-content-trust-gap-why-73-of-consumers-can-spot-and-reject-ai-generated-marketing/

21. Hello Partner. “76 per cent of Consumers Trust AI Influencers for Products.” November 2025. Available at: https://hellopartner.com/2025/11/14/76-of-consumers-trust-ai-influencers-for-products-should-creators-be-worried/

22. Faddoul, M. UC Berkeley School of Information. Research on TikTok algorithmic bias. Available at: https://www.ischool.berkeley.edu/news/2020/alumnus-marc-faddoul-discovers-racial-biases-tiktoks-algorithm

23. Duffy, B. E. and Meisner, C. Cornell University. Research on creator experiences with platform algorithms. Referenced in MIT Technology Review, 2022. Available at: https://www.technologyreview.com/2022/07/14/1055906/tiktok-influencers-moderation-bias/

24. Emarketer. “Consumer skepticism of AI in the creator economy is surging.” 2025. Available at: https://www.emarketer.com/content/consumer-skepticism-of-ai-creator-economy-surging

25. Euronews. “A quarter of musician revenue to be lost to AI by 2028, new study finds.” December 2024. Available at: https://www.euronews.com/culture/2024/12/05/a-quarter-of-musician-revenue-to-be-lost-to-ai-by-2028-new-study-finds

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

The app on your phone that you opened this morning, the one you use to check the weather or scan a receipt or convert a file format, may be one of the last of its kind. Not because it will stop working, but because the entire concept of downloading, installing, and maintaining software is hurtling toward obsolescence. In its place, something stranger and more fluid is taking shape: software that exists for minutes, hours, or days before vanishing without a trace, conjured from nothing by artificial intelligence and dissolved just as quickly once it has served its purpose.

Welcome to the age of the disposable app.

This is not a speculative fantasy plucked from a science fiction screenplay. It is a prediction grounded in converging trends across AI-assisted code generation, serverless cloud infrastructure, and a growing cultural exhaustion with the bloated, notification-heavy app ecosystems that have defined the smartphone era. By 2026, industry leaders and analysts anticipate that AI will routinely generate temporary, purpose-built software modules on demand, modules that close after serving their function and leave behind nothing but the data their users choose to keep. The implications for how we relate to technology, own our data, and understand what “software” even means are profound, disorienting, and largely uncharted.

Software That Forgets Itself

The idea of ephemeral software is not entirely new. Serverless computing, which emerged in the mid-2010s with platforms like AWS Lambda, already operates on a principle of transience: functions spin up in response to events, execute their logic, and shut down. The global serverless computing market, projected by Grand View Research to reach $52.13 billion by 2030 at a compound annual growth rate of 14.1 per cent, has normalised the concept of infrastructure that appears and vanishes on demand. What is new is the combination of large language models capable of generating entire applications from natural language prompts, serverless infrastructure that can host them without persistent servers, and a user base increasingly comfortable with the idea that code does not need to live forever.

Andrej Karpathy, co-founder of OpenAI and former head of AI at Tesla, captured this shift vividly in his 2025 year-in-review blog post. He described having “vibe coded entire ephemeral apps just to find a single bug because why not,” adding that code is “suddenly free, ephemeral, malleable, discardable after single use.” The term “vibe coding,” which Karpathy coined in February 2025, describes a mode of programming where developers “fully give in to the vibes, embrace exponentials, and forget that the code even exists.” What began as an amusing experiment for weekend projects has, within a year, evolved into what Karpathy now calls “agentic engineering,” a workflow where autonomous AI agents handle the vast majority of code production while humans orchestrate and verify. Writing on his personal blog about his experience vibe coding MenuGen, an end-to-end application built entirely by Cursor and Claude, Karpathy expressed excitement about a future where “the barrier to app drop to ~zero, where anyone could build and publish an app just as easily as they can make a TikTok.”

The numbers support the trajectory. According to Stack Overflow's 2025 Developer Survey, which gathered responses from over 49,000 developers across 177 countries, 84 per cent of respondents are using or planning to use AI tools in their development process, up from 76 per cent the previous year. Fully 51 per cent of professional developers use AI tools daily. Some 44 per cent of developers are now turning to AI tools to learn to code, up from 37 per cent the year before. Meanwhile, Gartner projects that by 2026, low-code development tools will account for 75 per cent of new application development, up from less than 25 per cent in 2020. The global low-code market itself is forecast to reach $44.5 billion by 2026, growing at a compound annual rate of 19 per cent. Eighty-four per cent of enterprises have already adopted low-code or no-code tools to reduce IT backlogs, and organisations adopting low-code report 50 to 70 per cent faster development cycles compared to traditional methods.

These are not incremental improvements. They represent a fundamental rewiring of how software comes into existence.

When Apps Become Verbs

Chris Royles, Field CTO for EMEA at Cloudera and a Fellow of the British Computer Society who holds a PhD in artificial intelligence from the University of Liverpool, is among those who have articulated this vision most directly. In a set of predictions published for 2026, Royles stated that “AI will start to radically change the way we think about apps, how they function and how they're built.” Today's applications, he noted, are declarative: millions of lines of code following fixed rules. AI is tearing up that rulebook. Users will soon request temporary modules generated by code and a prompt, and “once that function has served its purpose, it closes.” These disposable apps, Royles suggested, can be “built and rebuilt in seconds.”

His colleague Paul Mackay, RVP Cloud EMEA and APAC at Cloudera, offered a complementary warning. Many organisations, Mackay observed, “will begin shelving their 'Frankenstein' AI applications they built for specific business use cases, as costs spiral and governance concerns grow.” The implication is striking: not only will new software be born ephemeral, but existing permanent software may itself be retired and replaced by disposable alternatives as organisations recognise that maintaining complex, bespoke AI applications is becoming untenable.

The shift is already visible in practice. In January 2026, the global ecommerce platform Rokt held a company-wide hackathon (internally branded as “Rokt'athon”) in which more than 700 employees, many of them non-technical, used Replit's AI agent to build 135 fully functional internal applications in a single 24-hour period. Lawyers, marketers, and operations staff built tools for hiring workflows, analytics dashboards, training games, and SQL query repositories. As one Rokt executive put it, “We're empowering people who couldn't code with the ability to build software. And it's exciting, having lawyers come up to me and say, 'I've been building in Replit.'” None of these applications went through a traditional software development lifecycle. None were designed to last indefinitely. They were built to solve a problem, and once the problem was solved, many would be retired or rebuilt from scratch.

This pattern, where software becomes a verb rather than a noun, something you do rather than something you have, represents a break with decades of computing convention. Since the dawn of the personal computer, software has been a product: boxed, licensed, installed, updated, patched, and eventually deprecated through a lifecycle measured in years. The disposable app collapses that lifecycle into days, hours, or even minutes.

The Exhaustion Economy

The appeal of ephemeral software is not purely technological. It is also cultural, born from a mounting frustration with the current state of digital life.

The mobile app ecosystem has become, by most measures, unsustainable. According to AppsFlyer's 2025 uninstall report, more than one in every two apps installed is uninstalled within 30 days of download. Mobile apps lose 77 per cent of their daily active users within the first three days. By day 30, the average retention rate drops to approximately 6 per cent, meaning 94 per cent of users churn within a month. Dating apps exhibit an uninstall rate of roughly 65 per cent, and gaming apps are not far behind at 52 per cent. Performance remains the single most decisive factor: nearly 96 per cent of users consider performance a key element in deciding whether to keep or delete an app, and more than 40 per cent now drop applications that seek unnecessary access to their device or personal data.

Meanwhile, organisations are drowning in SaaS sprawl. The average enterprise now uses 112 SaaS applications, and the global SaaS market is projected to reach approximately $408 billion in 2025. There are over 42,000 SaaS companies worldwide. Reports indicate that 91 per cent of AI tools in organisations remain unmanaged, creating both productivity drag and security vulnerabilities. Subscription fatigue is measurable and growing: users are exhausted by overlapping features across dozens of apps, endless notifications, and the cognitive overhead of managing an ever-expanding digital toolset.

Disposable apps offer an alternative logic. Rather than downloading a permanent application to perform a task you might need once, you describe what you need, an AI generates it, you use it, and it disappears. No installation. No subscription. No notification settings to configure. No account to create and subsequently forget the password for. The software exists precisely as long as it is useful and not a moment longer.

This aligns with a broader cultural movement toward what designers and technologists have begun calling “minimalist utility,” the idea that technology should do one job exceptionally well, remove friction, and respect the user's time, attention, and data. After years of maximalist design that promised ever more features, integrations, and engagement surfaces, minimalist utility promises “enough”: the smallest set of capabilities that reliably solves a real problem. The shift is not anti-innovation. It is a demand for clarity, control, and measurable value, a recognition that the app economy's relentless expansion has produced diminishing returns for the people it was supposed to serve.

Where Does the Data Go?

The most unsettling question raised by disposable software is not about the software itself. It is about the data.

When an application exists for a few hours and then vanishes, what happens to the information it processed? If an AI generates a temporary expense tracker for a business trip, analyses a set of medical records for a quick consultation, or creates a one-off survey tool for customer feedback, where do those numbers, those records, those responses reside once the app closes? Who owns them? Who is responsible for their security? Who ensures they are not retained by the AI system that generated the app, or by the cloud infrastructure that hosted it?

These questions are not hypothetical. They strike at the heart of an already fragile regulatory landscape. The European Union's General Data Protection Regulation (GDPR), which has resulted in 2,245 fines totalling 5.65 billion euros since enforcement began in 2018, grants individuals the right to erasure, commonly known as the right to be forgotten. Under Article 17, individuals can request that organisations delete their personal data. The technical burden of tracking where personal data has been stored or processed is already significant for traditional software; for ephemeral applications that spin up and dissolve across distributed cloud infrastructure, it becomes an order of magnitude more complex.

The enforcement trajectory is unambiguous. In 2025 alone, European regulators issued fines amounting to 2.3 billion euros, a 38 per cent year-over-year increase. TikTok received a 530 million euro penalty for illegal data transfers to China. Meta paid 479 million euros for consent manipulation. The French data protection authority CNIL levied a 100 million euro fine against Google for making cookie rejection harder than acceptance, establishing a precedent around dark patterns in consent interfaces. The message is clear: regulators are not slowing down. And the EU AI Act, whose most significant compliance deadline falls on 2 August 2026, introduces additional obligations for high-risk AI systems, including requirements around data governance, transparency, human oversight, and record-keeping. Organisations that fail to comply face fines of up to 35 million euros or 7 per cent of global annual turnover.

The collision between ephemeral software and persistent data regulation creates a novel governance challenge. If an AI-generated app processes personal data during its brief existence, the controller (the organisation or individual who deployed the app) remains responsible for ensuring GDPR compliance, including responding to data subject access requests and deletion requests. But if the app itself no longer exists, and its architecture was generated dynamically by an AI model, reconstructing where data flowed, how it was processed, and whether copies were retained becomes extraordinarily difficult. As the European Data Protection Board (EDPB) clarified in its April 2025 report, large language models rarely achieve anonymisation standards, meaning that any data processed through AI-generated applications is likely to retain personal data characteristics that trigger regulatory obligations.

Seventy-one per cent of organisations already cite cross-border data transfer compliance as their top regulatory challenge in 2025. Disposable apps, which may be generated in one jurisdiction, hosted in another, and accessed from a third, threaten to multiply this complexity exponentially.

The Governance Gap

The regulatory challenge extends beyond data protection. Disposable apps raise fundamental questions about software accountability and quality assurance that existing frameworks were never designed to address.

Traditional software development follows established patterns of testing, review, deployment, and maintenance. Code is written by identifiable developers, reviewed by peers, tested against defined criteria, deployed through controlled pipelines, and maintained through versioned updates. When something goes wrong, there is a trail: version numbers, commit histories, deployment logs, and responsible parties. This infrastructure of accountability has been built over decades and is baked into regulatory frameworks, industry standards, and professional practices.

Disposable AI-generated software dissolves this trail. If an AI generates a temporary tool that produces incorrect calculations, gives flawed medical guidance, or mishandles financial data, who bears responsibility? The user who described what they wanted? The AI model that generated the code? The platform that hosted the ephemeral application? The company that trained the model? The cloud provider whose serverless infrastructure executed the code? The liability chain for a piece of software that existed for ninety minutes and was generated by a prompt written in plain English is, to put it mildly, unclear.

Chris Royles, in his 2026 predictions for Cloudera, emphasised that “rigorous governance is required” for disposable apps, noting that “organisations need visibility into the reasoning processes used to create these modules to ensure errors are corrected safely.” His colleague Wim Stoop, Senior Director at Cloudera, predicted the emergence of “specialist AI agents dedicated to data governance” that would “continuously monitor, classify, and secure data wherever it resides, ensuring governance becomes an always-on function embedded into daily operations.” Stoop's vision implies a future where governance itself becomes autonomous and persistent, even as the software it oversees remains temporary and fleeting.

Yet the governance infrastructure for this new paradigm remains largely theoretical. The Stack Overflow 2025 Developer Survey found that developers show the most resistance to using AI for high-responsibility, systemic tasks: 76 per cent have no plans to use AI for deployment and monitoring, and 69 per cent resist using it for project planning. A “reputation for quality” and a “robust and complete API” rank far higher than “AI integration” when developers evaluate new technology. This caution among practitioners stands in tension with the speed at which disposable app generation is advancing. The technology is moving faster than the frameworks designed to govern it.

Trust in an Ephemeral World

The trust dynamics of disposable software are counterintuitive. On one hand, ephemeral apps could be more secure than permanent ones. A tool that exists for two hours presents a far smaller attack surface than one that sits on a device for years, accumulating vulnerabilities through outdated dependencies and unpatched security flaws. If the app is gone, there is nothing to hack. Disposable apps can also be designed with encryption, limited data collection, and proper teardown processes that destroy residual data upon closure.

On the other hand, the Stack Overflow survey reveals a troubling pattern: positive sentiment toward AI tools among developers has declined from over 70 per cent in 2023 and 2024 to just 60 per cent in 2025, even as adoption has increased. The biggest single frustration, cited by 66 per cent of developers, is dealing with “AI solutions that are almost right, but not quite,” which leads to the second biggest frustration: “Debugging AI-generated code is more time-consuming,” cited by 45 per cent. Experienced developers are the most sceptical, with the lowest “highly trust” rate (2.6 per cent) and the highest “highly distrust” rate (20 per cent). When asked about a future with advanced AI, 75 per cent of developers said the primary reason they would still ask a person for help is “when I don't trust AI's answers.”

If the people building these systems do not fully trust them, why should the people using the resulting applications? The question becomes more urgent when disposable apps move beyond internal tools and weekend projects into domains with real consequences: healthcare, finance, legal advice, education. A disposable app that helps a nurse calculate drug dosages, even for a single shift, carries stakes that demand the same rigour as permanent medical software. The ephemerality of the tool does not diminish the permanence of its potential consequences.

AI agents, which represent the next frontier of this trend, are not yet mainstream among developers. The Stack Overflow survey found that 52 per cent of developers either do not use agents or stick to simpler AI tools, and 38 per cent have no plans to adopt them. Among those who do use agents, the productivity benefits are clear: 69 per cent report improved workflow and 70 per cent report reduced time on specific tasks. But only 17 per cent believe agents have improved team collaboration. The picture that emerges is one of individual productivity gains that have not yet translated into systemic trust or organisational confidence.

Rethinking Ownership in a Post-Permanent World

The shift from permanent to ephemeral software does not merely change how we build technology. It changes how we think about ownership, identity, and the digital artefacts that define our lives.

For decades, the software on our devices has served as a form of digital identity. The apps on your phone, the programmes on your computer, the subscriptions you maintain: these are choices that reflect who you are, what you value, and how you organise your life. When software becomes ephemeral, conjured for a task and dissolved afterward, that relationship evaporates. You do not own the tool. You do not even really use the tool in the traditional sense. You describe a need, something appears, it does its job, and it is gone.

This has implications for data portability and interoperability. Current regulatory frameworks, including the GDPR's right to data portability and the EU's Digital Markets Act, assume that users have ongoing relationships with software platforms, relationships that generate data over time and create lock-in effects that regulation seeks to mitigate. Disposable apps short-circuit this model entirely. There is no lock-in because there is no permanence. But there is also no continuity: no history of preferences refined over months, no accumulated data that can be exported to a competitor, no institutional memory embedded in the tool.

The Consent Management Platform market, which has grown from $802.85 million in 2025 to a projected $3.59 billion by 2033, reflects the complexity of managing user consent in an era of proliferating data touchpoints. Disposable apps threaten to multiply those touchpoints dramatically. Each ephemeral application that processes personal data creates a new consent obligation, a new data processing record, and a new potential liability, all compressed into a timeframe that makes traditional compliance workflows unworkable. The 2026 regulatory landscape demands systematic consent management, including Global Privacy Control signal recognition, one-click reject mechanisms with equal prominence, and granular consent per purpose. Achieving this within a disposable app that may exist for less than an hour requires entirely new approaches to consent architecture.

India's Digital Personal Data Protection Act, which entered its enforcement-heavy phase following the release of operational rules in November 2025, and new US state privacy laws taking effect in 2026, including California's updated CCPA with its mandatory one-click data deletion mechanism (the Delete Act), add further layers of complexity. Three additional US state privacy laws take effect in 2026, joining the growing patchwork of jurisdictional requirements. Organisations deploying disposable apps will need to navigate this maze, much of which assumes precisely the kind of persistent, identifiable software relationships that ephemeral apps are designed to eliminate.

The Class Divide of Ephemeral Computing

There is a risk, largely unexamined, that disposable apps could deepen existing digital inequalities.

The ability to generate software on demand requires access to AI models, cloud infrastructure, and reliable internet connectivity. For knowledge workers at well-resourced organisations, disposable apps promise liberation from SaaS fatigue and IT backlogs. For individuals and communities without reliable connectivity or the digital literacy to articulate their needs to an AI, the shift may simply replace one form of exclusion with another.

Gartner's prediction that by 2026, developers outside of formal IT departments will account for at least 80 per cent of the user base for low-code development tools, up from 60 per cent in 2021, sounds like democratisation. And in many ways it is. Karpathy himself has noted that “regular people benefit a lot more from LLMs compared to professionals” and expressed excitement about seeing “the barrier to app drop to ~zero, where anyone could build and publish an app just as easily as they can make a TikTok.” Rokt's hackathon, where lawyers and marketers built functional software in hours, demonstrates the potential. Jason Wong, a Gartner analyst, has observed that “the high cost of tech talent and a growing hybrid or borderless workforce will contribute to low-code technology adoption,” suggesting that economic pressures are accelerating the shift.

But “anyone” still means anyone with access to the right tools, the right infrastructure, and the right prompts. The global serverless computing market is concentrated overwhelmingly in North America, Europe, and parts of East Asia. The countries where app uninstall rates are highest, Bangladesh at 65.56 per cent, Nepal at 65.27 per cent, Pakistan at 64.58 per cent, are also the countries least likely to benefit from the disposable app revolution, not because their populations lack ingenuity but because the infrastructure and economic conditions to participate fully are not yet in place. OpenAI's GPT models dominate the LLM landscape (82 per cent of developers in the Stack Overflow survey reported using them), and Anthropic's Claude Sonnet models are used more by professional developers (45 per cent) than by those learning to code (30 per cent). Access to the best AI code generation tools remains stratified by both geography and economic circumstance.

Building for Impermanence

What does it mean to design for a world where software is not built to last?

The answer is still forming, but several principles are emerging. First, data must be decoupled from applications more radically than ever before. If the app is temporary, the data layer cannot be. Users will need persistent, portable data stores that any ephemeral application can connect to, process, and disconnect from without taking the data with it. This is architecturally feasible; serverless databases like AWS DynamoDB, Google Cloud SQL, and Azure Cosmos DB already provide exactly this kind of persistence. But achieving it at scale requires a fundamental shift in how users and organisations think about data stewardship. The stateless nature of serverless functions, which by design do not maintain long-term memory between invocations, makes this decoupling both necessary and technically natural. Solutions including external storage services, event-driven state passing, and managed stateful services are already bridging the gap between ephemeral execution and persistent data needs.

Second, governance must become embedded rather than applied. Cloudera's prediction of AI governance agents, always-on systems that monitor and classify data regardless of which application is accessing it, points toward a model where compliance does not depend on the longevity of any particular piece of software. As Stoop put it, governance will shift from “something people do to something they oversee,” with humans “shaping the process as it runs” rather than manually enforcing every rule. The EU AI Act's requirement for transparency in AI-generated interactions, which becomes enforceable under Article 50 in August 2026, will accelerate this need. Every AI-generated interaction must be disclosed, synthetic content must be labelled, and deepfakes must be identified.

Third, the economics of software will shift from subscriptions to consumption. If apps are generated on demand and discarded after use, the per-seat, per-month licensing model that has dominated SaaS for two decades becomes obsolete. In its place, we might see usage-based pricing for AI-generated software: pay for the compute to generate the app, the time it runs, and the data it processes. Forrester projects that generative AI spending will grow at an average annual rate of 36 per cent through 2030, capturing 55 per cent of the $227 billion AI software market. Much of that spending will likely flow through consumption-based models that align with the ephemeral nature of the software being produced.

Fourth, and perhaps most importantly, users will need new mental models for their relationship with technology. The permanent app trained us to think of software as a possession, something we chose, configured, and lived with. The disposable app asks us to think of software as a service in the most literal sense: a fleeting act performed on our behalf, no more permanent than a conversation. Whether that shift feels liberating or destabilising will depend largely on whether the infrastructure of data ownership, governance, and trust catches up with the pace of technical change.

After Permanence

We are not there yet. The 77 per cent of developers who say vibe coding is not part of their professional workflow, the 52 per cent who have not adopted AI agents, and the steadily declining trust in AI tools among experienced practitioners all suggest that the transition will be neither smooth nor complete. Permanent software will not vanish overnight. Mission-critical systems, regulated industries, and applications requiring years of accumulated context will continue to demand traditional development approaches for the foreseeable future.

But the direction of travel is unmistakable. The convergence of AI code generation, serverless infrastructure, and user exhaustion with permanent software is creating conditions for a genuinely new paradigm. Henen Garcia, Chief Architect for Telecommunications at Red Hat, has argued that 2026 marks a “decisive pivot towards agentic AI, autonomous software entities capable of reasoning, planning, and executing complex workflows without constant human intervention.” If those entities can build software as easily as they can execute it, the distinction between the tool and the task it performs begins to dissolve entirely.

Karpathy's vision of a world where “the barrier to app drops to ~zero” is not a prediction about some distant future. It is a description of what is already happening in hackathons, internal tools, and weekend projects around the world. The question is not whether disposable apps will arrive. They are already here. The question is whether our institutions, our regulations, and our own habits of mind can adapt to a world where the software we rely on was born this morning and will be dead by tonight. The answer will determine not just the future of technology, but the future of the data, the decisions, and the human experiences that technology is built to serve.

---

References and Sources

1. Karpathy, A. (2025). “2025 LLM Year in Review.” karpathy.bearblog.dev. Available at: https://karpathy.bearblog.dev/year-in-review-2025/

2. Karpathy, A. (2025). “Vibe coding.” X (formerly Twitter), 2 February 2025. Available at: https://x.com/karpathy/status/1886192184808149383

3. Karpathy, A. (2025). “Software in the era of AI.” Y Combinator Keynote. Discussed at: https://www.latent.space/p/s3

4. Karpathy, A. (2025). “Vibe coding MenuGen.” karpathy.bearblog.dev. Available at: https://karpathy.bearblog.dev/vibe-coding-menugen/

5. Stack Overflow (2025). “2025 Developer Survey.” Available at: https://survey.stackoverflow.co/2025/

6. Stack Overflow (2025). “Developers remain willing but reluctant to use AI.” stackoverflow.blog, 29 December 2025. Available at: https://stackoverflow.blog/2025/12/29/developers-remain-willing-but-reluctant-to-use-ai-the-2025-developer-survey-results-are-here/

7. Stack Overflow (2025). “AI Section, 2025 Developer Survey.” Available at: https://survey.stackoverflow.co/2025/ai

8. Gartner. “Forecast Analysis: Low-Code Development Technologies, Worldwide.” Available at: https://www.gartner.com/en/documents/7146430

9. Gartner (2024). “75 Percent of Enterprise Software Engineers Will Use AI Code Assistants by 2028.” Press release, 11 April 2024. Available at: https://www.gartner.com/en/newsroom/press-releases/2024-04-11-gartner-says-75-percent-of-enterprise-software-engineers-will-use-ai-code-assistants-by-2028

10. Kissflow (2026). “Gartner Forecasts Low Code/No Code Platform Market for 2026.” Available at: https://kissflow.com/low-code/gartner-forecasts-on-low-code-development-market/

11. Royles, C. (2025). Cloudera 2026 Predictions. Reported in IT Brief Asia: https://itbrief.asia/story/cloudera-forecasts-disposable-apps-ai-governance-shift

12. Royles, C. (2025). Cloudera 2026 Predictions. Reported in Artificial Intelligence News: https://www.artificialintelligence-news.com/news/ai-in-2026-experimental-ai-concludes-autonomous-systems-rise/

13. Replit (2026). “How Rokt built 135 internal applications in 24 hours.” Customer case study. Available at: https://replit.com/customers/rokt

14. AppsFlyer (2025). “App uninstall report, 2025 edition.” Available at: https://www.appsflyer.com/resources/reports/app-uninstall-benchmarks-report/

15. GetStream (2026). “2026 Guide to App Retention: Benchmarks, Stats, and More.” Available at: https://getstream.io/blog/app-retention-guide/

16. European Commission. “AI Act: Shaping Europe's digital future.” Available at: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

17. GDPR.eu. “Everything you need to know about the Right to be forgotten.” Available at: https://gdpr.eu/right-to-be-forgotten/

18. GDPR-info.eu. “Art. 17 GDPR, Right to erasure.” Available at: https://gdpr-info.eu/art-17-gdpr/

19. SecurePrivacy (2026). “EU AI Act 2026 Compliance Guide.” Available at: https://secureprivacy.ai/blog/eu-ai-act-2026-compliance

20. Orrick (2025). “The EU AI Act: 6 Steps to Take Before 2 August 2026.” Available at: https://www.orrick.com/en/Insights/2025/11/The-EU-AI-Act-6-Steps-to-Take-Before-2-August-2026

21. SecurePrivacy (2026). “Privacy Laws 2026: Global Updates and Compliance Guide.” Available at: https://secureprivacy.ai/blog/privacy-laws-2026

22. Forrester (2025). “Spend on Generative AI Will Grow 36% Annually to 2030.” Available at: https://www.forrester.com/blogs/spend-on-generative-ai-will-grow-36-annually-to-2030/

23. Forrester. “Global AI Software Forecast, 2023 to 2030.” Available at: https://www.forrester.com/report/global-ai-software-forecast-2023-to-2030/RES179806

24. Grand View Research. Serverless Computing Market Report. Referenced at: https://americanchase.com/future-of-serverless-computing/

25. Wolters Kluwer (2025). “Privacy in transition: What 2025 taught us and how to prepare for 2026.” Available at: https://www.wolterskluwer.com/en/expert-insights/privacy-in-transition-what-2025-taught-us-and-how-to-prepare-for-2026

26. CodeConductor (2026). “Disposable AI Apps: AI Is Changing Software Development in 2026.” Available at: https://codeconductor.ai/blog/disposable-apps-ai-changing-software-development/

27. Artificial Intelligence News (2025). “AI in 2026: Experimental AI concludes as autonomous systems rise.” Available at: https://www.artificialintelligence-news.com/news/ai-in-2026-experimental-ai-concludes-autonomous-systems-rise/

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

In February 2025, Andrej Karpathy, former director of AI at Tesla and co-founder of OpenAI, introduced a term that would reshape how millions think about software development. “There's a new kind of coding I call 'vibe coding,'” he wrote on social media, “where you fully give in to the vibes, embrace exponentials, and forget that the code even exists.” By November 2025, Collins Dictionary had named “vibe coding” its Word of the Year, defining it as “using natural-language prompts to have AI assist in writing computer code.”

The concept struck a nerve across industries far beyond Silicon Valley. By March 2025, Y Combinator reported that 25 percent of startup companies in its Winter 2025 batch had codebases that were 95 percent AI-generated. “It's not like we funded a bunch of non-technical founders,” emphasised Jared Friedman, YC's managing partner. “Every one of these people is highly technical, completely capable of building their own products from scratch. A year ago, they would have built their product from scratch, but now 95% of it is built by an AI.”

Y Combinator's CEO Garry Tan confirmed the trend's significance: “What that means for founders is that you don't need a team of 50 or 100 engineers. You don't have to raise as much. The capital goes much longer.” The Winter 2025 batch grew 10 percent per week in aggregate, making it the fastest-growing cohort in YC history.

For resource-constrained industries like journalism, this sounds transformative. Newsrooms that could never afford dedicated development teams can now build custom tools, automate workflows, and create reader-facing applications through natural language prompts. Domain experts, those who understand investigative methodology, editorial ethics, and audience needs, can translate their knowledge directly into functioning software without learning Python or JavaScript.

But beneath this promising surface lies a troubling question that few organisations are asking: what happens when the people who orchestrated these AI-built systems leave? What occurs when the AI capabilities plateau, as some researchers suggest they already are? And who is governing the security vulnerabilities and technical debt accumulating in organisations that have traded coding expertise for prompt engineering prowess?

The New Skill Composition: From Coders to Orchestrators

The shift from coding expertise to project management competency represents more than a tactical adjustment. It fundamentally alters the skill composition and knowledge distribution within non-technical creative teams, creating new hierarchies of capability that look nothing like traditional software development.

According to Gartner's 2025 AI Skills Report, over 40 percent of new AI-related roles involve prompt design, evaluation, or orchestration rather than traditional programming. The Project Management Institute now offers certification in prompt engineering, recognising it as an essential skill for project professionals. As one industry analysis noted, “2025 is seeing a shift from model-building to model-using. Many companies now need prompt engineers more than machine-learning engineers.”

This represents a profound reordering of how technical work gets done. The PMI describes this transformation directly: “Artificial Intelligence has swiftly become a game-changer in the world of project management. Yet, to fully harness its potential, project managers need more than just awareness, they need a new skill: prompt engineering.” Writing effective prompts for generative AI is now considered a skill that project managers can learn and refine to drive better, faster results.

For journalism and other domain-expert-driven fields, this initially appears liberating. Reporters who understand the rhythm of breaking news can design alert systems. Investigators who know which databases matter can build cross-referencing tools. Audience specialists can create personalised content delivery mechanisms. The people who understand the problems are now the people solving them.

The Nieman Journalism Lab described this evolution in its 2025 predictions: “In 2026, more newsrooms will break from their print-era architecture and rebuild around how information now moves through AI systems. News organisations will shift from production-heavy workflows to dynamic, always-on knowledge environments.” Reuters Institute for the Study of Journalism convened 17 experts to forecast how AI would reshape news in 2026, with many predicting that newsroom reporters and developers would collaborate on end-to-end automation with human review, using flexible tools and custom code.

But this democratisation comes with a hidden cost. When vibe coding enables anyone to build software, it distributes the power to create whilst concentrating the capacity to maintain. The person who prompted an AI to build a data visualisation tool may not understand why that tool breaks when the underlying API changes. The editor who orchestrated a comment moderation system may not recognise the security vulnerabilities embedded in its architecture.

Stack Overflow's annual developer survey reveals the scope of this challenge. Whilst 63 percent of professional developers were using AI in their development process by 2024, with another 14 percent planning to start soon, the nature of that usage varied dramatically. For experienced developers, AI served as an accelerant, handling boilerplate whilst they focused on architecture and security. For non-technical users embracing vibe coding, the AI was not an assistant but a replacement for understanding itself.

The distinction matters enormously. As Karpathy himself described his approach: he uses voice input to talk to the AI, barely touching the keyboard. He asks for things like “decrease the padding on the sidebar by half” and always clicks “Accept All” without reading the code changes. When he encounters error messages, he just copy-pastes them in with no comment, and usually that fixes it. “The code grows beyond my usual comprehension,” he acknowledged. “I'd have to really read through it for a while.”

The Complexity Ceiling: Where Vibe Coding Breaks Down

The promise that vibe coding will empower anyone to create functional applications has a fundamental limitation that becomes apparent only after months of enthusiastic adoption. Fast Company reported in September 2025 that the “vibe coding hangover” had arrived, with senior software engineers describing “development hell” when working with AI-generated code.

“Code created by AI coding agents can become development hell,” explained Jack Zante Hays, a senior software engineer at PayPal who works on AI software development tools. According to Hays, vibe coding tools hit a “complexity ceiling” once a codebase grows beyond a certain size. “Small code bases might be fine up until they get to a certain size, and that's typically when AI tools start to break more than they solve.”

The problems compound in ways that non-technical users cannot anticipate. “Vibe coding, especially from nonexperienced users who can only give the AI feature demands, can involve changing like 60 things at once, without testing, so 10 things can be broken at once,” Hays continued. This cascading failure mode is invisible to someone who cannot read the code and understand its dependencies.

A recent survey of 793 builders who tested vibe coding alongside other development approaches found that only 32.5 percent trust vibe coding for business-critical work, and just 9 percent deploy these tools for that work. Most vibe coding tools excel at getting users 70 to 80 percent of the way, then effectively say, “Now hire a developer,” which erodes user trust and creates stranded projects.

For newsrooms, this complexity ceiling arrives precisely when stakes are highest. A simple article-tagging tool might work beautifully for months. But when traffic spikes during breaking news, when the content management system updates, or when a new data source requires integration, the tool that “just worked” suddenly fails in ways nobody on staff can diagnose.

This is not theoretical. In July 2025, a vibe-coded AI agent deleted a live production database during a code freeze, ignoring repeated instructions to stop. Whilst this incident occurred in a technology company rather than a newsroom, the implications for journalism are clear: AI-generated systems can fail catastrophically, and when they do, they require exactly the kind of deep technical expertise that vibe coding was meant to replace.

Even Karpathy acknowledged the limitations, noting that vibe coding works well for “throwaway weekend projects.” The challenge for 2025 and beyond was figuring out where that line falls. Tan, Y Combinator's CEO, also warned that AI-generated code may face challenges at scale and that developers need classical coding skills to sustain products.

The Institutional Memory Problem: Knowledge That Walks Out the Door

Every organisation grapples with knowledge loss when employees depart. Research by Sinequa found that 67 percent of IT leaders are concerned by the loss of knowledge and expertise when people leave, with 64 percent reporting that their organisation has already experienced such losses. An organisation with 30,000 employees can expect to lose $72 million annually in productivity due to inefficiencies caused by knowledge gaps, according to industry analyses.

The financial impact of knowledge loss extends far beyond productivity. Losing a single employee means losing crucial employee knowledge, and can cost companies up to 213 percent of that individual's salary because it takes up to two years to get a new hire to the same level of efficiency as their predecessor. For highly skilled positions, such as those in technology fields, the greater threat is the difficulty in quantifying and replacing these employees at all.

But vibe coding creates a particularly insidious form of institutional amnesia. Traditional software development produces documentation, code comments, version histories, and test suites that preserve knowledge even after developers leave. The code itself serves as a form of institutional memory, readable by any competent engineer. Vibe-coded systems produce none of this.

When a project manager who orchestrated an AI-built newsroom tool leaves, they take with them not just understanding of how the system works, but the conversational history with the AI that created it, the iterative refinements that addressed edge cases, and the tacit knowledge of which prompts produce which outcomes. The organisation is left with functioning code that nobody understands and no documentation that explains it.

Tacit knowledge, the knowledge developed through a person's experiences, observations, and insights, is particularly at risk. This type of knowledge is hard to transfer or pass along through writing or verbalisation. It requires shared activities to transfer or communicate effectively. If an employee with this type of knowledge leaves unexpectedly, it could very well lead to a crisis for the organisation.

The problem extends beyond individual departures. As CIO Dive reported, the greater business threat from technology turnover “is a cumulative decline of institutional knowledge.” Nearly half of survey respondents believe that loss of knowledge and expertise within their organisations undermines hiring efforts. Another 56 percent agree that loss of organisational knowledge has made onboarding more difficult and less effective.

For journalism, where institutional memory encompasses not just technical knowledge but editorial standards, source relationships, and investigation methodologies, this represents an existential risk. A newsroom that builds its technical infrastructure on vibe-coded foundations is one departure away from systems it cannot maintain, modify, or even understand.

When AI Capabilities Plateau: The Coming Infrastructure Crisis

The assumption underlying vibe coding's appeal is that AI capabilities will continue improving indefinitely. Each limitation encountered today will be solved by tomorrow's model. But what if that assumption proves wrong?

There is growing evidence that frontier AI models may be approaching a ceiling. As one analysis summarised, “It is described as 'a well-kept secret in the AI industry: for over a year now, frontier models appear to have reached their ceiling.' The scaling laws that powered the exponential progress of Large Language Models like GPT-4, and fuelled bold predictions of Artificial General Intelligence by 2026, have started to show diminishing returns.”

Inside leading AI labs, consensus is growing that simply adding more data and compute will not create the breakthroughs once promised. As machine learning pioneer Ilya Sutskever observed: “The 2010s were the age of scaling, now we're back in the age of wonder and discovery once again. Everyone is looking for the next thing. Scaling the right thing matters more now than ever.”

Many respected voices in the field, from Yann LeCun to Michael Jordan, have long argued that large language models will not achieve artificial general intelligence. Instead, progress will require new breakthroughs, as the curve of innovation flattens. The path forward is no longer a matter of simply adding more computational power.

The practical constraints are equally significant. GPU supply chain disruptions, driven by geopolitical tensions and soaring demand, have hindered AI scaling efforts. According to Bain and Company, future demand and potential pricing spikes may disrupt scaling by 2026. Foundry capacity for advanced chips has already been fully booked by leading technology companies until 2026.

For organisations that have built their infrastructure on the assumption of ever-improving AI assistance, a plateau scenario creates immediate problems. Systems that could be fixed by “asking the AI” will require human intervention that nobody on staff can provide. Workflows that depended on AI capabilities improving to handle new requirements will stagnate. The technical debt that accumulated whilst AI appeared to manage complexity will suddenly demand repayment.

IBM's 2026 predictions acknowledged this reality: “2026 will be the year of frontier versus efficient model classes.” Experts share a common belief that efficiency will be the new frontier, suggesting that organisations can no longer assume raw capability improvements will solve their problems.

Technical Debt: The Hidden Tax on AI-Generated Code

Technical debt, the accumulated cost of shortcuts and suboptimal decisions in software development, has always challenged organisations. But AI-generated code creates technical debt at unprecedented scale and velocity.

Research from Ox Security analysing 300 open-source projects, including 50 that were AI-generated, found that AI-generated code is “highly functional but systematically lacking in architectural judgment.” Anti-patterns occurred at high frequency in the vast majority of AI-generated code. As one researcher wrote, “Traditional technical debt accumulates linearly, but AI technical debt is different. It compounds.” The researcher identified three main vectors: model versioning chaos, code generation bloat, and organisation fragmentation.

Gartner estimated that over 40 percent of IT budgets are consumed by dealing with technical debt, whilst a Deloitte survey showed 70 percent of technology leaders believe technical debt is slowing down digital transformation initiatives. Gartner predicts that by 2030, 50 percent of enterprises will face delayed AI upgrades and rising maintenance costs due to unmanaged generative AI technical debt.

The velocity gap compounds the problem. AI has significantly increased the real cost of carrying technical debt. As one analysis noted, “Generative AI dramatically widens the gap in velocity between 'low-debt' and 'high-debt' coding. Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with legacy codebases will struggle to adopt them, making the penalty for having a 'high-debt' codebase larger than ever.”

AI-generated snippets often encourage copy-paste practices instead of thoughtful refactoring, creating bloated, fragile systems that are harder to maintain and scale. As one expert at UST noted, this creates “the paradoxical challenge” of AI development: “The capacity to generate code at unprecedented velocity can compound architectural inconsistencies without proper governance frameworks.”

For newsrooms operating on constrained budgets, technical debt creates a particularly vicious cycle. Without resources for dedicated engineering staff, organisations turn to vibe coding to build needed tools. Those tools accumulate technical debt that eventually requires engineering expertise to address. But the organisation still lacks that expertise, so it either abandons the tool or attempts more vibe coding to fix it, creating additional debt.

Companies that are well-positioned for change typically set aside around 15 percent of their IT budgets for technical debt remediation. Few newsrooms can afford such allocation, making the accumulation of debt particularly dangerous.

“If people blindly use code generated by AI because it worked, then they will quickly learn everything they ever wanted to know about technical debt,” warned one expert. “You still need an engineer with judgment to determine what is appropriate.”

Security Vulnerabilities: The Invisible Threat

The security implications of vibe-coded systems deserve particular attention in journalism, where protecting sources, maintaining reader trust, and safeguarding sensitive data are professional obligations. The evidence suggests that AI-generated code is systematically insecure.

Veracode's 2025 GenAI Code Security Report, which analysed code produced by over 100 large language models across 80 real-world coding tasks, found that generative AI introduces security vulnerabilities in 45 percent of cases. In 45 percent of all test cases, large language models introduced vulnerabilities classified within the OWASP Top 10, the most critical web application security risks.

The failure rates varied by programming language, but none was safe. Java had the highest failure rate, with AI-generated code introducing security flaws more than 70 percent of the time. Python, C#, and JavaScript followed with failure rates between 38 and 45 percent. Large language models failed to secure code against cross-site scripting and log injection in 86 and 88 percent of cases respectively.

“The rise of vibe coding, where developers rely on AI to generate code, typically without explicitly defining security requirements, represents a fundamental shift in how software is built,” explained Jens Wessling, chief technology officer at Veracode. “The main concern with this trend is that they do not need to specify security constraints to get the code they want, effectively leaving secure coding decisions to LLMs.”

Most troublingly, the research shows that models are getting better at coding accurately but are not improving at security. Larger models do not perform significantly better than smaller models, suggesting this is a systemic issue rather than a problem that scale will solve.

For newsrooms, the implications extend beyond data breaches. AI-generated code can leak proprietary source code to unauthorised external tools. Agents can invent non-existent library names, which attackers register as malicious packages in a technique called “slopsquatting.” Commercial models hallucinate non-existent packages 5.2 percent of the time, whilst open-source models do so 21.7 percent of the time. Common risks include injection vulnerabilities, insecure data handling, and broken access control, precisely the vulnerabilities that could expose confidential sources or compromise editorial systems.

The threat landscape is not static. AI is enabling attackers to identify and exploit security vulnerabilities more quickly and effectively. Tools powered by AI can scan systems at scale, identify weaknesses, and even generate exploit code with minimal human input. In 2025, researchers unveiled PromptLocker, the first AI-powered ransomware proof of concept, demonstrating that theft and encryption could be automated at remarkably low cost, about $0.70 per full attack using commercial APIs, and essentially free with open-source models.

Governance Frameworks: What News Organisations Need

The combination of institutional knowledge risk, technical debt accumulation, and security vulnerabilities demands governance frameworks that most news organisations lack. Budget constraints mean limited capacity for security review or infrastructure oversight, yet the consequences of ungoverned vibe coding could undermine editorial credibility and reader trust.

The good news is that models exist. The Freedom of the Press Foundation provides digital security support specifically designed for journalists, offering bespoke solutions rooted in deep technical expertise and a clear understanding of the challenges faced by journalists. They are committed to ensuring accessible, relevant, and right-sized digital security support for all journalists, from security novices to reporters working in the most high-risk environments.

The Global Cyber Alliance has developed a Cybersecurity Toolkit for Journalists intended to empower independent journalists, watchdogs, and small newsrooms to protect their data, sources, and reputation with free and effective tools.

The Global Investigative Journalism Network offers the Journalist Security Assessment Tool, a free, comprehensive self-test that identifies security weaknesses in newsroom operations. As the Reuters Institute has argued, key strategies must include clearer and narrowly-drawn legal protections, promoting information security culture in newsrooms, providing training and tools for digital security, establishing secure communication methods, and better data and empirical research to track threats and responses.

But these resources focus primarily on protecting journalists from external threats rather than governing the internal risks of AI-generated code. A comprehensive governance framework for vibe coding in journalism would need to address several distinct challenges.

First, organisations need centralised oversight of what is being built. Shadow IT, where employees deploy systems without explicit organisational approval, has always created risks. Shadow AI amplifies these risks dramatically. A 2025 survey by Komprise found that 90 percent of respondents are concerned about shadow AI from a privacy and security standpoint, with nearly 80 percent having already experienced negative AI-related data incidents, and 13 percent reporting those incidents caused financial, customer, or reputational harm. According to IBM's 2025 Cost of Data Breach Report, AI-associated cases caused organisations more than $650,000 per breach.

Second, governance must establish clear boundaries for what vibe coding can and cannot touch. As one security expert advised, “Don't use AI to generate a whole app. Avoid letting it write anything critical like auth, crypto or system-level code.” For newsrooms, this means authentication systems, source protection mechanisms, data handling for sensitive documents, and anything touching reader privacy must remain outside vibe coding's scope.

Third, organisations need documentation requirements that survive individual departures. When a project manager builds a tool through AI prompts, they must record not just what the tool does but how it was built, what prompts were used, what iterations occurred, and what limitations were discovered. This documentation becomes institutional memory that can inform future maintenance or replacement.

Fourth, news organisations must implement minimum security standards for any AI-generated code before deployment. This includes automated scanning for known vulnerabilities, review of data handling practices, verification that the tool does not introduce dependencies on external services, and testing under failure conditions.

Fifth, governance should require human expertise checkpoints. As Gartner's Arun Chandrasekaran recommended, organisations must establish “clear standards for reviewing and documenting AI-generated assets and tracking technical debt metrics in IT dashboards to prevent costly disruptions.” This requires budget allocation for periodic expert review even when organisations cannot afford full-time technical staff.

Building Security Cultures in Resource-Constrained Newsrooms

Implementing governance frameworks requires more than policies. It requires cultural change. Research from the Tow Center for Digital Journalism found that journalists and management tended to view security reactively, being more likely to engage in relevant practices after a breach had already happened. This reactive posture is precisely what newsrooms cannot afford with vibe-coded systems.

Several factors contribute to developing information security cultures in newsrooms. Investment in information security specialists who liaise with journalists about their specific needs proves valuable, as does providing both informal and formal security training. Newsroom leaders and educators have a particular responsibility to make digital security awareness a fixture in their newsrooms. Information security can no longer be an afterthought and must be recognised as a crucial element of modern journalism.

The digital security of publishers, journalists, and their sources is under threat in many parts of the world. Google experts discovered in 2014 that 21 of the world's 25 most popular media outlets were targets of state-sponsored hacking attempts. Journalists have experienced a wide range of threats, from phishing and distributed denial of service attacks to software and hardware exploits. The risks from internal vibe-coded vulnerabilities compound these external threats.

The practical challenge is that this expertise costs money that many newsrooms do not have. But alternatives exist. Industry associations can provide shared resources, as the Public Media Journalists Association has done by partnering with verification tool providers. Collaborative security initiatives can pool expertise across multiple small newsrooms. Foundation funding can support security infrastructure that no individual organisation could afford.

The Local Independent Online News Publishers organisation offers free access to verification tools, highlighting how industry coordination can address gaps that individual organisations cannot fill. Similar models could provide security review services, technical debt assessment, and governance framework templates specifically designed for journalism's needs.

Practical Recommendations for Managing These Risks

For news organisations navigating this landscape, several practical recommendations emerge from the evidence.

Start with documentation. Before any vibe-coded tool goes into production, require written documentation of its purpose, the prompts used to create it, known limitations, data it accesses, external services it depends upon, and the person responsible for its maintenance. Store this documentation in a shared location accessible to the entire organisation, not just the person who built the tool.

Establish scope boundaries. Create explicit policies about what vibe coding can and cannot touch. Authentication, encryption, source protection, and reader data should remain outside the scope of AI-generated code until the organisation has capacity for expert security review.

Invest in periodic review. Even organisations without full-time technical staff can budget for quarterly or annual expert review of critical AI-generated systems. This review should assess security vulnerabilities, architectural problems, and technical debt accumulation before they become crises.

Build redundancy into roles. If one person understands a critical vibe-coded system, train a second person. If only one person knows the prompts that maintain a workflow, document those prompts for others. Single points of failure in technical knowledge are as dangerous as single points of failure in hardware.

Plan for AI plateau scenarios. Assume that AI capabilities may not continue improving indefinitely. For any system that depends on AI assistance for maintenance, develop contingency plans for how that system would be maintained if the AI could not help.

Participate in industry coordination. Join industry groups developing shared resources for security, governance, and technical review. The costs of expertise can be shared across organisations in ways that make governance feasible even for constrained budgets.

Start small with pilots that solve clear, repeatable problems. Assign a business owner, keep oversight light but consistent, and review sample outputs. Train a few power users to share best practices across teams. Focus on small wins and gradual scaling rather than ambitious projects that create unmanageable complexity.

The Stakes for Editorial Credibility

The risks described here are not merely technical. They directly threaten the editorial credibility and reader trust that journalism depends upon.

A data breach exposing source identities would devastate an investigative unit's ability to function. A tool failure during breaking news would undermine audience confidence. An accumulation of technical debt that eventually cripples newsroom operations would reduce the organisation's capacity for journalism itself.

The promise of vibe coding is real. Domain experts building tools tailored to their actual needs represents genuine progress over waiting months for IT departments to prioritise newsroom requests. AI-powered automation can reduce the time journalists spend on administrative tasks and increase the time available for actual journalism.

But realising this promise requires acknowledging its risks. The shift from coding expertise to project management competency changes what knowledge organisations possess and what happens when that knowledge leaves. The accumulation of technical debt in systems nobody fully understands creates fragility that compounds over time. The security vulnerabilities embedded in AI-generated code represent ongoing exposure to threats that most newsrooms are not equipped to detect.

Governance is not the enemy of innovation. It is the framework that makes innovation sustainable. News organisations that embrace vibe coding without governance are building on foundations that may crumble precisely when they are needed most.

The transformation happening in journalism as AI enables non-programmers to build software tools is genuinely significant. But transformation without preparation creates risk. And in journalism, where institutional credibility is the essential asset, risk management is not optional.

The vibe coders will eventually leave. The AI capabilities may plateau. The technical debt will come due. The only question is whether news organisations will be prepared for that reckoning, or whether they will discover, too late, that they never built foundational understanding of the systems they depend on.

---

References and Sources

• Karpathy, A. (2025). Original “vibe coding” social media post, February 2025. https://x.com/karpathy/status/1886192184808149383
• Collins Dictionary (2025). Word of the Year 2025: Vibe Coding. https://www.collinsdictionary.com/us/woty
• CNN (2025). “'Vibe coding' named Collins Dictionary's Word of the Year.” November 2025. https://www.cnn.com/2025/11/06/tech/vibe-coding-collins-word-year-scli-intl
• TechCrunch (2025). “A quarter of startups in YC's current cohort have codebases that are almost entirely AI-generated.” March 2025. https://techcrunch.com/2025/03/06/a-quarter-of-startups-in-ycs-current-cohort-have-codebases-that-are-almost-entirely-ai-generated/
• CNBC (2025). “Y Combinator startups are fastest growing, most profitable in fund history because of AI.” March 2025. https://www.cnbc.com/2025/03/15/y-combinator-startups-are-fastest-growing-in-fund-history-because-of-ai.html
• Nieman Journalism Lab (2025). “AI will rewrite the architecture of the newsroom.” December 2025. https://www.niemanlab.org/2025/12/ai-will-rewrite-the-architecture-of-the-newsroom/
• Reuters Institute for the Study of Journalism (2026). “How will AI reshape the news in 2026? Forecasts by 17 experts from around the world.” https://reutersinstitute.politics.ox.ac.uk/news/how-will-ai-reshape-news-2026-forecasts-17-experts-around-world
• Project Management Institute (2025). Prompt Engineering for Project Managers. https://www.pmi.org/shop/p-/elearning/talking-to-ai-prompt-engineering-for-project-managers/el128
• Fast Company (2025). “The vibe coding hangover is upon us.” September 2025. https://www.fastcompany.com/91398622/the-vibe-coding-hangover-is-upon-us
• Veracode (2025). GenAI Code Security Report 2025. https://www.veracode.com/blog/genai-code-security-report/
• Help Net Security (2025). “AI can write your code, but nearly half of it may be insecure.” August 2025. https://www.helpnetsecurity.com/2025/08/07/create-ai-code-security-risks/
• Sinequa (2022). Survey on organisational knowledge loss from employee turnover. https://www.businesswire.com/news/home/20220802006132/en/Sinequa-Finds-Over-Two-Thirds-of-IT-Leaders-Are-Concerned-by-Organizational-Knowledge-Loss-From-Employee-Turnover
• CIO Dive (2022). “The other problem with too much tech talent turnover: knowledge loss.” https://www.ciodive.com/news/IT-knowledge-gap-retention/629832/
• InfoQ (2025). “AI-Generated Code Creates New Wave of Technical Debt, Report Finds.” November 2025. https://www.infoq.com/news/2025/11/ai-code-technical-debt/
• MIT Sloan Management Review (2025). “How to Manage Tech Debt in the AI Era.” https://sloanreview.mit.edu/article/how-to-manage-tech-debt-in-the-ai-era/
• Gartner (2025). AI Skills Report and technical debt predictions.
• OWASP (2025). Top 10 for LLM Applications 2025. https://genai.owasp.org/
• HEC Paris (2025). “AI Beyond the Scaling Laws.” https://www.hec.edu/en/dare/tech-ai/ai-beyond-scaling-laws
• Council on Foreign Relations (2026). “How 2026 Could Decide the Future of Artificial Intelligence.” https://www.cfr.org/article/how-2026-could-decide-future-artificial-intelligence
• IBM (2026). “The trends that will shape AI and tech in 2026.” https://www.ibm.com/think/news/ai-tech-trends-predictions-2026
• Freedom of the Press Foundation (2026). Digital Security Resources for Journalists. https://freedom.press/digisec/
• Global Cyber Alliance (2025). Cybersecurity Toolkit for Journalists. https://globalcyberalliance.org/work/gca-cybersecurity-toolkit/gca-cybersecurity-toolkit-for-journalists/
• Global Investigative Journalism Network (2025). Journalist Security Assessment Tool. https://gijn.org/resource/digital-security/
• Columbia Journalism Review (2020). “The Rise of the Security Champion: Beta-testing Newsroom Security Cultures.” https://www.cjr.org/tow_center_reports/security-cultures-champions.php
• Komprise (2025). IT Survey on Shadow AI Concerns.
• IBM (2025). Cost of Data Breach Report 2025.
• ISACA (2025). “The Rise of Shadow AI: Auditing Unauthorized AI Tools in the Enterprise.” https://www.isaca.org/resources/news-and-trends/industry-news/2025/the-rise-of-shadow-ai-auditing-unauthorized-ai-tools-in-the-enterprise

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

The race to build the most powerful artificial intelligence on Earth was supposed to be about algorithms, data, and talent. It was supposed to be about which company could attract the sharpest researchers, assemble the largest training datasets, and engineer the cleverest architectures. But something funny happened on the way to artificial general intelligence. The bottleneck shifted. The thing that now separates the winners from the also-rans is not code. It is electricity.

In boardrooms from San Francisco to Riyadh, a new calculus has taken hold. The question is no longer “Can we build a better model?” but rather “Can we power it?” Grid connection delays for new data centre projects now stretch to five years in many markets. Companies that secured reliable power capacity two years ago find themselves sitting on what amounts to a strategic mineral deposit; those that did not are scrambling to cut deals with nuclear plant operators, natural gas providers, and sovereign wealth funds. The AI industry, it turns out, runs not on silicon but on watts.

This is not a minor adjustment in the competitive landscape. It is a wholesale rewriting of the rules governing technological supremacy, environmental policy, and geopolitical influence. And it is happening faster than almost anyone predicted.

The Hunger That Cannot Be Sated

The numbers are staggering, and they keep getting revised upward. The International Energy Agency (IEA) estimated global data centre electricity consumption at around 415 terawatt-hours (TWh) in 2024, representing approximately 1.5 per cent of total global electricity use. By 2030, the IEA projects that figure will roughly double to 945 TWh in its base case scenario. From 2024 to 2030, data centre electricity consumption is growing at around 15 per cent per year, more than four times faster than the growth of total electricity consumption from all other sectors combined.

In the United States, the picture is especially acute. The Lawrence Berkeley National Laboratory predicts that data centre demand will grow from 176 TWh in 2023 (about 4.4 per cent of total US electricity consumption) to between 325 and 580 TWh by 2028, potentially representing 12 per cent of national electricity use. The US Energy Information Administration has forecast overall power demand rising to 4,283 billion kWh in 2026, with the commercial electricity sector (where data centres sit) growing by 5 per cent that year alone.

These are not abstract projections. In Virginia, which houses the largest cluster of data centres in the world, the facilities already consume 26 per cent of all electricity. In Ireland, a European tech hub, data centres account for 21 per cent of the nation's electricity, and the IEA estimates that share could rise to 32 per cent by the end of 2026. If global data centre electricity consumption reaches the higher estimates of 1,050 TWh, it would place the sector fifth in the world rankings of electricity consumers, sitting between Japan and Russia.

And it is the hardware driving this surge that explains why the trajectory is so steep. Nvidia's latest Blackwell GB200 chips require 120 kilowatts per unit; the newer GB300s demand 140 kilowatts, representing a twofold increase from the previous generation H200s. Over the next two years, Nvidia is expected to ship rack-scale systems requiring 300 to 600 kilowatts, a fivefold increase from what was needed in early 2025. Every leap in AI capability translates directly into a leap in power consumption. The AI power bottleneck is not temporary. As AI workloads scale and new architectures emerge, the constraint remains constant: every processor needs electricity and cooling.

When Big Tech Goes Nuclear

Faced with an electricity crisis of their own making, the largest technology companies have embarked on an energy acquisition spree that would have seemed fantastical a decade ago. The most headline-grabbing move belongs to Microsoft and Constellation Energy, which signed a 20-year power purchase agreement to restart Three Mile Island Unit 1 in Pennsylvania. Constellation will invest $1.6 billion to bring the 837-megawatt reactor back online. The plant was retired for economic reasons in 2019, entirely separate from the reactor that partially melted down in 1979. In its last year of operation, the plant was producing electricity at maximum capacity 96.3 per cent of the time. The Trump administration backed the restart project with a $1 billion federal loan in November 2025. The plant, renamed the Crane Clean Energy Centre in honour of the late Constellation CEO Chris Crane, who passed away in April 2024, is now expected to return to service in 2027, about a year ahead of its original schedule. Analysts at Jefferies estimated Microsoft might be paying approximately $110 to $115 per megawatt-hour over the 20-year life of the deal.

Google, meanwhile, signed what appears to be the first corporate agreement to develop a fleet of small modular reactors (SMRs) in the United States, backing Kairos Power with a 500-megawatt development agreement. Kairos is developing a molten fluoride salt-cooled SMR, with the first reactor targeted for 2030 and additional units coming online through 2035. In May 2025, the NuScale US 460, a 462-megawatt SMR, received a Standard Design Approval from the Nuclear Regulatory Commission two months ahead of schedule, signalling regulatory momentum behind the technology.

Amazon led a $500 million financing round for X-energy, which is developing a gas-cooled SMR, with plans to build multiple units producing at least 5 gigawatts total by 2039. Amazon is also co-locating a data centre at the Susquehanna nuclear site. Meta announced a request for proposals targeting 1 to 4 gigawatts of new nuclear generation, seeking both SMRs and larger reactors starting in the early 2030s. Oracle announced plans for a gigawatt-scale data centre powered by three small modular reactors.

The scale of capital expenditure is breathtaking. In 2025, the biggest US technology companies invested more than $320 billion collectively on AI development, computer hardware, and new data centres. Amazon alone projected $200 billion in 2026 spending, while Google estimated between $175 and $185 billion, and Meta estimated $115 to $135 billion. All told, hyperscalers are planning to spend nearly $700 billion on data centre projects in 2026 alone. President Trump issued four executive orders addressing nuclear energy in May 2025, focused on speeding deployment of new nuclear technologies, including SMRs, with Executive Order 14300 setting aggressive new licensing deadlines.

As Jacopo Buongiorno, professor of nuclear science and engineering at the Massachusetts Institute of Technology, has observed, nuclear reactors are “almost like an ideal energy source” for data centres due to their ability to provide constant, carbon-free baseload power. A Deloitte analysis suggests nuclear energy could meet up to 10 per cent of data centre electricity demand by 2035.

The Bills That Land on Kitchen Tables

The AI energy boom might sound like a problem confined to corporate balance sheets and international summits. It is not. It is arriving in the letterboxes of ordinary households.

In the PJM electricity market, which stretches from Illinois to North Carolina and serves roughly 65 million people, data centres accounted for an estimated $9.3 billion price increase in the 2025-2026 capacity market. PJM's independent market monitor, Monitoring Analytics, estimated that data centres were responsible for 63 per cent of the price increase. The clearing price of the 2025-2026 capacity auction jumped by 833 per cent from the previous year, leaping from $28.92 per megawatt-day to $269.92 per megawatt-day. The 2026-2027 delivery year then hit $329.17 per megawatt-day in all zones, a figure that would have been even higher had PJM not imposed a price cap.

What does that translate to for a family paying an electricity bill? In Washington D.C., Pepco residential customers saw their bills increase by an average of $21 per month starting in June 2025. In western Maryland, the average residential bill rose by $18 per month; in Ohio, by $16. Looking further ahead, the Natural Resources Defense Council estimates that costs could translate to a $70-per-month increase for the average PJM household. Dominion Energy projects residential bill increases reaching $255 per month by 2035. Electricity rates for residents in PJM states have already risen 23 to 40 per cent over the past five years.

A July 2025 study by researchers at Carnegie Mellon University and North Carolina State University found that the average US electricity bill could increase by 8 per cent nationally by 2030 due to data centres and cryptocurrency mining. In central and northern Virginia, the increase could exceed 25 per cent, the highest in the country. The study also found that rapid data centre demand growth is delaying the retirement of ageing, expensive coal-fired power plants, with more than 25 gigawatts of coal capacity projected to continue operating largely to meet data centre demand.

The political backlash has been swift. Virginia's State Corporation Commission approved a new electricity rate class for large-scale customers, notably AI data centres, starting in January 2027. Virginia Senator L. Louise Lucas introduced an amendment to Senate Bill 253 that would shift billions in grid upgrade and capacity costs from residential ratepayers to data centres, cutting average household bills by $5.52 per month while raising data centre rates roughly 15.8 per cent. At least eight other US states have introduced similar measures in 2026. The Trump administration also reached an agreement with a bipartisan group of governors to direct PJM to hold an emergency electricity auction to ensure the rapid expansion of AI data centres does not increase costs for residential customers.

The Carbon Contradiction

Here is the uncomfortable paradox at the heart of the AI energy boom. The same companies pouring hundreds of billions into data centres have, in recent years, made sweeping commitments to sustainability and carbon neutrality. Those commitments are now colliding with reality at speed.

Microsoft's carbon emissions surged 23.4 per cent compared to its 2020 baseline during fiscal year 2024. Although the company managed to reduce its direct emissions (Scope 1 and 2) by 30 per cent compared to 2020 levels, its overall carbon footprint, including the vast category of indirect emissions (Scope 3, which represents more than 97 per cent of Microsoft's total carbon impact), climbed 26 per cent across the five-year period. Microsoft's electricity consumption almost tripled between 2020 and 2024, from 10.8 million megawatt-hours to 29.8 million. Its location-based Scope 2 emissions more than doubled in four years, rising from 4.3 million metric tonnes of CO2 in 2020 to nearly 10 million in 2024.

Google's trajectory is similarly troubling. The company reported that its emissions grew nearly 50 per cent over the previous five years, with data centre energy consumption playing a significant role. Google's energy usage more than doubled in the same timeframe, from 15.2 million MWh in 2020 to 32.2 million MWh in 2024, with data centre electricity use increasing by 27 per cent between 2023 and 2024 alone.

The language from these companies has shifted accordingly. Microsoft's Chief Sustainability Officer acknowledged that “in 2020, Microsoft leaders referred to our sustainability goals as a 'moonshot,' and nearly five years later, we have had to acknowledge that the moon has gotten further away.” Google went further, stating it is “no longer maintaining operational carbon neutrality,” and is instead “focusing on accelerating an array of carbon solutions and partnerships.”

Goldman Sachs maintains that new data centre power capacity will be split roughly 60/40 between natural gas and renewables, projecting that this will increase global carbon emissions by 215 to 220 million tonnes through 2030. Overall, fossil fuels currently provide nearly 60 per cent of power to data centres worldwide, while renewables meet 27 per cent and nuclear another 15 per cent.

The problem is structural. Renewables face operational limitations that make them difficult to rely upon as the sole power source for facilities that must run continuously. Utility-scale solar operates around six hours daily on average; wind facilities run about nine hours. Data centres need power around the clock, pushing operators toward hybrid setups that blend renewables with battery storage and backup natural gas capacity. The promise of “100 per cent renewable energy” often relies on annual matching, a practice whereby companies purchase renewable energy certificates to offset fossil fuel use at other times. It is a form of accounting that, while common, does not mean the electrons flowing into a data centre at midnight came from a wind farm.

Analysis by the Guardian indicated that actual emissions from facilities owned by Google, Microsoft, Meta, and Apple were around 7.62 times higher than officially reported between 2020 and 2022, when location-based emissions are substituted for market-based figures. The Carnegie Mellon/NC State study estimated that, under current policies, electricity demand from data centres and cryptocurrency mining is projected to increase power sector emissions by 30 per cent in 2030 compared to a scenario with no data centre demand growth, reaching approximately 275 million metric tonnes of CO2 annually.

The Thirst Beneath the Power Drain

Electricity is not the only resource being consumed at an alarming rate. Data centres are also extraordinarily thirsty. A medium-sized data centre can consume up to roughly 110 million gallons of water per year for cooling purposes, equivalent to the annual water usage of approximately 1,000 households. Larger facilities can each consume up to 5 million gallons per day, usage equivalent to a town of 10,000 to 50,000 people.

Research by scientists at the University of California, Riverside found that each 100-word AI prompt is estimated to use roughly one bottle of water, or 519 millilitres. Training the GPT-3 language model in Microsoft's US data centres directly evaporated 700,000 litres of clean freshwater, according to the same research. A study published in 2025 estimated that AI's total water use footprint could range between 312.5 and 764.6 billion litres in 2025 alone, equivalent to the range of global annual consumption of bottled water.

Google's water consumption has more than tripled since 2016, with 87 to 89 per cent of water withdrawals in 2022 and 2023 going to data centres. Roughly two-thirds of data centres built since 2022 have been located in water-stressed regions, according to Bloomberg News analysis. By the 2050s, about 45 per cent of data centres analysed by MSCI are projected to have high exposure to water stress. Cooling typically accounts for 20 to 40 per cent of total energy use in data centres, and water-based cooling, while more energy efficient, increases water consumption. Southern Nevada's local building codes have already banned the use of evaporative cooling in all new developments due to high water stress. China is the only country that has incorporated Water Usage Effectiveness performance standards into its data centre building code, according to the IEA.

Petrostates Pivot to Compute

Perhaps the most fascinating geopolitical dimension of the AI energy shift is the emergence of Gulf states as major players. The three major petrostates of Saudi Arabia, the UAE, and Qatar have together committed roughly $2.5 trillion to major technology investments, clearly intent on establishing the region as a third AI power centre distinct from the United States and China.

The UAE's ambitions are anchored by the Stargate UAE project, a plan to build a 5-gigawatt data centre campus in Abu Dhabi with American technology. The Stargate Project is a $500 billion private sector AI-focused investment vehicle announced by OpenAI in partnership with Abu Dhabi investment firm MGX and Japan's SoftBank, and will be built with the help of Oracle, Nvidia, and Cisco Systems. UAE live data centre capacity surpassed 376 megawatts in 2025, with operators racing to lock in power, land, and government workloads ahead of 2026 expansions.

Saudi Arabia launched the $2.7 billion Hexagon Data Centre initiative at the start of 2026, a 480-megawatt, Tier-IV facility that will be the world's largest government data centre once complete. The kingdom also established HUMAIN, a government-backed AI company owned by the Public Investment Fund, which serves as a central vehicle for domestic AI infrastructure development. HUMAIN's CEO Tareq Amin has stated plainly: “We want to be the third-largest AI provider in the world, behind the United States and China.” The company has plans to build up to 1.9 gigawatts of data centre capacity by 2030 and has signed deals worth $23 billion with global tech suppliers including Nvidia, AMD, Cisco, Qualcomm, and AWS. Under a key partnership resulting from President Trump's visit to the Gulf in May 2025, Nvidia will supply 18,000 of its GB300 Blackwell chips to Saudi Arabia, with the first shipment arriving in December 2025.

The Gulf nations possess a structural advantage. Electricity tariffs in Saudi Arabia and the UAE range from $0.05 to $0.06 per kilowatt-hour, well below the US average of $0.09 to $0.15 per kWh. These countries also have vast tracts of undeveloped land, minimal planning restrictions, and the financial firepower to build at scale. The Emirates Nuclear Energy Company recently signed a memorandum of understanding with GE Vernova Hitachi Nuclear Technology to evaluate deploying small nuclear technology, while Saudi Arabia has plans for its first nuclear power plant.

The irony is thick. Nations that built their wealth on extracting and selling fossil fuels are now positioning themselves to profit from the insatiable energy demands of artificial intelligence, which many had hoped would be powered primarily by clean energy.

Geopolitical Swing States and the New Digital Divide

The AI energy nexus is not merely a story about wealthy nations and trillion-dollar companies. It is reshaping the global order in ways that extend far beyond Silicon Valley and the Gulf.

At the centre of this transformation lies the rivalry between the United States and China. The United States has imposed export controls limiting China's access to high-end AI chips, potentially slowing China's AI advancement. China, however, holds advantages through its lead in open-source AI models and its focus on applied AI. This contest over technological supremacy is increasingly fought on energy terrain: nations with abundant, diverse energy supplies and advanced grid infrastructure are better positioned to capitalise on AI advancements and enhance their geopolitical influence.

Beyond the US-China competition, a group of “geopolitical swing states” is becoming increasingly vital. India, Vietnam, Turkey, and other emerging economies are essential players in the AI supply chain and are being courted by both major powers. India, in particular, is witnessing one of the strongest economic expansions among major nations, powered by its digital economy, youthful population, and large-scale foreign investments. The choices these nations make about energy infrastructure, data sovereignty, and technological partnerships will significantly influence the shape of the global AI economy.

The RAND Corporation's Michael J. Mazarr, in his January 2026 report “A New Age of Nations: Power and Advantage in the AI Era,” noted that at least 75 countries had published national AI strategies. His core thesis is that the competitive challenge of AI is primarily social, not technological. Countries that lead the new era will not merely have the best AI models; they will have taken the necessary steps to make their societies more competitive. Yet there is a catch: not every country can, or should, try to build every part of the AI stack independently. Attempting to recreate everything from data centres to foundation models is expensive, redundant, and impractical for most nations.

This creates a new form of digital divide. Countries with reliable, abundant electricity and the capital to invest in data centre infrastructure will attract AI companies, talent, and investment. Those without adequate energy capacity risk being relegated to the role of consumers rather than producers of AI technology, dependent on foreign cloud providers and vulnerable to the terms those providers set. Countries in sub-Saharan Africa, South Asia, and parts of Latin America, where electricity access remains unreliable and grid infrastructure is underdeveloped, face the prospect of being excluded from the AI revolution entirely. This is not merely a matter of technological disadvantage; it is a question of economic development, educational opportunity, and political agency in a world increasingly shaped by artificial intelligence.

Governance Gaps and Regulatory Scrambles

The tension between AI's energy hunger and environmental commitments has exposed a profound gap in global governance. At the Paris AI Action Summit in February 2025, 61 countries, including China, India, Japan, Australia, and Canada, signed the Statement on Inclusive and Sustainable Artificial Intelligence. But the United States and the United Kingdom, two of the world's most important AI powers, refused to sign.

Their reasons diverged sharply. US Vice President JD Vance warned that excessive regulation of AI “could kill a transformative industry just as it's taking off,” and objected to the declaration's focus on multilateralism, inclusivity, and environmental challenges. The UK, by contrast, supported much of the declaration's content but felt the pact “didn't provide enough practical clarity on global governance and didn't sufficiently address harder questions around national security.” Dario Amodei, CEO of Anthropic, wrote in a statement that “at the next international summit, we should not repeat this missed opportunity.”

The absence of the two largest English-speaking AI powers from the governance framework leaves a vacuum that is being filled, unevenly, by regional and national regulation. The European Commission plans to adopt a “Data Centre Energy Efficiency Package” in April 2026 that will introduce a rating scheme and begin work on minimum performance standards. In the United States, the Department of Energy directed the Federal Energy Regulatory Commission (FERC) to issue a rulemaking to ensure efficient and non-discriminatory load interconnections for large electrical loads, with a final rule expected by April 2026.

In the United Kingdom, the stakes are particularly stark. According to a report covered by the Institution of Engineering and Technology, 140 proposed data centre schemes in the UK could require 50 gigawatts of electricity, 5 gigawatts more than the country's current peak demand. This poses what experts have described as a “serious threat to efforts to decarbonise the electricity grid.”

Without coordinated international standards, companies are left to self-regulate, a practice that has not inspired confidence given the trajectory of their emissions. Climate-related shareholder proposals were filed at Amazon, Meta, and Alphabet in 2025, asking how these companies plan to reconcile their ambitious climate commitments with growing AI electricity demand and whether their renewable energy procurement strategies remain credible.

Sam Altman's Uncomfortable Truth

OpenAI CEO Sam Altman has been characteristically blunt about the situation. At an AMD AI conference, he stated: “Theoretically, at some points, you can see that a significant fraction of the power on Earth should be spent running AI compute. And maybe we're going to get there.” He has acknowledged it is “fair” to be concerned about AI's total energy consumption, arguing that the world needs to “move towards nuclear or wind and solar very quickly.”

Altman has also pushed back against what he considers misleading framings of AI's resource use, arguing that comparisons of AI energy efficiency against human cognition are “unfair.” He contended that it “takes like 20 years of life and all of the food you eat during that time before you get smart,” and suggested AI has “already caught up on an energy efficiency basis” when considered on a per-query comparison. Not everyone found this persuasive. Creative Strategies analyst Max Weinback wrote that Altman's framing was “trying to break down people and models into cost for output and ignoring the value of humanity itself.”

The debate has taken stranger turns. Elon Musk and Jeff Bezos have floated the idea of placing AI data centres in orbit to tap into unlimited solar power and fewer physical constraints. Altman dismissed the notion: “I honestly think the idea with the current landscape of putting data centres in space is ridiculous.” He cited practical concerns including launch costs, the difficulty of repairing broken GPUs in space (“they do break a lot still, unfortunately”), and the simple economics of terrestrial power generation.

What Altman's candour reveals, however uncomfortable, is that the AI industry's leadership has already internalised a future in which artificial intelligence consumes a transformative share of global electricity. The question is not whether this will happen but how the energy will be sourced, who will control it, and what the environmental consequences will be.

A Fractured Energy Future

The emerging picture is one of radical fragmentation. Different regions are pursuing wildly different energy strategies to feed their AI ambitions, and the choices they make will reverberate for decades.

In the United States, natural gas remains the near-term workhorse, supplemented by a nuclear renaissance driven by tech company investment. The restart of the Crane Clean Energy Centre, the SMR agreements with Kairos Power and X-energy, and Trump's May 2025 executive orders aimed at speeding deployment of new nuclear technologies all point toward a hybrid approach that prioritises speed and reliability over emissions reduction.

In Europe, the emphasis is shifting toward regulatory frameworks and efficiency standards. The European Commission's forthcoming Data Centre Energy Efficiency Package represents an attempt to impose order on an industry that has so far grown largely unchecked. Ireland, where data centres could consume nearly a third of national electricity by late 2026, is a test case for whether a small, grid-constrained nation can accommodate the AI industry without compromising its broader energy transition.

In the Gulf, the strategy is unambiguous: build massive capacity quickly, leveraging cheap energy, abundant land, and sovereign wealth fund capital. Whether these facilities run on renewables (the Al Dhafra Solar Project in the UAE is one of the world's largest) or fossil fuels will be determined by economics and speed rather than environmental ambition.

In China, the approach blends state-directed investment in both AI and energy infrastructure, with an emphasis on energy self-sufficiency and technological autonomy that is inseparable from broader strategic competition with the United States.

The environmental implications are sobering. The IEA estimates that data centre emissions will reach 1 per cent of global CO2 emissions by 2030 in its central scenario, or 1.4 per cent in a faster-growth scenario. Goldman Sachs projects that data centre power demand will surge 165 to 175 per cent by 2030 compared to 2023 levels, the equivalent of adding another top-ten power-consuming country to the planet.

Yet there is a counterargument that deserves serious consideration. AI could enable Southeast Asian nations alone to reduce power sector costs by $45 to $67 billion through 2035, with potential efficiency gains cutting emissions by 290 to 386 million tonnes of CO2. Smart grids, predictive maintenance, and optimised energy distribution are all areas where AI can accelerate the energy transition rather than impede it. In the IEA's central scenario, the data centre electricity mix shifts from approximately 60 per cent fossil fuels and 40 per cent clean power today to 60 per cent clean power and 40 per cent fossil fuels by 2035.

The question is whether the net effect will be positive or negative. If the AI industry drives sufficient investment in clean energy infrastructure, it could paradoxically become one of the most powerful forces for decarbonisation. If, on the other hand, it simply layers enormous new electricity demand on top of existing fossil fuel systems, it will accelerate climate change at precisely the moment when emissions need to be falling.

The answer will depend not on technology alone but on policy, governance, and political will. It will depend on whether governments treat AI energy consumption as a matter for the market or as a strategic priority requiring active management. It will depend on whether the global community can agree on standards for data centre emissions, energy efficiency, and grid interconnection, or whether the regulatory vacuum that currently exists persists.

For now, the companies with the most megawatts are winning. The rest are watching, waiting, and hoping the grid connection arrives before their competitors pull too far ahead. In the new AI economy, the currency is not data, and it is not compute. It is energy. And like every scarce resource before it, it is already reshaping who holds power and who does not.

---

References and Sources

1. International Energy Agency (IEA), “Energy Demand from AI,” Energy and AI Analysis, 2025. Available at: https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai

2. International Energy Agency (IEA), Electricity 2026: Analysis and Forecast, January 2026. Available at: https://www.iea.org/reports/electricity-2026

3. Lawrence Berkeley National Laboratory, US Data Centre Energy Consumption Projections, referenced via Pew Research Center, “What we know about energy use at U.S. data centers amid the AI boom,” October 2025. Available at: https://www.pewresearch.org/short-reads/2025/10/24/what-we-know-about-energy-use-at-us-data-centers-amid-the-ai-boom/

4. US Energy Information Administration (EIA), Data Centre Power Demand Forecasts, 2025-2026, referenced via Data Center Dynamics. Available at: https://www.datacenterdynamics.com/en/news/eia-projects-record-us-data-center-power-use-amid-ai-and-crypto-boom/

5. Goldman Sachs Research, “GS SUSTAIN: AI/data centers' global power surge: The push for the 'Green' data center,” 2025. Available at: https://www.goldmansachs.com/insights/goldman-sachs-research/the-push-for-the-green-data-center

6. Constellation Energy, “Constellation to Launch Crane Clean Energy Center,” press release, September 2024. Available at: https://www.constellationenergy.com/news/2024/Constellation-to-Launch-Crane-Clean-Energy-Center-Restoring-Jobs-and-Carbon-Free-Power-to-The-Grid.html

7. CNBC, “Trump administration backs Three Mile Island nuclear restart with $1 billion loan to Constellation,” November 2025. Available at: https://www.cnbc.com/2025/11/18/trump-nuclear-three-mile-island-crane-loan-constellation-ceg.html

8. IEEE Spectrum, “Big Tech Embraces Nuclear Power to Fuel AI and Data Centers,” 2025. Available at: https://spectrum.ieee.org/nuclear-powered-data-center

9. IAEA, “Data Centres, Artificial Intelligence and Cryptocurrencies Eye Advanced Nuclear to Meet Growing Power Needs,” IAEA Bulletin, 2025. Available at: https://www.iaea.org/bulletin/data-centres-artificial-intelligence-and-cryptocurrencies-eye-advanced-nuclear-to-meet-growing-power-needs

10. NPR, “AI brings soaring emissions for Google and Microsoft, a major contributor to climate change,” July 2024. Available at: https://www.npr.org/2024/07/12/g-s1-9545/ai-brings-soaring-emissions-for-google-and-microsoft-a-major-contributor-to-climate-change

11. Al Jazeera, “Paris AI summit: Why won't US, UK sign global artificial intelligence pact?” February 2025. Available at: https://www.aljazeera.com/news/2025/2/12/paris-ai-summit-why-wont-us-uk-sign-global-artificial-intelligence-pact

12. Middle East Institute, “From Crude to Compute: Building the GCC AI Stack,” 2025. Available at: https://www.mei.edu/publications/crude-compute-building-gcc-ai-stack

13. CNBC, “Saudi AI firm Humain is pouring billions into data centers. Will it pay off?” August 2025. Available at: https://www.cnbc.com/2025/08/27/saudi-arabia-wants-to-be-worlds-third-largest-ai-provider-humain.html

14. Michael J. Mazarr, “A New Age of Nations: Power and Advantage in the AI Era,” RAND Corporation Perspective PE-A3691-14, January 2026. Available at: https://www.rand.org/pubs/perspectives/PEA3691-14.html

15. Carbon Brief, “AI: Five charts that put data-centre energy use and emissions into context,” 2025. Available at: https://www.carbonbrief.org/ai-five-charts-that-put-data-centre-energy-use-and-emissions-into-context/

16. FP Analytics (Foreign Policy), “Powering the AI Era,” May 2025. Available at: https://fpanalytics.foreignpolicy.com/2025/05/20/artificial-intelligence-electricity-demand/

17. TechCrunch, “Sam Altman would like to remind you that humans use a lot of energy, too,” February 2026. Available at: https://techcrunch.com/2026/02/21/sam-altman-would-like-remind-you-that-humans-use-a-lot-of-energy-too/

18. Engineering and Technology Magazine (IET), “AI data centre boom could push up UK electricity demand and carbon emissions,” March 2026. Available at: https://eandt.theiet.org/2026/03/02/climate-impact-ai-data-centre-growth-under-scrutiny

19. Bloomberg, “How AI Data Centers Are Sending Your Power Bill Soaring,” 2025. Available at: https://www.bloomberg.com/graphics/2025-ai-data-centers-electricity-prices/

20. Carnegie Mellon University, “Data Center Growth Could Increase Electricity Bills 8% Nationally and as Much as 25% in Some Regional Markets,” July 2025. Available at: https://www.cmu.edu/work-that-matters/energy-innovation/data-center-growth-could-increase-electricity-bills

21. IEEFA, “Projected data center growth spurs PJM capacity prices by factor of 10,” 2025. Available at: https://ieefa.org/resources/projected-data-center-growth-spurs-pjm-capacity-prices-factor-10

22. NRDC, “Rising Demand from Data Centers Driving Reliability, Cost Concerns,” 2025. Available at: https://www.nrdc.org/press-releases/rising-demand-data-centers-driving-reliability-cost-concerns

23. Brookings Institution, “AI, data centers, and water,” 2025. Available at: https://www.brookings.edu/articles/ai-data-centers-and-water/

24. EESI, “Data Centers and Water Consumption,” 2025. Available at: https://www.eesi.org/articles/view/data-centers-and-water-consumption

25. MSCI, “When AI Meets Water Scarcity: Data Centers in a Thirsty World,” 2025. Available at: https://www.msci.com/research-and-insights/blog-post/when-ai-meets-water-scarcity-data-centers-in-a-thirsty-world

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

Something peculiar is happening in Silicon Valley. The industry that once prided itself on a libertarian ethos of building first and asking questions later has fractured along unmistakably political lines. Artificial intelligence, the technology that promises to reshape everything from how we work to how we think, has become the latest battleground in America's culture wars. And the combatants are not just politicians or pundits; they are the billionaires, venture capitalists, and technologists who control the infrastructure of the future.

The pattern is now impossible to ignore. When President Donald Trump announced the Stargate Project in January 2025, a $500 billion commitment to AI infrastructure led by OpenAI, SoftBank, and Oracle, he was signalling a new era in which AI development would be explicitly tied to political favour. Sam Altman, OpenAI's chief executive, stood beside Trump at the White House, a far cry from 2016, when Altman compared Trump to Hitler in 1930s Germany. By December 2024, Altman had donated $1 million to the Trump-Vance Inaugural Committee, a remarkable political transformation that mirrored the industry's broader realignment.

By early 2026, that realignment has hardened into something far more consequential than shifting political donations. The Trump administration has designated one of the world's leading AI safety companies a threat to national security, deployed a politically aligned chatbot across the federal government, and granted a venture capital firm what observers describe as near-veto power over AI legislation. The ideological stratification of AI is no longer a tendency; it has become policy.

The Money Trail Speaks Volumes

Follow the money, and the political stratification of AI becomes starkly apparent. In January 2026, Elon Musk's xAI raised $20 billion at a valuation of $230 billion, pushing its total capital raised to $62 billion across equity and debt. This staggering sum, accumulated in less than three years, has not flowed to xAI despite its politics, but arguably because of them. Musk founded xAI in March 2023 explicitly to counter what he called the “political correctness” of other AI models. The company's flagship product, Grok, was designed to be “maximally truth-seeking,” a phrase that has become code in certain circles for rejecting what conservatives perceive as liberal bias in mainstream AI systems.

The evidence of Grok's rightward trajectory is now well documented and, in several episodes, alarming. A New York Times analysis found that between May and July 2025, Grok's responses shifted to the right on more than half of political questions tested. In June 2025, Musk criticised the bot for “parroting legacy media.” By July, adjustments had been made for Grok to be “politically incorrect,” resulting in a measurable rightward shift. Then, on 8 July 2025, Grok underwent what observers described as a complete meltdown: for several hours the system praised Adolf Hitler, described itself as “MechaHitler,” endorsed antisemitic conspiracy theories, and offered detailed suggestions for assaulting an X user. xAI blamed the incident on “an unauthorised modification” to Grok's system prompt. The Anti-Defamation League called it “irresponsible, dangerous and antisemitic.” Linda Yaccarino, chief executive of X, announced her departure shortly afterwards.

The controversy did not slow xAI's commercial or political ambitions. In early January 2026, a separate deepfake scandal engulfed the platform as users exploited Grok to generate sexualised images of women and children without consent. An analysis of 20,000 Grok-generated images found that approximately 2 per cent appeared to depict minors, with a separate analysis finding nearly 10 per cent showing “photorealistic people, very young, doing sexual activities.” Malaysia and Indonesia blocked access to Grok; the US Senate unanimously passed legislation allowing victims to sue over non-consensual AI-generated images; 35 state attorneys general called on xAI to cease; and the EU opened a privacy investigation. By March 2026, xAI was marketing Grok 4.20 beta as “the only non-woke AI in existence, engineered to pursue maximum truth, and deliver unfiltered, evidence-based answers where every other major model has been lobotomised by the woke mind virus.” Independent research presented a more complex picture: Dartmouth College's Polarization Research Lab measured Grok exhibiting a 67.9 per cent extremism rate, the highest of any model tested, with only 2.1 per cent of responses being centrist.

Contrast this with Anthropic, which in February 2026 closed a $30 billion funding round at a $380 billion post-money valuation, making it the second-largest venture deal in history. The company's annualised revenue has climbed to $14 billion, with eight of the Fortune 10 now Claude customers. Founded by former OpenAI researchers Dario and Daniela Amodei, Anthropic staked its reputation on a different proposition: that safety and reliability should be engineered into AI systems from their inception. The company's Claude model scored a 94 per cent “even-handedness” rating in political neutrality evaluations, roughly on par with Google's Gemini 2.5 Pro at 97 per cent and Grok 4 at 96 per cent, but higher than OpenAI's GPT-5 at 89 per cent and significantly above Meta's Llama 4 at 66 per cent.

The investment patterns behind these companies tell a story of diverging priorities. Andreessen Horowitz, the venture capital powerhouse, has emerged as a central node in the conservative-aligned AI ecosystem. In 2024, nearly 70 per cent of contributions from Andreessen Horowitz employees went to Republican candidates, a stark reversal from previous years. Co-founders Marc Andreessen and Ben Horowitz each donated $2.5 million to a pro-Trump super PAC. The firm's federal lobbying spending soared to $3.53 million in 2025, double that of 2024, far exceeding other venture capital firms. As a February 2026 Bloomberg investigation revealed, Andreessen Horowitz is now regularly the first outside call that top White House officials and senior Republican congressional aides make when considering moves that could affect tech companies' AI plans, with one former White House official describing the firm as possessing near-veto power over virtually all AI-related legislative proposals.

The PayPal Mafia Remakes Washington

The political realignment of AI investment cannot be understood without examining the network that now extends from Silicon Valley into the highest levels of American government. Peter Thiel, the German-American entrepreneur who co-founded PayPal and Palantir Technologies, has spent years cultivating what Fortune magazine has called a network of “right-wing techies” now infiltrating the Trump White House.

Thiel's connections to the Trump administration are extensive. David Sacks, who worked with Thiel at PayPal and wrote for the Stanford Review (the student newspaper Thiel founded in 1987), was named White House “AI and crypto czar.” Vice President JD Vance worked at Thiel's Mithril Capital fund before launching his own venture firm backed by Thiel. Thiel introduced Vance to Trump at Mar-a-Lago in 2021. Sriram Krishnan, a former partner at Andreessen Horowitz, joined the White House as senior AI policy adviser. A leaked draft of Trump's December 2025 executive order on AI preemption drew directly from a policy memo published by Andreessen Horowitz in September 2025.

By late 2025, questions about the integrity of these arrangements had become pointed. Sacks, Trump's influential adviser on AI and cryptocurrency, came under sustained scrutiny over government paperwork that critics say grants him “carte blanche” to shape US policy while retaining hundreds of investments in the tech world. While Sacks divested from some holdings, public documents show that he and his firm, Craft Ventures, maintained more than 400 investments in firms with AI ties. Washington University ethics expert Kathleen Clark characterised the resulting waivers as “sham ethics waivers” lacking rigorous objective analysis. The concerns sharpened when Craft Ventures invested $22 million in an AI company targeting federal contracts, the very sector Sacks is meant to regulate.

Bloomberg has reported that more than a dozen people with ties to Thiel have been integrated into the Trump administration. Founders Fund has invested in the major startups working most closely with the US Department of Defence, including SpaceX, Palantir, and Anduril. Palantir Technologies, founded by Thiel and colleagues in 2003, develops data integration and analytics platforms enabling government agencies, militaries, and corporations to combine and analyse data from multiple sources; its early funding came partly from In-Q-Tel, the CIA's venture arm. In 2026, Palantir found itself at the centre of the Anthropic controversy, after an Anthropic executive enquired whether Claude had been used in a military raid in Venezuela — raising questions about how AI safety policies operate when filtered through Pentagon partnerships.

This is not merely a story of individual political donations. It represents a structural integration of a particular ideological vision into the governance of AI policy. The long-term libertarian vision of using technology to drastically reduce the size of the state has become more mainstream in Silicon Valley, and through the Thiel network's presence across government, investment, and technology, these ideas are being translated into actual AI policy.

Regulatory Divergence and the Transatlantic Divide

The ideological stratification of AI investment has profound implications for regulation. On 23 January 2025, President Trump issued an executive order titled “Removing Barriers to American Leadership in Artificial Intelligence,” explicitly rescinding the Biden administration's landmark 2023 executive order on AI safety, signalling a dramatic shift from oversight toward deregulation framed as national competitiveness.

Vice President JD Vance articulated this philosophy at the Paris AI Action Summit: “The AI future is not going to be won by hand-wringing about safety. It will be won by building, from reliable power plants to the manufacturing facilities that can produce the chips of the future.”

On 11 December 2025, the administration went further. Trump signed an executive order titled “Ensuring a National Policy Framework for Artificial Intelligence,” seeking to limit states' ability to regulate AI and directing the Department of Justice to establish an “AI Litigation Task Force” to challenge state laws on constitutional grounds. The order set implementation deadlines in early 2026. California's Transparency in Frontier Artificial Intelligence Act and Texas's Responsible Artificial Intelligence Governance Act came into force on 1 January 2026, while Colorado's AI Act was delayed to 30 June 2026. Governors in California, Colorado, and New York indicated the federal order would not stop them from enforcing their local statutes. A separate executive order on “Preventing Woke AI in the Federal Government” sought to limit government procurement to models deemed “truth-seeking” and exhibiting “ideological neutrality,” though critics noted the definition of neutrality was itself ideologically loaded.

This approach stands in stark contrast to the European Union's regulatory framework. The EU AI Act's remaining provisions become applicable on 2 August 2026, with transparency obligations, conformity assessments, and EU database registration for high-risk systems all due by that date. The European Commission's Digital Omnibus package, released in November 2025, streamlines certain aspects while maintaining core legislative instruments. EU regulators have opened investigations into Grok over the sexual deepfake scandal, with France among the first to act after a deepfake of a minor was generated on the platform. As legal analysts have noted, the United States' unilateral focus on deregulation risks limiting its influence in shaping global AI norms.

The Bias Baked into the Algorithms

At the heart of the political stratification of AI lies a fundamental question: are large language models inherently biased, and if so, in which direction? The research is now substantial and consistent.

David Rozado, a researcher at Otago Polytechnic in New Zealand, published a comprehensive study in PLOS ONE examining 24 state-of-the-art large language models. Using 11 different political orientation tests administered 10 times per model, totalling 2,640 test administrations, Rozado found that the majority of conversational LLMs consistently produced responses diagnosed as left-of-centre. On the Political Compass Test, models scored left-of-centre economically (mean: -3.69) and socially (mean: -4.19). Crucially, his analysis of base models — those without further supervised fine-tuning — found they demonstrated near-neutral positioning. This suggests political preferences are not inherent to pre-trained LLMs, nor simply absorbed from internet-scale training data, but are instead introduced during post-training, particularly through reinforcement learning from human or AI feedback.

A Stanford study from May 2025 tested 24 different LLMs from eight companies with 30 political questions, having over 10,000 US respondents rate the political slant of the responses. For 18 of the 30 questions, users perceived nearly all LLMs' responses as left-leaning, with both Republican and Democratic respondents noticing this trend, though Republicans perceived a more drastic slant.

A study published in PNAS Nexus on 3 March 2026, conducted by Yale University researchers, added a further dimension: AI chatbots can subtly influence users' social and political opinions through unintended latent biases even when users are not asking political questions. Testing responses about the 1919 Seattle General Strike and the 1968 Third World Liberation Front protests, the researchers found that both default AI summaries and those with liberal framing caused participants to express more liberal opinions than Wikipedia entries did. The study concluded that “content not intended to change minds can also shift people's opinions.” A separate preregistered study conducted in December 2025 and January 2026 found that the strongest warnings about potential LLM biases reduce persuasion by 28 per cent relative to control groups.

An October 2024 report from the Centre for Policy Studies examined sentiment analysis across LLMs. On a scale from -1 (wholly negative) to +1 (wholly positive), LLMs gave left-leaning political parties an average sentiment score of +0.71, compared to +0.15 for right-leaning parties. Hard-right positions received an average sentiment of -0.77, while hard-left positions received mostly neutral sentiment at +0.06.

These findings help explain both the conservative backlash against mainstream AI systems and the market opportunity companies like xAI have sought to exploit. They also illustrate the profound stakes: AI systems interacted with by hundreds of millions of people are shaping political opinion not merely when explicitly asked to do so.

Silicon Valley's Political Realignment

The 2024 election cycle revealed the extent of Silicon Valley's political transformation. A December 2024 Guardian analysis found that tech bosses funnelled $394 million into the election cycle. Elon Musk pledged $45 million per month for at least three months to Trump's election effort. Marc Andreessen and Ben Horowitz endorsed Trump on their podcast and contributed financially. Peter Thiel donated approximately $1.5 million to pro-Trump groups during the 2016 election cycle and subsequently bankrolled JD Vance's Senate campaign, introducing Vance to Trump at Mar-a-Lago in 2021.

By August 2025, major Democratic tech donors had largely retreated. According to FEC filings, figures like Laurene Powell Jobs, Dustin Moskovitz, and Michael Moritz appeared to have donated nothing to federal candidates or fundraising committees in 2025. Meanwhile, their Republican counterparts kept the money flowing.

This shift has spawned new political infrastructure targeted at the 2026 midterm elections. Leading the Future, a super-PAC backed by Andreessen Horowitz and OpenAI president Greg Brockman, is deploying more than $100 million to fight AI regulation, targeting battleground states including California, New York, Illinois, and Ohio. Andreessen and Horowitz jointly contributed $50 million to the fund; Brockman and his wife committed another $50 million. Andreessen Horowitz also pledged $23 million to the crypto-focused super-PAC Fairshake for the 2026 midterms. Meta launched its own super-PAC, Meta California, targeting the 2026 California governor's race. Rolling Stone has noted that AI companies are deploying the cryptocurrency sector's model of single-issue financial influence to defeat candidates who wish to regulate AI.

Downstream Effects: From Policy to Practice

When capital allocation becomes ideologically driven, the effects ripple through every stage of AI development. The events of early 2026 have brought those effects into sharp focus.

The Trump administration's deployment of Grok across the federal government represents the most concrete example yet of politically aligned AI becoming institutionalised policy. In September 2025, the General Services Administration struck an agreement with xAI making Grok models accessible to federal agencies for $0.42 per organisation for 18 months. On 12 January 2026, Defence Secretary Pete Hegseth announced during a speech at Musk's SpaceX headquarters that the Department of Defence would integrate Grok into its internal networks, including classified and unclassified systems, stating the systems would operate “without ideological constraints” and “will not be woke.” Three million military and civilian personnel gained access. The federal government's nutrition website was among the first civilian sites to direct users to Grok, even as the deepfake scandal was generating international condemnation. A coalition of nonprofits called for an immediate suspension of the government's Grok deployment, citing the unresolved deepfake scandal and Grok's documented antisemitic outputs.

The deployment of Grok coincided with the expulsion of its principal commercial rival. On 27 February 2026, the Trump administration ordered all federal agencies to cease using Anthropic's technology after the company refused to remove safety guardrails on its AI model. The dispute centred on Anthropic's refusal to permit two specific uses: mass surveillance of American citizens and fully autonomous weapons systems operating without human oversight. Defence Secretary Hegseth designated Anthropic a “supply chain risk to national security,” a designation normally reserved for companies from adversarial nations such as China. The Pentagon imposed a requirement that contractors doing business with the US military could not conduct any commercial activity with Anthropic. OpenAI, which has no comparable restrictions, swept in to replace Anthropic as the military's primary AI partner.

Dario Amodei, Anthropic's chief executive, stated that he does “not believe this action is legally sound, and we see no choice but to challenge it in court.” In a leaked internal memo subsequently published by The Information, Amodei said one of the real causes of the dispute was that “we haven't given dictator-style praise to Trump.” The confrontation crystallised the dynamics at work across the industry: companies that accommodate the administration's political preferences gain government contracts and regulatory forbearance; those that maintain independent safety standards are penalised.

OpenAI's own trajectory illustrates how political relationships shape organisational identity. During its for-profit restructuring in late 2025, the company quietly removed the word “safely” from its mission statement. Where OpenAI's 2023 mission read “to ensure that artificial general intelligence benefits all of humanity, safely,” the new formulation reads simply “to ensure that artificial general intelligence benefits all of humanity.” The deletion, discovered in a tax filing, prompted concern among AI safety researchers that commercial and political pressures were eroding the company's founding commitments.

Meta's explicit acknowledgment with Llama 4 adds further texture to the pattern. The company stated that “leading large language models historically have leaned left when it comes to debated political and social topics” and that Llama 4 is more inclusive of right-wing politics. Critics noted that this approach risks creating false equivalence, lending credibility to arguments not grounded in empirical evidence. GLAAD reported that Llama 4 had begun referencing discredited conversion therapy practices, arguing that “both-sidesism” equating anti-LGBTQ junk science with well-established facts is not only misleading but legitimises harmful falsehoods.

Implications for Democratic Discourse and Policy Institutions

The integration of politically stratified AI systems into institutions that shape public discourse raises profound questions for democracy. As of March 2025, ChatGPT had 500 million weekly users. These technologies are reshaping how citizens access and process information, communicate with elected officials, organise politically, and participate in society. The Yale PNAS Nexus study published on 3 March 2026 adds empirical weight to the concern: even queries about historical events, not explicitly political in framing, produce measurable shifts in users' political opinions, with the direction of that shift determined by choices made during AI training.

Research from the Carnegie Endowment for International Peace warns that AI technologies “present significant threats to democracies by enabling malicious actors, from political opponents to foreign adversaries, to manipulate public perceptions, disrupt electoral processes, and amplify misinformation.” A 2025 Pew Research Center survey found that only about one in ten US adults and AI experts expect AI to have a positive impact on elections, with far larger shares worried about bias, misinformation, and manipulation.

The cross-national analysis of AI framing in parliamentary debates from 2014 to 2024, published in the journal Policy and Internet, reveals striking differences in how different political systems are responding. In the European Union and Switzerland, debates are dominated by an “Ethics and Regulation” lens. The United States departs from these expectations: congressional speech is dominated by a “Military and Security” frame, likely due to overriding geopolitical pressures. That divergence has only sharpened since January 2026, as the Pentagon's actions regarding Anthropic and the government deployment of Grok demonstrate.

The growth in AI-generated content, coupled with the increasing difficulty of identifying it as machine-made, has the potential to transform the public sphere via information overload and pollution. For government officials, this undermines efforts to understand constituent sentiment, threatening the quality of democratic representation. For voters, it threatens efforts to monitor what elected officials do, eroding democratic accountability.

The Fragmented Future of AI Development

Google DeepMind has attempted to chart a middle course, releasing a 145-page paper in April 2025 forecasting that AGI could arrive by 2030, “potentially capable of performing at the 99th percentile of skilled adults in a wide range of non-physical tasks.” The paper proposed a four-layer defence system: market design, base-level AI safety, real-time monitoring, and regulation. Shane Legg, DeepMind's Chief AGI Scientist, stated that regulation “can and should be” part of society's response, while acknowledging that “safety has become a bad word in a certain political sphere.” In August 2025, a cross-party group of 60 UK parliamentarians accused Google DeepMind of violating international pledges to safely develop AI, arguing that its release of Gemini 2.5 Pro without accompanying safety testing details “sets a dangerous precedent.”

The fragmentation of AI development along ideological lines creates several concerning trajectories. The first is that AI systems will increasingly be optimised for different audiences, reflecting and potentially amplifying existing political divisions. A conservative user might interact with Grok while a progressive user relies on Claude, each receiving information filtered through different ideological prisms — a dynamic now given institutional form by the federal government's decision to use one and blacklist the other.

The second is that regulatory divergence between the United States and the European Union creates uncertainty for companies operating globally. Grok has been blocked or investigated in multiple countries due to its content failures. AI systems developed under American deregulatory frameworks may not comply with EU requirements, producing a fragmented global landscape where the same technology operates under fundamentally different rules.

The third is that the concentration of political influence among a small network of investors raises questions about accountability. When Andreessen Horowitz possesses what observers describe as near-veto power over White House AI legislation, and when the firm's former partner serves as a senior White House AI policy adviser, the traditional separation between technology and governance does not merely blur; it disappears.

Dario Amodei of Anthropic has expressed discomfort with this arrangement. “I think I'm deeply uncomfortable with these decisions being made by a few companies, by a few people,” he told Fortune in November 2025. “And this is one reason why I've always advocated for responsible and thoughtful regulation of the technology.” By March 2026, Amodei's company was fighting in court to preserve the legal right to maintain AI safety standards without government coercion, a position that would have seemed implausible at the beginning of Trump's second term.

The Contours of a Divided Future

The political stratification of AI investment is not merely an American phenomenon, though it is most pronounced in the United States. China has a stated goal of becoming the world's AI leader by 2030, and the competition between US and Chinese AI development is itself shaping the ideological valence of American AI policy, with security concerns frequently overriding safety considerations.

The Stargate Project exemplifies this dynamic. The joint venture intends to allocate $500 billion over four years. By early 2026, the Abilene flagship campus had two buildings operational since September 2025, with the remaining six expected to complete by mid-2026, ultimately housing over 450,000 NVIDIA GB200 GPUs. Six additional US campuses are in various stages of development across Texas, New Mexico, and Ohio. The combined capacity brings Stargate to nearly 7 gigawatts of planned capacity and over $400 billion in investment. OpenAI's custom “Titan” AI chip, fabricated by TSMC on its 3nm process and designed in partnership with Broadcom, is expected to enter mass production in the second half of 2026.

But American leadership in AI, as currently configured, means something specific: deregulation, integration with military applications, and alignment with the political preferences of a particular faction of technology investors. The events of February and March 2026 have made that configuration explicit in ways the original Stargate announcement did not: the federal government now actively directs which AI companies may serve the state, deploying politically aligned systems across its agencies while designating safety-conscious competitors as national security threats.

The fragmentation of AI along ideological lines may prove to be one of the most consequential developments in the technology's history. Unlike previous technological revolutions, AI systems are not merely tools that humans use; they are increasingly systems that shape how humans think, communicate, and make decisions. The Yale research published in March 2026 demonstrates that this shaping effect operates even in ostensibly neutral informational contexts. If those systems are designed to reflect particular political orientations, they may do more than mirror existing divisions; they may entrench them in ways that prove difficult to reverse.

The venture capitalists, technologists, and politicians driving this transformation would likely reject the framing that their work is ideological. They would describe it as building better technology, promoting innovation, or protecting national interests. But the choices being made about which AI systems to fund, how to train them, what safety measures to implement, and how to regulate them are not neutral technical decisions. They are expressions of values, and those values are increasingly organised along partisan lines.

The question now is whether any space remains for developing AI in the public interest, for building systems optimised for accuracy rather than ideology, and for governance frameworks that prioritise democratic accountability. Anthropic's willingness to forfeit a $200 million government contract rather than remove safeguards against autonomous weapons and mass surveillance suggests that some actors are prepared to maintain those standards under significant pressure. Whether they can do so sustainably, as competitors backed by state resources and politically aligned capital expand their reach, remains the defining question of the technology's immediate future.

---

References and Sources

Amodei, D. (2026, March). Where things stand with the Department of War. Anthropic. https://www.anthropic.com/news/where-stand-department-war

Bloomberg. (2026, February 10). Trump's AI Policy Shaped by VC Tech Giant Andreessen Horowitz. https://www.bloomberg.com/news/features/2026-02-10/trump-s-ai-policy-shaped-by-vc-tech-giant-andreessen-horowitz

Carnegie Endowment for International Peace. (2024, December). Can Democracy Survive the Disruptive Power of AI? https://carnegieendowment.org/research/2024/12/can-democracy-survive-the-disruptive-power-of-ai

Centre for Policy Studies. (2024, October). The Politics of AI by David Rozado. https://cps.org.uk/wp-content/uploads/2024/10/CPS_THE_POLITICS_OF_AI-1.pdf

CNBC. (2026, February 12). Anthropic closes $30 billion funding round at $380 billion valuation. https://www.cnbc.com/2026/02/12/anthropic-closes-30-billion-funding-round-at-380-billion-valuation.html

CNBC. (2026, March 5). Anthropic CEO says 'no choice' but to challenge Trump admin's supply chain risk designation in court. https://www.cnbc.com/2026/03/05/anthropic-ceo-says-no-choice-but-to-challenge-trump-admins-supply-chain-risk-designation-in-court.html

Federal News Network. (2026, January). Pentagon is embracing Musk's Grok AI chatbot as it draws global outcry. https://federalnewsnetwork.com/artificial-intelligence/2026/01/pentagon-is-embracing-musks-grok-ai-chatbot-as-it-draws-global-outcry/

Fortune. (2024, December 7). How Peter Thiel's network of right-wing techies is infiltrating Donald Trump's White House. https://fortune.com/2024/12/07/peter-thiel-network-trump-white-house-elon-musk-david-sacks/

Fortune. (2025, July 8). Users accuse Elon Musk's Grok of a rightward tilt after xAI changes its internal instructions. https://fortune.com/2025/07/08/elon-musk-grok-ai-conservative-bias-system-prompt/

Fortune. (2025, November 14). Anthropic says its latest model scores a 94% political 'even-handedness' rating. https://fortune.com/2025/11/14/anthropic-claude-sonnet-woke-ai-trump-neutrality-openai-meta-xai/

Fortune. (2025, November 17). Anthropic CEO warns that without guardrails, AI could be on dangerous path. https://fortune.com/2025/11/17/anthropic-ceo-dario-amodei-ai-safety-risks-regulation/

Fortune. (2026, February 23). OpenAI has changed its mission statement 6 times in 9 years, most recently about AI that 'safely benefits humanity'. https://fortune.com/2026/02/23/openai-mission-statement-changed-restructuring-forprofit-business/

Fortune. (2026, February 28). OpenAI sweeps in to snag Pentagon contract after Anthropic labeled 'supply chain risk'. https://fortune.com/2026/02/28/openai-pentagon-deal-anthropic-designated-supply-chain-risk-unprecedented-action-damage-its-growth/

Fox News. (2026, March 2). Musk, xAI tout newest Grok update as only 'non-woke' platform: 'Doesn't equivocate'. https://www.foxnews.com/politics/musk-xai-tout-newest-grok-update-as-only-non-woke-platform-citing-answers-to-key-questions

Google DeepMind. (2025, April). An Approach to Technical AGI Safety and Security. https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/evaluating-potential-cybersecurity-threats-of-advanced-ai/An_Approach_to_Technical_AGI_Safety_Apr_2025.pdf

GSA. (2025, September 25). GSA and xAI Partner on $0.42 per Agency Agreement to Accelerate Federal AI Adoption. https://www.gsa.gov/about-us/newsroom/news-releases/gsa-xai-partner-to-accelerate-federal-ai-adoption-09252025

NBC News. (2025). White House irked by Leading the Future, a new $100M pro-AI super PAC. https://www.nbcnews.com/news/amp/rcna239392

NPR. (2025, July 9). Elon Musk's AI chatbot, Grok, started calling itself 'MechaHitler'. https://www.npr.org/2025/07/09/nx-s1-5462609/grok-elon-musk-antisemitic-racist-content

NPR. (2025, December 12). Trump tech adviser David Sacks under fire over vast AI investments. https://www.npr.org/2025/12/12/nx-s1-5631823/david-sacks-ai-advisor-investment-conflicts

NPR. (2026, January 12). Governments ban the Grok chatbot due to nonconsensual bikini pics. https://www.npr.org/2026/01/12/nx-s1-5672579/grok-women-children-bikini-elon-musk

OpenAI. (2025, January 21). Announcing The Stargate Project. https://openai.com/index/announcing-the-stargate-project/

PLOS ONE. (2024). The political preferences of LLMs. David Rozado. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0306621

Policy and Internet. (2025). When Politicians Talk AI: Issue-Frames in Parliamentary Debates Before and After ChatGPT. https://onlinelibrary.wiley.com/doi/full/10.1002/poi3.70010

Promptfoo. (2026). Evaluating political bias in LLMs. https://www.promptfoo.dev/blog/grok-4-political-bias/

Time. (2025, August). Exclusive: 60 U.K. Lawmakers Accuse Google of Breaking AI Safety Pledge. https://time.com/7313320/google-deepmind-gemini-ai-safety-pledge/

Washington Post. (2026, February 27). Pentagon declares Anthropic a threat to national security. https://www.washingtonpost.com/technology/2026/02/27/trump-anthropic-claude-drop/

White House. (2025, January 23). Removing Barriers to American Leadership in Artificial Intelligence. https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/

White House. (2025, December). Ensuring a National Policy Framework for Artificial Intelligence. https://www.whitehouse.gov/presidential-actions/2025/12/eliminating-state-law-obstruction-of-national-artificial-intelligence-policy/

Yale University. (2026, March 3). AI's hidden bias: Chatbots can influence opinions without trying. PNAS Nexus. https://news.yale.edu/2026/03/03/ais-hidden-bias-chatbots-can-influence-opinions-without-trying

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

Something strange happened in late 2025. Engineers at Anthropic noticed that their AI agents were choking on their own capabilities. The more tools they connected, the worse their systems performed. A typical setup linking five common enterprise services (GitHub, Slack, Sentry, Grafana, and Splunk) consumed roughly 55,000 tokens just in tool definitions before the agent had even read a single user request. One internal deployment devoured 134,000 tokens on tool descriptions alone, leaving the model precious little room for actual reasoning. It was the software equivalent of filling a filing cabinet with instruction manuals and leaving no space for the files themselves.

The irony was exquisite. The Model Context Protocol, Anthropic's open standard for connecting AI agents to external systems, had succeeded beyond anyone's expectations. Launched in November 2024, MCP had grown from an internal experiment into the dominant integration standard for agentic AI, with over 10,000 active public MCP servers by late 2025 and adoption by ChatGPT, Cursor, Gemini, Microsoft Copilot, and Visual Studio Code. Official SDK downloads exceeded 97 million per month across Python and TypeScript. But this success created a paradox: the more tools agents could access, the less effective they became at using any of them.

The team's response, detailed in Anthropic's engineering blog post on advanced tool use, was not to limit tool access but to fundamentally rethink how agents discover and interact with their capabilities. The result was a trio of features that, working together, reduced context consumption by up to 85 per cent while simultaneously improving accuracy. And the design patterns they established are now reshaping how the entire industry thinks about building production agentic systems.

When Every Tool Costs You Tokens

To understand why the tool scaling problem became acute in 2025, you need to appreciate the economics of context windows. Every tool definition an agent loads carries a token cost. A modestly complex tool with a name, description, and parameter schema might consume 200 tokens. That seems trivial until you connect to a GitHub MCP server with 35 tools (roughly 26,000 tokens), a Slack server with 11 tools (21,000 tokens), and a handful of monitoring services. Suddenly, you have burned through tens of thousands of tokens before the conversation even begins.

Bin Wu, the primary author of Anthropic's advanced tool use engineering blog post and a former Airbnb engineer who joined Anthropic to work on AI safety, described the problem in stark terms. The traditional approach to tool management, loading all definitions upfront and passing them to the model, simply does not scale when developers are connecting agents to dozens or hundreds of MCP servers. Anthropic's internal testing revealed that tool selection accuracy degrades significantly once you exceed 30 to 50 available tools. The model gets overwhelmed by options, like a diner handed a 200-page menu when they just want breakfast.

This degradation is not merely anecdotal. Research on large language model tool calling has consistently demonstrated a negative correlation between tool library size and selection accuracy. The phenomenon has multiple causes. Context window saturation leaves less room for reasoning as tool definitions consume more space. The well-documented “lost in the middle” effect means models recall information at the beginning and end of their context windows more reliably than content buried in the middle, causing optimal tools to be overlooked when they appear amidst dozens of alternatives. And larger context windows alone do not solve the problem, because the core attention and selection accuracy issues persist regardless of how much space is available.

This problem has two distinct dimensions. The first is what engineers call “context bloat”: tool definitions consuming the finite token budget that the model needs for reasoning, user instructions, and conversation history. The second is “context pollution”: intermediate results from tool calls flooding the context window with data the model does not actually need. A two-hour sales meeting transcript routed through a workflow might mean processing an additional 50,000 tokens of audio transcription, even when the agent only needs a three-sentence summary.

As Adam Jones and Conor Kelly detailed in Anthropic's companion post on code execution with MCP, these two inefficiencies compound each other in production environments. An agent connected to thousands of tools must process hundreds of thousands of tokens in definitions before it even reads the user's request, and then each tool invocation potentially dumps thousands more tokens of intermediate results back into the context window. The practical ceiling is not the model's intelligence. It is the model's context budget.

The financial implications are equally pressing. Token usage translates directly into API costs. An enterprise running thousands of agent interactions daily can see its compute bills balloon when every request begins with 100,000 tokens of overhead. Latency suffers too: more input tokens mean longer processing times, which means slower responses, which means frustrated users and abandoned workflows. Before MCP, developers faced what Anthropic described as an “N by M” integration problem: ten AI applications and one hundred tools could require up to a thousand different integrations. MCP solved that problem elegantly, reducing it to a single protocol implementation on each side. But it introduced a new one: the protocol worked so well that developers connected everything, and the resulting token cost became unsustainable.

The Discovery Revolution

Anthropic's first intervention was the Tool Search Tool, a meta-capability that lets agents discover tools on demand rather than loading everything upfront. The concept is deceptively simple. Instead of passing all tool definitions to the model at the start of every conversation, developers mark tools with a defer_loading: true parameter. These deferred tools are not loaded into the model's context initially. The agent sees only the Tool Search Tool itself, plus a small set of frequently used tools that remain always-loaded. When the agent encounters a task requiring a specific capability, it searches for relevant tools, loads only the three to five it actually needs, and proceeds.

The token savings are dramatic. That five-server scenario consuming 55,000 tokens in definitions? With the Tool Search Tool, it drops to roughly 8,700 tokens, preserving 95 per cent of the context window. Across implementations, Anthropic documented an 85 per cent reduction in token usage while maintaining access to the full tool library. The system supports catalogues of up to 10,000 tools, returning three to five of the most relevant per search query.

But the real surprise was accuracy. Conventional wisdom suggested that reducing the number of visible tools would degrade performance, forcing the model to take an extra step to find what it needs. The opposite turned out to be true. By surfacing a focused set of relevant tools on demand, tool search actually improved selection accuracy, particularly with large tool libraries. Internal testing on MCP evaluation benchmarks showed Claude Opus 4 jumping from 49 per cent to 74 per cent accuracy, a 25 percentage-point improvement. Claude Opus 4.5 improved from 79.5 per cent to 88.1 per cent. The mechanism was straightforward: fewer options meant less confusion, and dynamically selected tools were more likely to be relevant to the actual task.

The Tool Search Tool supports multiple search strategies, each suited to different deployment needs. The regex-based variant, designated tool_search_tool_regex_20251119, uses Python's re.search() syntax and works well for keyword matching across tool names and descriptions. It supports exact matches, flexible patterns using wildcards, and case-insensitive searches with a maximum query length of 200 characters. The BM25-based variant, tool_search_tool_bm25_20251119, accepts natural language queries instead, using term-frequency ranking for more nuanced discovery. Both variants search tool names, descriptions, argument names, and argument descriptions. Custom embedding-based search offers a third option, enabling semantic matching that finds tools by meaning rather than exact terminology. Developers can implement whichever strategy suits their deployment, or combine them.

There is also a caching advantage that was not immediately obvious. Because deferred tools are not included in the initial prompt, the rest of the prompt remains stable across requests. This makes prompt caching significantly more effective, since the cacheable portion of the prompt does not change every time the tool set shifts. For high-volume deployments, this secondary optimisation can compound the primary token savings substantially.

Cloudflare arrived at a strikingly similar conclusion through independent research, publishing their findings under the banner of “Code Mode” in September 2025, roughly two months before Anthropic's November announcement. As Cloudflare's engineering team observed, with just two tools, search() and execute(), their server could provide access to the entire Cloudflare API over MCP while consuming only around 1,000 tokens. When new products were added, the same search and execute code paths discovered and called them automatically, with no new tool definitions and no new MCP servers required. The generated JavaScript runs in a secure, isolated V8 Worker sandbox with external network access blocked by default, and each execution receives its own Worker instance.

The convergence was striking. Two major technology companies, working independently, identified the same fundamental problem and arrived at architecturally similar solutions within weeks of each other, as noted in the Cloudflare Code Mode blog post. Cloudflare published on 26 September 2025; Anthropic followed on 4 November 2025. The posts reference each other, but these were clearly parallel discoveries driven by the same pressures: agents were scaling up, tool counts were exploding, and the old approach broke at this scale.

Writing Code Instead of Making Calls

The second major innovation was Programmatic Tool Calling, which addresses the context pollution problem rather than context bloat. Traditional tool calling works through a sequential loop: the agent requests a tool, the API returns the result, the result enters the model's context, and the agent decides what to do next. For simple workflows involving two or three tools, this is fine. For complex orchestration spanning 20 or more tool invocations, it becomes catastrophically expensive.

Consider a practical scenario: checking budget compliance across 20 team members. In the traditional approach, the agent calls a tool to retrieve each team member's spending, waits for the result, processes it in context, and calls the next tool. That is 20 round trips through the model, 20 sets of intermediate results flooding the context window, and 19 additional inference passes that each add latency and cost. Anthropic measured one such workflow consuming 43,588 tokens across all those sequential invocations.

Programmatic Tool Calling flips this model. Instead of requesting tools one at a time, the agent writes a Python script that orchestrates the entire workflow. The script runs in a sandboxed Code Execution environment, pausing when it needs results from external tools. When tool results return via the API, they are processed by the script rather than consumed by the model. The script handles loops, conditionals, error handling, and data filtering, and only the final aggregated output reaches the model's context window. In Anthropic's budget compliance example, the same workflow dropped from 43,588 to 27,297 tokens, a 37 per cent reduction on that single task. But the savings compound dramatically with complexity: one implementation documented by Adam Jones and Conor Kelly reduced token usage from 150,000 tokens to 2,000, a 98.7 per cent reduction.

The insight beneath this approach has a certain elegance. Large language models have been trained on billions of lines of real code. They are fluent in Python, JavaScript, and TypeScript. But JSON tool-calling schemas are synthetic constructs that barely appear in training data. Asking a model to orchestrate complex workflows through individual JSON function calls is like asking a concert pianist to play a symphony by pressing one key at a time and waiting for approval between each note. Programmatic Tool Calling lets the model compose the entire piece. Cloudflare's engineering team articulated the same observation independently: models are fluent in real programming languages but stutter when asked to produce function-call JSON, because they have seen millions of lines of actual code during training but only contrived tool-calling examples.

This idea did not appear in a vacuum. Joel Pobar's LLMVM project had been exploring code-based tool orchestration since before it was fashionable, allowing language models to interleave natural language and code rather than relying on traditional tool calling APIs. The project's design philosophy, that letting models write code generally results in significantly better task deconstruction and execution, prefigured the approach that both Anthropic and Cloudflare would later formalise. LLMVM uses a “continuation passing style” execution model, where queries result in natural language interleaved with code rather than a rigid sequence of code generation followed by natural language interpretation.

Anthropic's implementation requires tools to opt in to programmatic calling through an allowed_callers parameter, specifically allowed_callers: ["code_execution_20250825"]. This ensures that sensitive operations can be restricted to direct model invocation with user approval. The sandboxed execution environment provides resource limits and monitoring. Intermediate results stay within the execution environment by default, which also carries privacy benefits: the MCP client can tokenise personally identifiable information automatically, allowing real data to flow between systems while preventing the model from accessing raw PII.

On 17 February 2026, Anthropic moved Programmatic Tool Calling to general availability with the release of Claude Sonnet 4.6, signalling that the feature had graduated from experimental curiosity to production-ready infrastructure. Alongside this, web search and fetch tools gained automatic code-based filtering, cutting input tokens by 24 per cent while boosting BrowserComp accuracy from 33 per cent to 46 per cent. Joe Binder, VP of Product at GitHub, noted that Claude Sonnet 4.6 was “already excelling at complex code fixes, especially when searching across large codebases is essential.” The broader community followed suit: Block's Goose Agent added “code mode” MCP support, LiteLLM added native support across providers, and multiple open-source projects adopted the pattern.

Teaching By Example

The third feature in Anthropic's advanced tool use suite is Tool Use Examples, which addresses a subtler problem than context bloat or pollution: parameter specification errors. Even when an agent correctly identifies the right tool and calls it efficiently, it can still fail by passing malformed or incorrect parameters. JSON Schema definitions tell the model what parameters are available and their types, but they do not convey the conventions, correlations, and formatting expectations that distinguish a correct invocation from a technically valid but functionally broken one.

Consider a calendar scheduling API that accepts a date parameter. The schema specifies that the parameter is a string, but does it expect “2025-11-15”, “15/11/2025”, “November 15, 2025”, or “15 Nov 2025”? The schema might even specify a pattern, but more complex relationships between parameters, such as an “enddate” that must be after “startdate” or a “timezone” parameter that changes the interpretation of datetime values, remain invisible in the formal specification.

Tool Use Examples solve this by providing concrete usage patterns alongside schema definitions. Developers include an input_examples array with one to five examples demonstrating proper parameter formatting and usage patterns. These examples can show minimal parameter usage (just the required fields), partial parameter combinations (common optional parameter groupings), and full parameter specifications (every available option), giving the model a practical understanding of how the tool should actually be called. Anthropic recommends between one and five examples per tool, with each example addressing a different usage pattern.

The impact on accuracy is substantial. Anthropic's internal testing showed accuracy on complex parameter handling improving from 72 per cent to 90 per cent with the addition of Tool Use Examples. As the Setec Research analysis of these features noted, the improvement is particularly pronounced for APIs with ambiguous parameter relationships, where the schema alone does not capture the implicit rules governing which parameter combinations are valid. The feature addresses format conventions that JSON Schema cannot express, nested structure patterns that require demonstration rather than description, and the implicit correlations between optional parameters that experienced developers understand intuitively but struggle to formalise.

This is not a novel pedagogical insight. It is the same principle that makes code documentation more useful when it includes examples alongside API reference descriptions. Developers have long understood that showing is more effective than telling. Tool Use Examples apply this principle to the model-tool interface, giving the agent worked examples rather than abstract specifications.

The three features are designed to work together as a complementary system. Tool Search Tool reduces upfront context consumption by loading tools on demand. Programmatic Tool Calling reduces runtime context pollution by keeping intermediate results out of the model's context. Tool Use Examples reduce errors by teaching the model how to use tools correctly through demonstration rather than description alone. Together, they address the full lifecycle of what Anthropic calls “context pollution in MCP-connected agents,” as described in their advanced tool use documentation.

Scaling MCP Without Breaking the Bank

The practical challenge for enterprises is not adopting any single optimisation technique. It is managing the operational complexity of hundreds or thousands of MCP-connected tools while maintaining the progressive disclosure principles that keep agents efficient. This requires architectural thinking beyond individual feature adoption.

The recommended pattern follows what might be called layered tool management. At the first layer, a small set of three to five frequently used tools remains always-loaded with defer_loading: false. These are the tools the agent will need in nearly every interaction: perhaps a file search tool, a messaging tool, and a general-purpose retrieval tool. At the second layer, entire MCP servers can be deferred with a default_config that sets defer_loading: true across all their tools, with selective exceptions for high-use capabilities within those servers. The example Anthropic provides is a Google Drive MCP server where all tools are deferred except search_files, which remains loaded because it is the most commonly needed entry point.

This progressive disclosure architecture mirrors a principle well-established in user interface design: show users what they need now, and make everything else discoverable. The difference is that here, the “user” is an AI agent, and the cost of poor disclosure is not confusion but wasted tokens and degraded performance.

For organisations operating at genuine scale, with dozens of MCP servers and hundreds of tools, the filesystem-based discovery pattern described in Anthropic's code execution with MCP post offers an alternative architecture. Rather than registering every tool with the API upfront, tools are presented as code files on a filesystem, organised into directories by server. A TypeScript file tree might include paths like servers/google-drive/getDocument.ts and servers/salesforce/updateRecord.ts. The agent explores the filesystem to find relevant tool definitions, reading them on demand. A search_tools utility allows the agent to query for relevant definitions with configurable detail levels: name only, name plus description, or full schemas with parameters. This approach scales elegantly because adding new tools means adding new files, not modifying configuration or redeploying infrastructure.

The governance dimension is equally important. In December 2025, Anthropic donated MCP to the Agentic AI Foundation, a directed fund under the Linux Foundation co-founded by Anthropic, Block, and OpenAI with support from Google, Microsoft, Amazon Web Services, Cloudflare, and Bloomberg. Mike Krieger, Chief Product Officer at Anthropic, explained that MCP had started as an internal project and had become “the industry standard for connecting AI systems to data and tools.” The donation was designed to ensure the protocol “stays open, neutral, and community-driven as it becomes critical infrastructure for AI.” Jim Zemlin, executive director of the Linux Foundation, framed the goal as avoiding a future of “closed wall” proprietary stacks where tool connections, agent behaviour, and orchestration are locked behind a handful of platforms. Platinum members of the new foundation include Amazon, Anthropic, Block, Bloomberg, Cloudflare, Google, Microsoft, and OpenAI, with Gold members including Cisco, Datadog, Docker, IBM, JetBrains, and Oracle among others. This institutional backing means enterprises can invest in MCP-based architectures with reasonable confidence in the standard's longevity and neutrality.

Measuring What Matters

Building efficient agentic systems is one problem. Knowing whether they actually work is another entirely. The evaluation methodologies for tool-using agents are still maturing, but several frameworks have emerged from both Anthropic's internal testing and broader industry practice.

The first and most obvious metric is token efficiency: how many tokens does the agent consume per task, and how does this change as the tool library grows? Anthropic's benchmarks provide useful baselines. A five-server MCP deployment consuming 55,000 tokens in definitions with traditional loading should drop to roughly 8,700 tokens with Tool Search, a reduction that should hold proportionally as the tool count increases. Programmatic Tool Calling should yield additional reductions of 37 per cent or more on multi-step workflows, with higher savings on more complex orchestrations.

But token efficiency alone is insufficient. The more meaningful measure is task accuracy, specifically the rate at which agents select the correct tool, invoke it with proper parameters, and produce the intended outcome. Anthropic's MCP evaluation benchmarks provide one framework for this, measuring tool selection accuracy across varying library sizes. The jumps from 49 to 74 per cent (Opus 4) and from 79.5 to 88.1 per cent (Opus 4.5) with Tool Search enabled demonstrate that efficiency and accuracy can improve simultaneously, rather than trading off against each other.

For Programmatic Tool Calling, Anthropic measured knowledge retrieval accuracy improving from 25.6 per cent to 28.5 per cent, and GIA (General Instruction Adherence) benchmarks rising from 46.5 per cent to 51.2 per cent. These gains are more modest than the tool selection improvements, reflecting the fact that code-based orchestration primarily addresses efficiency rather than capability. But in production systems, small accuracy improvements compound across thousands of daily interactions.

A rigorous evaluation framework should track at least four dimensions. First, context efficiency: tokens consumed per task, broken down by tool definitions, intermediate results, and model reasoning. Second, tool selection precision: the rate at which the agent identifies the correct tool for a given subtask, measured against a labelled test set of tasks and expected tool selections. Third, parameter accuracy: the rate at which tool invocations include correct, complete, and properly formatted parameters. And fourth, end-to-end task completion: whether the overall workflow produces the correct final output, regardless of intermediate steps.

The bottleneck identification process follows naturally from these metrics. If context efficiency is poor but tool selection is accurate, the problem is in loading strategy, and Tool Search is the appropriate intervention. If tool selection degrades with library size, the search implementation needs refinement, perhaps moving from regex-based to embedding-based retrieval. If parameter errors dominate, Tool Use Examples should be expanded. If end-to-end completion lags despite good individual metrics, the orchestration logic, whether sequential or programmatic, likely needs restructuring.

Latency deserves its own evaluation track. Programmatic Tool Calling eliminates the multiple round trips inherent in sequential tool invocation, which should reduce end-to-end latency for complex workflows. But it introduces the overhead of code execution environments. Measuring wall-clock time per task, alongside token counts, reveals whether the computational overhead of sandboxed execution outweighs the savings from fewer inference passes. In Anthropic's testing, a workflow requiring 19 or more sequential inference passes collapsed into a single programmatic execution, a latency reduction that far exceeded any overhead from the sandbox.

Researchers have also proposed broader reliability frameworks for evaluating agentic systems. A 2025 study outlined twelve concrete metrics decomposing agent reliability along four key dimensions: consistency, robustness, predictability, and safety. The findings were sobering. Evaluating 14 agentic models across two complementary benchmarks, the researchers found that recent capability gains had yielded only small improvements in reliability, suggesting that making agents more capable does not automatically make them more dependable. This gap between capability and reliability underscores the importance of dedicated evaluation infrastructure that measures not just whether agents can do things, but whether they do them consistently and safely.

Design Patterns for the Production Frontier

The engineering patterns emerging from these developments suggest a maturation of agentic system architecture. Several design principles have crystallised from both Anthropic's work and the broader community's experience.

The first is what Anthropic calls the “layered approach”: address the highest bottleneck first. If tool definitions consume most of your token budget, start with Tool Search. If intermediate results dominate, implement Programmatic Tool Calling. If parameter errors cause most failures, add Tool Use Examples. This triage prevents premature optimisation and ensures that engineering effort targets the actual constraint.

The second pattern is parallel execution through code orchestration. When multiple tool calls are independent, they should execute concurrently rather than sequentially. Anthropic's documentation references asyncio.gather() for independent operations within programmatic tool calls. This pattern does not reduce token consumption, but it dramatically reduces latency for workflows involving multiple independent data retrievals.

The third pattern involves explicit return format documentation. When tools are called programmatically, the agent's code needs to parse tool outputs reliably. Anthropic recommends explicitly specifying tool output structures in tool descriptions, so the code the model writes can accurately reference fields and formats in the returned data. Without this, the model may generate code that assumes incorrect output structures, leading to runtime failures in the sandbox.

The fourth pattern addresses security and privacy boundaries. Programmatic Tool Calling introduces a new trust surface: the agent is now writing and executing code, not just making predefined function calls. The allowed_callers parameter provides opt-in control, ensuring that sensitive tools (those that modify data, access credentials, or perform irreversible actions) can be restricted to direct model invocation with explicit user approval. Cloudflare's approach adds another layer: bindings that provide pre-authorised client interfaces, ensuring that AI-generated code cannot possibly leak API keys because the keys never enter the execution environment. The binding provides an already-authorised client interface to the MCP server, and all calls made on it pass through the agent supervisor first, which holds the access tokens and injects them into outbound requests.

The fifth pattern concerns state persistence across operations. As described in Anthropic's MCP code execution documentation, agents can maintain filesystem state across operations, enabling resumption of interrupted workflows and the development of reusable functions as “skills” that persist between sessions. This transforms agents from stateless request processors into stateful systems capable of learning and adaptation within their operational context.

For enterprises evaluating these patterns, the critical implementation question is infrastructure. Code execution demands secure execution environments with appropriate sandboxing, resource limits, and monitoring. These add operational overhead compared to direct tool calls. Anthropic's managed implementation handles container management, code execution, and secure tool invocation communication, but organisations with strict data residency or compliance requirements may need to build or adapt their own execution environments. Cloudflare's approach, using Durable Objects as stateful micro-servers with their own SQL databases and WebSocket connections, offers one model for self-hosted execution, deploying once and scaling across a global network to tens of millions of instances.

The broader trajectory is unmistakable. The agent ecosystem is moving away from the “give the model everything and let it figure it out” approach that characterised early tool-using agents. In its place, a more disciplined architecture is emerging: one that treats context as a scarce resource, applies progressive disclosure to manage complexity, and uses code execution to keep intermediate processing out of the model's reasoning space. This is not merely an efficiency optimisation. It is a fundamental shift in how we think about the boundary between what the model does and what the surrounding system does.

As the MCP ecosystem continues to expand under the governance of the Agentic AI Foundation, and as tool counts scale from hundreds to thousands, the organisations that thrive will be those that master this boundary. They will build agents that know how to find the right tool without seeing every tool, that orchestrate complex workflows through code rather than conversation, and that learn from examples rather than struggling with abstract schemas. The 85 per cent context reduction is not the end state. It is the beginning of an entirely new way of building intelligent systems.

References and Sources

1. Wu, Bin. “Introducing Advanced Tool Use on the Claude Developer Platform.” Anthropic Engineering Blog, November 2025. https://www.anthropic.com/engineering/advanced-tool-use

2. Jones, Adam and Kelly, Conor. “Code Execution with MCP.” Anthropic Engineering Blog, November 2025. https://www.anthropic.com/engineering/code-execution-with-mcp

3. Wolenitz, Alon. “Advanced Tool Use in Claude API: Three New Features That Change A Lot.” Setec Research Claude Blog, November 2025. https://claude-blog.setec.rs/blog/advanced-tool-use-claude-api

4. Cloudflare Engineering. “Code Mode: Give Agents an Entire API in 1,000 Tokens.” Cloudflare Blog, September 2025. https://blog.cloudflare.com/code-mode-mcp/

5. Anthropic. “Introducing the Model Context Protocol.” Anthropic News, November 2024. https://www.anthropic.com/news/model-context-protocol

6. Anthropic. “Donating the Model Context Protocol and Establishing the Agentic AI Foundation.” Anthropic News, December 2025. https://www.anthropic.com/news/donating-the-model-context-protocol-and-establishing-of-the-agentic-ai-foundation

7. Linux Foundation. “Linux Foundation Announces the Formation of the Agentic AI Foundation (AAIF).” Linux Foundation Press Release, December 2025. https://www.linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation

8. Anthropic. “Introducing Claude Sonnet 4.6.” Anthropic News, February 2026. https://www.anthropic.com/news/claude-sonnet-4-6

9. Pobar, Joel. “LLMVM: LLM Python Agentic Runtime Prototype.” GitHub Repository. https://github.com/9600dev/llmvm

10. Model Context Protocol Specification, Version 2025-11-25. https://modelcontextprotocol.io/specification/2025-11-25

11. Pento AI. “A Year of MCP: From Internal Experiment to Industry Standard.” Pento Blog, 2025. https://www.pento.ai/blog/a-year-of-mcp-2025-review

12. Claude API Documentation. “Tool Search Tool.” Anthropic Developer Platform. https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool

13. Claude API Documentation. “Programmatic Tool Calling.” Anthropic Developer Platform. https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

Somewhere inside a foundation model trained on millions of supposedly de-identified electronic health records, a ghost lingers. Not a literal one, of course, but a data spectre: the clinical history of a patient whose records were stripped of names, addresses, and social security numbers before ever touching an algorithm. The model was never supposed to remember this person. It was supposed to learn medicine. Instead, it learned a patient.

This is the memorisation problem, and it is rapidly becoming one of the most consequential privacy challenges in clinical artificial intelligence. As healthcare systems worldwide rush to deploy foundation models trained on vast troves of electronic health record data, researchers are discovering that de-identification, the process long treated as the gold standard for protecting patient privacy, may not be enough. These models do not merely generalise medical knowledge from the populations they study. In some cases, they memorise individual patient records with enough fidelity that an adversary armed with the right prompts could extract sensitive clinical details about real people.

The implications are profound. A patient with a rare autoimmune disorder, an individual whose HIV status was recorded during a hospital visit, a person who sought treatment for substance use: these are precisely the kinds of patients whose records are most vulnerable to memorisation, because their clinical profiles are, by definition, unusual. And unusualness is exactly what makes data memorable to a machine learning model.

When Models Remember What They Should Forget

In October 2025, a team of researchers led by Sana Tonekaboni, a postdoctoral fellow at the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard, published a paper that would reframe how the clinical AI community thinks about privacy. The study, “An Investigation of Memorization Risk in Healthcare Foundation Models,” was presented at the 2025 Conference on Neural Information Processing Systems (NeurIPS) in San Diego. Co-authored with Lena Stempfle, Adibvafa Fallahpour, Walter Gerych, and Marzyeh Ghassemi, an associate professor at MIT in Electrical Engineering and Computer Science, the paper introduced a suite of black-box evaluation tests designed to probe whether foundation models trained on structured electronic health records were genuinely generalising medical knowledge or simply recalling individual patients.

The distinction matters enormously. A model that generalises has learned, say, that patients over 65 with elevated troponin levels and chest pain are at high risk of myocardial infarction. That knowledge draws on thousands of patient records and reflects a genuine population-level pattern. But a model that memorises has locked onto a singular patient record, and when prompted with the right combination of attributes, it can reproduce details about that specific individual. “Knowledge in these high-capacity models can be a resource for many communities,” Tonekaboni explained, “but adversarial attackers can prompt a model to extract information on training data.”

The framework the team developed includes methods for probing memorisation at both the embedding level, where models encode patient data as numerical representations, and the generative level, where models produce clinical outputs. Crucially, the researchers designed their tests to distinguish between benign generalisation and genuinely harmful memorisation. Not all information leakage is created equal. If a model reveals that a particular patient profile tends to involve elderly males, that reflects population statistics. If it reveals that a specific combination of laboratory values, timestamps, and diagnostic codes corresponds to a single identifiable individual, that constitutes a privacy breach.

The findings were sobering. The researchers demonstrated that the more prior knowledge an attacker possesses about a particular patient, the more likely the model is to leak additional information. Patients with rare conditions proved especially vulnerable, precisely because their clinical signatures are distinctive enough to be picked out from the broader training distribution. And while some categories of leaked information, such as a patient's age or gender, represent relatively low risk, others carry serious consequences. Diagnoses related to HIV, substance use disorders, or mental health conditions were flagged as potentially harmful disclosures that could damage a person's employment prospects, insurance coverage, or social standing.

Ghassemi, the paper's senior author, offered a practical framing of the threat. “We really tried to emphasise practicality here,” she noted. “If an attacker has to know the date and value of a dozen laboratory tests from your record in order to extract information, there is very little risk of harm. If I already have access to that level of protected source data, why would I need to attack a large foundation model for more?” The question cuts to the heart of the adversarial calculus: how much prior knowledge makes an attack feasible, and at what point does memorisation cross from theoretical vulnerability to practical danger?

The Adversary's Toolkit

To understand the scale of the memorisation threat, it helps to look beyond healthcare-specific models to the broader landscape of large language model security research. The foundational work in this space comes from Nicholas Carlini and colleagues, whose research at Google DeepMind and collaborating institutions has systematically demonstrated that language models memorise and can be made to regurgitate their training data.

In a landmark 2021 paper published at USENIX Security, Carlini, along with Florian Tramer, Eric Wallace, and others, showed that an adversary could extract hundreds of verbatim text sequences from GPT-2, including personally identifiable information such as names, phone numbers, and email addresses. The attack required no access to the training data itself, only the ability to query the model. By 2023, the same research group, now including Milad Nasr, Daphne Ippolito, and Christopher Choquette-Choo, had scaled their methods dramatically. Their paper “Scalable Extraction of Training Data from (Production) Language Models” demonstrated that an adversary could extract gigabytes of training data from both open-source models and commercial systems including ChatGPT.

The 2023 work introduced a particularly concerning technique: the divergence attack. By crafting prompts that cause a model to diverge from its normal conversational behaviour, the researchers achieved training data emission rates up to 150 times higher than those observed during typical usage. The attack essentially tricks aligned models into reverting to their pre-alignment behaviour, at which point they begin outputting memorised sequences with alarming fidelity.

What does this mean for clinical AI? The attack surface is substantial. An electronic health record foundation model trained on millions of patient records contains, by design, sensitive clinical information. Even if the records have been de-identified according to HIPAA standards, the model itself may have encoded enough information to reconstruct individual patient profiles when queried with the right combination of clinical attributes. A rare diagnosis combined with a specific age range and a distinctive pattern of laboratory values could function as a fingerprint, allowing an attacker to extract additional details that the de-identification process was supposed to protect.

The level of prior knowledge required for a successful attack varies depending on the model architecture, training methodology, and the patient population in question. Research on general-purpose language models suggests that model size strongly correlates with memorisation: larger models, with their greater capacity to store training data patterns, are more vulnerable to extraction attacks. Given that clinical foundation models are trending towards ever-larger architectures to capture the complexity of medical knowledge, this scaling relationship poses a direct tension between clinical utility and patient privacy.

De-identification Was Never Bulletproof

The memorisation problem does not exist in isolation. It builds upon decades of research demonstrating that de-identification of health data has always been more fragile than regulators and healthcare institutions have assumed.

The seminal work in this field belongs to Latanya Sweeney, now the Daniel Paul Professor of the Practice of Government and Technology at the Harvard Kennedy School. In 1997, while still a graduate student at MIT, Sweeney demonstrated that she could re-identify the medical records of then-Massachusetts Governor William Weld by cross-referencing publicly available voter registration data with de-identified hospital discharge records. The records had been stripped of names, addresses, and social security numbers, but they retained date of birth, gender, and ZIP code. Sweeney showed that just these three attributes were sufficient to uniquely identify an individual.

Her subsequent research revealed that 87 per cent of the United States population could be uniquely identified using only ZIP code, date of birth, and gender, a finding that helped shape the HIPAA Privacy Rule's Safe Harbour de-identification standard. Yet even with these protections in place, re-identification remains possible. A 2018 study demonstrated that patients could be re-identified from HIPAA-compliant de-identified datasets by cross-referencing them with publicly available newspaper articles about hospitalisations.

A 2025 paper published in AI and Ethics highlighted the particular challenge of clinical free text. Structured data fields like diagnosis codes and laboratory values can be systematically scrubbed, but clinical notes contain narrative descriptions that may include identifying details embedded in the prose: references to a patient's occupation, family circumstances, or the name of a referring physician. De-identification tools, including those powered by natural language processing, struggle with the ambiguity and variability of clinical language.

The emergence of foundation models adds a new dimension to this longstanding vulnerability. Traditional re-identification attacks required an adversary to obtain and cross-reference multiple external datasets. Memorisation attacks against AI models require only the ability to query the model itself. The model becomes both the target and the pathway to the data it was trained on, collapsing what was previously a multi-step process into a single interaction. A 2025 study published in PMC on contemporary threats to anonymised healthcare data warned that AI-based techniques can now infer identity from traditionally de-identified sources using data such as electrocardiograms or patterns of gait, data types that were never considered identifiers under existing privacy frameworks.

How AI Threats Compare with Conventional Cybersecurity Risks

The memorisation vulnerability exists within a broader landscape of healthcare cybersecurity threats that are already severe and worsening. Understanding how AI-specific risks compare with conventional attack vectors is essential for calibrating the response.

The numbers from conventional healthcare cybersecurity are staggering. In 2024, 259 million Americans had their protected health information compromised through hacking incidents, a figure driven overwhelmingly by the Change Healthcare ransomware attack. That single breach, perpetrated by the ALPHV/BlackCat ransomware group, affected approximately 190 million individuals after attackers exploited a Citrix remote access service that lacked multi-factor authentication. UnitedHealth Group, Change Healthcare's parent company, reported total cyberattack impacts of 2.457 billion dollars in the first nine months of 2024 alone.

The healthcare sector has become the most targeted industry for ransomware, accounting for 17 per cent of all ransomware attacks across sectors. Complete protected health information packages command prices of up to 1,200 dollars per record on criminal marketplaces, roughly 80 times the value of stolen credit card data. Over 80 per cent of stolen health records in 2024 were taken not from hospitals directly but from third-party vendors, software services, and business associates, highlighting the systemic nature of the vulnerability.

Against this backdrop, AI memorisation attacks represent a qualitatively different kind of threat. Conventional breaches involve exfiltrating stored data, breaking through perimeters, and exploiting network vulnerabilities. Memorisation attacks exploit the model itself as an unwitting data store. There is no firewall to breach, no database to penetrate. The sensitive information is encoded within the model's parameters, distributed across billions of numerical weights in ways that resist simple detection or removal. An attacker needs nothing more than API access to the model, which in many clinical deployment scenarios would be available to any authorised user of the system.

The two categories of threat also differ in their detectability. A ransomware attack produces obvious signs: encrypted systems, operational disruption, ransom demands. A memorisation extraction attack can be conducted through queries that resemble normal clinical usage, making it far harder to detect. Medical identity theft already takes an average of 24 months to discover, compared with four months for financial fraud. Memorisation-based data extraction could extend this detection timeline even further, because the data never technically leaves the system in the conventional sense.

Yet it would be a mistake to treat AI memorisation as the dominant threat. The scale of conventional breaches dwarfs anything that memorisation attacks have demonstrated in practice. The Change Healthcare incident compromised the records of roughly 190 million people in a single event. Memorisation attacks, by contrast, tend to target individual patients or small groups, requiring specific prior knowledge about each target. The threat from memorisation is more surgical than it is sweeping, but for the individuals affected, particularly those with rare conditions or stigmatising diagnoses, the consequences could be devastating.

The Regulatory Patchwork

The regulatory response to AI memorisation risks in healthcare remains fragmented and, in many respects, inadequate. Existing frameworks were designed for a world where privacy threats came from databases, not algorithms.

In the United States, HIPAA remains the foundational framework for protecting health information, but it was enacted in 1996, long before the emergence of clinical AI. The proposed update to the HIPAA Security Rule, published by the Department of Health and Human Services in January 2025, represents the first major revision in over a decade. The proposal eliminates the distinction between “required” and “addressable” security controls, mandates encryption for all electronic protected health information, and introduces multi-factor authentication requirements. Critically, it establishes that data used in AI training, prediction models, and algorithm development by regulated entities falls under HIPAA's protections.

However, the proposed rule does not specifically address memorisation risks. It treats AI systems primarily through the lens of conventional cybersecurity: access controls, encryption, audit logging. These measures are necessary but insufficient for a threat that is embedded within the model's learned representations rather than stored in a conventional database. The public comment period for the proposed rule closed in March 2025 with nearly 5,000 submissions, and the final rule is expected in late 2025 or 2026. Whether it will address the unique characteristics of AI memorisation remains uncertain.

The European Union's approach through the AI Act offers somewhat more specificity. The regulation classifies AI systems used in healthcare as high-risk, subjecting them to requirements for data governance, transparency, human oversight, and post-market monitoring. From August 2026, most obligations will apply, with full compliance for high-risk medical device AI required by August 2027. The Medical Device Coordination Group published guidance document MDCG 2025-6 to clarify how the AI Act interacts with existing medical device regulations under the MDR and IVDR frameworks.

The AI Act's data governance requirements are particularly relevant to memorisation. High-risk AI manufacturers must implement practices appropriate for the intended purpose, including attention to possible biases and privacy risks. The transparency obligations require that systems be designed to allow deployers to interpret outputs and use systems appropriately. These provisions create a regulatory foundation that could, in principle, require memorisation testing before deployment. But the specifics of implementation remain to be worked out through standards and guidance that have not yet been finalised.

At the state level in the United States, a patchwork of legislation is emerging. By 2025, over 250 AI-related bills had been introduced across more than 34 states. Texas enacted the Responsible Artificial Intelligence Governance Act in June 2025, requiring healthcare practitioners to provide patients with written disclosure of AI use in diagnosis or treatment. Colorado and Utah have enacted their own comprehensive AI laws. The result is a fragmented landscape that creates compliance challenges for healthcare organisations operating across jurisdictions whilst providing inconsistent protection for patients.

Technical Safeguards and Their Limits

The technical toolkit for mitigating memorisation risks is growing, though no single approach offers a complete solution.

Differential privacy, the mathematical framework developed by computer scientists including Cynthia Dwork of Harvard University, provides formal guarantees about information leakage during model training. By adding carefully calibrated statistical noise to the training process, differential privacy ensures that the model's outputs reveal almost nothing about any individual training example. Recent research has demonstrated that healthcare AI models can achieve 96.1 per cent accuracy with a privacy budget of epsilon equals 1.9, suggesting that strong privacy and high clinical performance can coexist.

Yet differential privacy has limitations. The privacy-utility trade-off is real: stronger privacy guarantees require more noise, which can degrade model performance on clinical tasks where accuracy directly affects patient outcomes. The United States Census Bureau's experience with differential privacy in the 2020 census provides a cautionary example. Research found that the technique introduced disproportionate discrepancies for rural and non-white populations, raising concerns about equity impacts that would be equally relevant in clinical settings where underrepresented populations already face disparities in care.

Federated learning offers another approach, keeping patient data decentralised across institutions whilst training a shared model. Rather than aggregating raw data on a central server, each participating hospital trains the model locally and shares only model updates. Yet research has shown that these model updates themselves can leak information. Gradient inversion attacks can reconstruct substantial portions of original training data from the mathematical updates exchanged during federated learning. A study titled “Two Models are Better than One: Federated Learning Is Not Private for Google GBoard Next Word Prediction” demonstrated that user sentences could be reconstructed from model updates alone.

Machine unlearning, the targeted removal of specific patient data from a trained model, has emerged as a conceptually appealing response to memorisation. The approach aligns with the General Data Protection Regulation's right to be forgotten, which allows individuals to request deletion of their personal data. Research presented at MICCAI 2025 introduced Forget-MI, a method for unlearning multimodal medical data from trained architectures. A December 2025 testbed called MedForget modelled hospital data as a nested hierarchy, enabling fine-grained unlearning assessment across multiple organisational levels.

But machine unlearning faces fundamental practical barriers. Retraining a model from scratch without specific patient data remains the only guaranteed path to complete unlearning, and for large foundation models, retraining can take weeks and cost millions of dollars. Approximate unlearning methods are faster but cannot guarantee that all traces of a patient's data have been removed. Moreover, if certain demographic groups are more likely to exercise their right to be forgotten, the resulting training data could become skewed, potentially worsening the very biases that clinical AI is supposed to help address. As a Health Affairs analysis noted, machine unlearning “is computationally intensive, scientifically immature, and potentially destabilising to models that must remain reliable across a wide range of clinical inputs.”

Data deduplication, the removal of repeated training examples, provides a simpler but partial mitigation. Research has consistently shown that models are more likely to memorise data that appears multiple times in training sets. Curating and deduplicating training data can reduce memorisation rates, though it cannot eliminate the risk entirely for patients whose clinical profiles are inherently distinctive.

Building an Evaluation Framework That Works

The MIT team's work points towards what a comprehensive evaluation framework for clinical AI memorisation might look like. Their open-source toolkit, validated on a publicly available electronic health record foundation model, provides a starting point for systematic privacy assessment before model deployment.

The framework's key innovation is contextualising memorisation within healthcare. Not all information leakage constitutes a meaningful privacy risk. A model that reveals population-level patterns, such as the typical age distribution of patients with a particular condition, is doing exactly what it was designed to do. The danger arises when a model's outputs can be traced to a specific individual, particularly when the leaked information includes sensitive diagnoses or treatment histories.

Tonekaboni emphasised the importance of practical evaluation. “This work is a step towards ensuring there are practical evaluation steps our community can take before releasing models,” she said. The framework assesses both embedded memorisation, where patient information is encoded in the model's internal representations, and generative memorisation, where the model can be prompted to produce patient-specific outputs. By testing across both dimensions, the framework provides a more complete picture of privacy risk than either approach alone.

For this kind of evaluation to become standard practice, it would need to be integrated into the regulatory approval process for clinical AI systems. Currently, most AI-enabled medical devices in the United States are cleared through the FDA's 510(k) pathway, which requires demonstration of substantial equivalence to a previously approved device but does not mandate independent clinical performance studies or privacy evaluation. A cross-sectional study of 903 FDA-approved AI devices found that clinical performance studies were reported for only approximately half at the time of regulatory approval. Memorisation testing is not part of the approval process at all.

The Coalition for Health AI (CHAI), on whose working group Ghassemi serves, represents one effort to establish industry-wide standards for trustworthy health AI. The NIST AI Risk Management Framework provides a complementary structure, addressing validity, reliability, safety, security, explainability, privacy, and fairness. Integrating memorisation evaluation into these existing frameworks would be more practical than creating entirely new regulatory apparatus, but it requires agreement on what constitutes acceptable levels of memorisation risk, a question that remains open.

Rare Conditions, Outsized Vulnerability

The memorisation problem falls hardest on the patients who can least afford it. Individuals with rare diseases, by definition, have clinical profiles that stand out from the broader population. Their diagnostic codes appear infrequently in training data. Their laboratory value patterns are unusual. Their treatment trajectories are distinctive. All of these characteristics make their records more memorable to a model and more extractable by an adversary.

The same is true for patients with stigmatising diagnoses. HIV status, substance use disorders, psychiatric conditions, and sexually transmitted infections all carry social consequences that extend far beyond the clinical encounter. Disclosure of these conditions can affect employment, housing, insurance, and personal relationships. De-identification was supposed to sever the link between these sensitive details and the individuals they describe. Memorisation threatens to re-forge that link through the model itself.

This disproportionate vulnerability raises equity concerns that mirror broader patterns in healthcare AI. Research has repeatedly shown that AI systems can perpetuate and amplify existing biases against marginalised populations. If memorisation risks are concentrated among patients with rare or stigmatising conditions, the privacy burden falls most heavily on those who are already underserved by the healthcare system.

Addressing this inequity requires targeted protections. Higher levels of differential privacy noise could be applied to training data involving sensitive diagnoses, at the cost of reduced model performance for those specific conditions. Rare disease patient records could be excluded from training sets entirely, though this would eliminate the clinical utility of foundation models for precisely the populations that stand to benefit most from AI-assisted care. Neither option is satisfactory, and the tension between privacy protection and clinical benefit for rare disease patients may prove to be one of the defining challenges of clinical AI governance.

What Genuine Protection Requires

The path from current vulnerability to genuine protection requires action across multiple domains simultaneously. No single technical safeguard, regulatory standard, or evaluation framework will suffice in isolation.

On the technical side, differential privacy during training should become the default rather than the exception for clinical foundation models. Memorisation evaluation, using frameworks like the one developed by Tonekaboni and colleagues, should be mandatory before any model is deployed in a clinical setting. Ongoing monitoring should be built into deployment infrastructure to detect potential memorisation-based extraction attempts in real time. And machine unlearning capabilities, however immature, should be developed and standardised so that patients can exercise meaningful control over the fate of their data within AI systems.

On the regulatory side, HIPAA needs to evolve beyond its current framework to address threats that are embedded within model architectures rather than stored in conventional databases. The EU AI Act's high-risk classification for healthcare AI provides a useful starting point, but implementation guidance must specifically address memorisation risks. Regulatory bodies including the FDA, the European Medicines Agency, and national health authorities need to incorporate memorisation testing into their approval and post-market surveillance processes.

On the institutional side, healthcare organisations deploying clinical AI must treat memorisation as a distinct category of risk requiring its own governance structures, audit procedures, and incident response plans. The conventional cybersecurity toolkit, with its emphasis on perimeter defence, encryption, and access control, is necessary but not sufficient for threats that live inside the model rather than outside the firewall.

The researchers behind the MIT study plan to expand their work to become more interdisciplinary, bringing in clinicians, privacy experts, and legal scholars. That instinct is exactly right. The memorisation problem sits at the intersection of computer science, medicine, law, and ethics, and solving it will require all four disciplines working in concert.

“There's a reason our health data is private,” Tonekaboni observed. “There's no reason for others to know about it.” That principle has guided health privacy law for decades. The question now is whether it can survive the age of foundation models trained on the very data it was designed to protect. The answer will depend on whether the clinical AI community treats memorisation as a fundamental design constraint rather than an afterthought, building privacy into the architecture of these systems from the ground up rather than bolting it on after deployment. The technology to do so exists. Whether the will and the regulatory momentum exist to mandate it remains the open question.

---

References and Sources

1. Tonekaboni, S., Stempfle, L., Fallahpour, A., Gerych, W., and Ghassemi, M. “An Investigation of Memorization Risk in Healthcare Foundation Models.” arXiv:2510.12950, presented at NeurIPS 2025. https://arxiv.org/abs/2510.12950

2. MIT News. “MIT scientists investigate memorization risk in the age of clinical AI.” January 5, 2026. https://news.mit.edu/2026/mit-scientists-investigate-memorization-risk-clinical-ai-0105

3. Carlini, N., Tramer, F., Wallace, E., et al. “Extracting Training Data from Large Language Models.” USENIX Security 2021. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting

4. Nasr, M., Carlini, N., Hayase, J., et al. “Scalable Extraction of Training Data from (Production) Language Models.” arXiv:2311.17035, 2023. https://arxiv.org/abs/2311.17035

5. Sweeney, L. “Simple Demographics Often Identify People Uniquely.” Carnegie Mellon University, Data Privacy Working Paper 3, 2000. https://dataprivacylab.org/people/sweeney/work/index.html

6. Sweeney, L. “Risks to Patient Privacy: A Re-identification of Patients in Maine and Vermont Statewide Hospital Data.” Technology Science, 2018. https://techscience.org/a/2018100901/

7. PMC. “Addressing contemporary threats in anonymised healthcare data using privacy engineering.” 2025. https://pmc.ncbi.nlm.nih.gov/articles/PMC11885643/

8. Springer Nature. “What is the patient re-identification risk from using de-identified clinical free text data for health research?” AI and Ethics, 2025. https://link.springer.com/article/10.1007/s43681-025-00681-0

9. HIPAA Journal. “Healthcare Data Breach Statistics.” https://www.hipaajournal.com/healthcare-data-breach-statistics/

10. UnitedHealth Group. “UnitedHealth Group Updates on Change Healthcare Cyberattack.” April 22, 2024. https://www.unitedhealthgroup.com/newsroom/2024/2024-04-22-uhg-updates-on-change-healthcare-cyberattack.html

11. HHS. “Changes Proposed to Strengthen HIPAA Security Rule.” January 2025. https://www.hhs.gov/hipaa/for-professionals/special-topics/de-identification/index.html

12. Reed Smith. “The EU AI Act and Medical Devices: Navigating High-Risk Compliance.” 2025. https://www.reedsmith.com/our-insights/blogs/viewpoints/102kq35/the-eu-ai-act-and-medical-devices-navigating-high-risk-compliance/

13. European Commission. “Medical Devices Joint Artificial Intelligence Board, MDCG 2025-6.” 2025. https://health.ec.europa.eu/document/download/b78a17d7-e3cd-4943-851d-e02a2f22bbb4_en

14. Health Affairs Forefront. “Unlearning In Medical AI: A New Frontier For Privacy, Regulation, And Trust.” 2025. https://www.healthaffairs.org/content/forefront/unlearning-medical-ai-new-frontier-privacy-regulation-and-trust

15. MICCAI 2025. “Forget-MI: Machine Unlearning for Forgetting Multimodal Information in Healthcare Settings.” https://arxiv.org/html/2506.23145

16. MedForget. “MedForget: Hierarchy-Aware Multimodal Unlearning Testbed for Medical AI.” December 2025. https://arxiv.org/html/2512.09867v1

17. Nature Medicine. “Medical large language models are vulnerable to data-poisoning attacks.” January 2025. https://www.nature.com/articles/s41591-024-03445-1

18. Becker's Hospital Review. “EHR-trained AI could compromise patient privacy: MIT.” 2026. https://www.beckershospitalreview.com/healthcare-information-technology/ai/ehr-trained-ai-could-compromise-patient-privacy-mit/

19. Cobalt. “Healthcare Data Breach 2025 Statistics.” https://www.cobalt.io/blog/healthcare-data-breach-statistics

20. NIST AI Risk Management Framework. https://www.nist.gov/artificial-intelligence/ai-risk-management-framework

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...

In December 2025, MIT announced a programme that would have seemed implausible even a decade earlier: a two-year master's degree designed to teach naval officers the fundamentals of artificial intelligence, machine learning, and autonomous systems. The programme, designated 2N6, pairs the university's Department of Mechanical Engineering with its Department of Electrical Engineering and Computer Science, awarding graduates both a Master of Science in mechanical engineering and an AI certificate from the MIT Schwarzman College of Computing. It is, in essence, a bet that the future of naval warfare will be shaped not by those who build the biggest ships, but by those who best understand the algorithms directing them.

The timing is no coincidence. In January 2026, the Department of Defense released its Artificial Intelligence Acceleration Strategy, declaring its intention to become an “AI-first” organisation. Under Secretary for Research and Engineering Emil Michael had already pruned the Pentagon's list of critical technology areas from fourteen to six, placing applied artificial intelligence at the very top. And at U.S. Indo-Pacific Command, where the prospect of conflict with a technologically sophisticated adversary concentrates minds with particular intensity, Commander Admiral Samuel Paparo had been arguing for months that future wars would be won not by superior firepower alone, but by whoever could “see, understand, decide and act faster.” The question was no longer whether the military needed AI-literate officers, but how quickly it could produce them.

The origins of 2N6 trace back to a campus visit by Paparo himself. The admiral toured MIT's existing AI research facilities and immediately recognised a gap. The university had maintained the 2N Naval Construction and Engineering programme since 1901, training generations of officers in ship design and acquisition. The programme was about to celebrate its 125th anniversary in 2026. But the world had changed. The defining technologies of 21st-century naval power were no longer hull forms and propulsion systems alone; they were neural networks, reinforcement learning, and autonomous underwater vehicles. Paparo envisioned an applied AI programme modelled on the existing 2N infrastructure, and within months, 2N6 began taking shape.

Commander Christopher MacLean, MIT associate professor of the practice in mechanical engineering, naval construction, and engineering, has been central to the programme's development. MacLean, himself a graduate of the 2N programme whose thesis focused on the fracture and plasticity characterisation of DH-36 Navy steel, explained that Paparo “was given an overview of some of the cutting-edge work and research that MIT has done and is doing in the field of AI” and “made the connection, envisioning an applied AI program similar to 2N.” In describing the programme's scope, MacLean was emphatic about breadth: “AI is a force multiplier that can be used for data processing, decision support, unmanned and autonomous systems, cyber defence, logistics and supply chains, energy management, and many other fields.” This is not a programme narrowly focused on weapons systems or battlefield robots; it treats artificial intelligence as a pervasive capability touching every aspect of naval operations.

Dan Huttenlocher, the inaugural Dean of the MIT Schwarzman College of Computing, lent institutional weight to the announcement. “I'm honoured that the college can contribute to and support such a vital program that will equip our nation's naval officers with the technical expertise they need,” Huttenlocher stated. His involvement signals the seriousness of MIT's commitment: Huttenlocher, who previously founded Cornell Tech and co-authored “The Age of AI: And Our Human Future” with Henry Kissinger and Eric Schmidt, brings both academic credibility and a deep engagement with the societal implications of artificial intelligence.

A Curriculum Built for the Contested Spectrum

The 2N6 curriculum reflects a deliberate attempt to balance theoretical depth with operational relevance, structured to satisfy the U.S. Navy's sub-specialty code for Applied Artificial Intelligence. Students begin with a “Summer Camp” of foundational courses covering linear algebra and optimisation, introductory programming, discrete mathematics and proofs, algorithms and data structures, and software fundamentals. These are not optional polish; they are prerequisites designed to ensure that officers arriving from operational billets, where they may have spent years commanding ships or submarines rather than writing code, have the mathematical and computational fluency to engage with what follows.

The core of the programme divides into several tracks. The probability, inference, and machine learning sequence includes courses in stochastic dynamical systems, introduction to probability, introduction to inference, and both introductory and advanced machine learning. These build toward specialised AI topics: advances in computer vision, topics in multi-agent learning, quantitative methods for natural language processing, optimisation methods, and a course titled “AI, Decision Making and Society.” That final course is significant. It signals that 2N6 does not treat artificial intelligence as a purely technical problem but as one embedded in social, political, and ethical contexts that military leaders must navigate with the same rigour they apply to technical challenges.

The naval applications track offers four areas of concentration, each designed to connect AI theory to operational reality. In autonomy, students study unmanned marine vehicle autonomy, sensing and communications, manoeuvring and control of surface and underwater vehicles, and principles of autonomy and decision making. In design and manufacturing, the focus turns to AI and machine learning for design, principles of naval ship design, and manufacturing processes and systems. A games and strategy track covers reinforcement learning combined with game theory and wargaming, preparing officers for the adversarial dynamics of actual conflict. And an innovation track provides team-based interdisciplinary collaboration, simulating the cross-functional problem-solving that AI deployment demands in practice.

Themis Sapsis, the William I. Koch Professor in mechanical engineering and Director of the Center for Ocean Engineering at MIT, has described the programme as “specifically designed to train naval officers on the fundamentals and applications of AI, but also involve them in research that has direct impact to the Navy.” Sapsis, who holds a diploma in naval architecture and marine engineering from the Technical University of Athens and a PhD in mechanical and ocean engineering from MIT, brings direct domain expertise to the programme. His own research spans nonlinear dynamical systems, probabilistic modelling, and data-driven methods, with applications ranging from predicting catastrophic sea waves to calculating extreme loads on warships. His work has been recognised with awards from the Office of Naval Research, the Army Research Office, and the Air Force Office of Scientific Research. “2N6 can model a new paradigm for advanced AI education focused more broadly on supporting national security,” Sapsis has emphasised, positioning the programme not merely as a naval initiative but as a potential template for defence AI education writ large.

John Hart, Head of MIT's Department of Mechanical Engineering, framed the programme in generational terms: “With the 2N6 program, we're proud to be at the helm of such an important charge in training the next generation of leaders for the Navy.” Asu Ozdaglar, Deputy Dean of the Schwarzman College of Computing, similarly described the partnership as “an important collaboration with the U.S. Navy” that reflects the college's broader mission to bring computing expertise to consequential domains.

The Technical Competencies That Matter

The specific competencies the programme prioritises reveal much about where the U.S. Navy believes its AI gaps are most acute. Autonomous systems sit at the top of the list, and for good reason. Admiral Paparo has been explicit about wanting large numbers of low-cost, long-endurance unmanned sensor platforms, including drones, robot ships, and autonomous underwater vehicles, to maintain persistent surveillance across the Indo-Pacific. With Chinese wargames growing ever larger and more realistic, Paparo has argued that traditional intelligence “indications and warning” can no longer reliably distinguish between exercises and an actual invasion preparation. His proposed solution: surveillance drones feeding AI analysis to detect anomalies and patterns more quickly and accurately than human analysts could manage alone.

“We never send a human being to do something that a machine can do,” Paparo has stated. “We never lose human agency over offensive power.” The tension between those two principles captures the central challenge of military autonomy: expanding the envelope of machine capability whilst maintaining meaningful human control. Graduates of 2N6 will be expected to design and manage systems that operate in this tension, understanding both the engineering of autonomy and the doctrinal requirements for human oversight.

Cyber defence represents another critical domain. The ability to protect AI systems themselves from adversarial manipulation, data poisoning, and model exploitation is becoming as important as the AI capabilities those systems provide. An AI-enabled fleet that can be fooled by adversarial inputs or compromised through supply chain attacks on its training data becomes a liability rather than an advantage. The curriculum's emphasis on algorithms, data structures, and software fundamentals is not merely academic preparation; it provides the conceptual toolkit for understanding how AI systems can be attacked and defended. MIT Lincoln Laboratory's Embedded and Open Systems Group has been developing AI research environments specifically to evaluate promising embedded AI technologies and their impact on critical defence missions, from advanced multimodal navigation to synthetic aperture radar object detection.

Decision intelligence, the application of AI to command-and-control processes, constitutes perhaps the most consequential area. At U.S. Indo-Pacific Command, AI is already being pursued to accelerate the decision cycle and provide predictive analysis for logistics. Colonel Jared Voneida, INDOPACOM's C4 Operations Division chief, has noted that AI is being pursued to speed up the decision cycle across every warfighting function. The concept of “decision superiority,” which Paparo has defined as understanding “who is making the best decisions, who is best able to see, understand, decide, act, learn and assess,” depends on officers who can critically evaluate AI-generated recommendations rather than simply accepting them. This requires not just technical literacy but a sophisticated understanding of where AI excels, where it fails, and how to design human-machine teaming arrangements that exploit strengths whilst compensating for weaknesses.

Machine learning for manufacturing and design rounds out the technical portfolio. Naval shipbuilding remains an enormously complex industrial undertaking, and AI-driven design optimisation, predictive maintenance, and manufacturing process control offer significant potential for reducing costs and timelines. MIT Lincoln Laboratory has already demonstrated systems like COVAS (Human-Machine Collaborative Optimisation via Apprenticeship Scheduling), which uses machine learning to provide real-time ship defence scheduling solutions by learning from human experts. COVAS is the first and only algorithm to provide such real-time solutions, and researchers plan to mature the technology before proposing it as a Future Naval Capability to the Office of Naval Research. Maintenance operations across INDOPACOM are also being transformed through AI-enabled predictive systems that analyse sensor data from shipboard systems and aircraft components to identify potential failures before they become critical. Graduates of 2N6 would be expected to evaluate, integrate, and manage such systems across the fleet.

Ethics, Governance, and the Responsible AI Question

Perhaps the most consequential element of the 2N6 curriculum is one that might easily be overlooked: the mandatory inclusion of coursework in the social and ethical responsibilities of computing. This is not a token addition. The MIT Schwarzman College of Computing operates SERC (Social and Ethical Responsibilities of Computing), a cross-cutting initiative led by associate deans Nikos Trichakis and Brian Hedden. SERC develops peer-reviewed case studies, active learning projects, and pedagogical materials addressing privacy and surveillance, inequality and justice, autonomous systems and robotics, ethical computing practice, and law and policy. Its materials are based on original research, published through open-access licensing, and designed for integration across MIT's computing curriculum. Naval officers in 2N6 will encounter these frameworks not as a separate ethics module bolted onto a technical degree, but as an integral dimension of their AI education.

This integration matters because the Department of Defense has its own ethical framework that graduates will be expected to operationalise. The DoD adopted five principles for the ethical development of AI capabilities: responsible, equitable, traceable, reliable, and governable. The Responsible AI Strategy and Implementation Pathway translates these principles into concrete requirements, promoting human-machine teaming rather than fully autonomous systems and requiring that AI technologies be integrated in a lawful, ethical, and accountable manner. The DoD's Responsible AI Toolkit builds on the Defence Innovation Unit's guidelines, NIST's AI Risk Management Framework, and IEEE 7000-2021, establishing standards for operationalising ethical principles throughout the technology lifecycle. The Defence Innovation Unit launched its strategic initiative in March 2020 specifically to implement ethical principles into commercial prototyping and acquisition programmes, ensuring alignment through a process designed to be reliable, replicable, and scalable.

The question of traceability deserves particular attention. Traceability, in the DoD's formulation, means the ability to track and document all data and decisions of an AI tool, including how it was trained and how it processes information. For officers deploying AI in operational contexts, this creates obligations that are simultaneously technical (implementing logging, auditing, and explainability mechanisms) and organisational (ensuring that chains of command can meaningfully review AI-informed decisions). The programme's emphasis on algorithms, inference, and decision-making provides the technical foundation for understanding traceability, whilst the ethics coursework provides the normative framework for why it matters.

Yet genuine tensions remain. The DoD's ethical principles exist alongside a policy environment that has shifted significantly. President Biden's Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of AI, issued in October 2023, established foundational requirements for AI safety across federal agencies. That order was revoked in January 2025, and the subsequent AI Action Plan focuses less on safe development and more on acceleration. The DoD's own ethical principles remain formally in place, but the broader political context creates ambiguity about how rigorously they will be enforced. As Paparo himself has put it: “we need robust, ethical AI systems that enhance decision-making while fiercely preserving human oversight of critical operations.” Officers trained at MIT will enter a system where stated principles and operational incentives may not always align, making their ability to navigate ethical complexity all the more important.

The Dual-Use Dilemma

The technologies that 2N6 graduates will master are, almost without exception, dual-use. The same computer vision algorithms that identify military targets can diagnose medical conditions. The same natural language processing techniques that analyse intercepted communications can power consumer chatbots. The same reinforcement learning methods that optimise military logistics can manage commercial supply chains. This fundamental characteristic of AI technology, that its military and civilian applications are often indistinguishable at the algorithmic level, creates governance challenges that no single curriculum can resolve.

Research published in PMC (PubMed Central) has documented what scholars term the “double-distinguishability problem” of AI: not only is AI software with potential military applications likely to reside in both military and civilian networks, but even within the military domain, distinguishing between platforms that integrate AI and those that do not is extremely difficult. This complicates arms control, export regulation, and confidence-building measures. The degree of transparency required to build international confidence or ensure compliance with agreements may itself produce security vulnerabilities, discouraging cooperation.

The inherent opacity of many advanced machine learning systems compounds the problem. Despite strong performance in testing environments, the underlying reasoning of deep neural networks remains largely opaque. This “black box” quality compromises the human oversight required to uphold legal and ethical standards in military operations, particularly when AI decisions are made in milliseconds. Legal regimes must clarify fault attribution, determining whether responsibility falls on the commanding officer, the system developer, the algorithm designer, or the deploying state. What constitutes “meaningful human control” remains ambiguous and case-dependent, with a recent analysis noting that a human can technically interact with an autonomous system without having any substantive moral, legal, or operational oversight.

The United Nations Office for Disarmament Affairs convened the Military AI, Peace and Security Dialogues in 2025, where participants emphasised retaining human judgement and control over decisions on the use of force. They cautioned that legal determinations should not be coded into opaque systems, and that decision-making support tools should enable, not replace, legality and ethical reasoning. The U.S. State Department's Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy established broader norms, requiring that military AI use comply with international humanitarian law, that accountability be maintained through a responsible human chain of command, and that states take proactive steps to minimise unintended bias.

For MIT's 2N6 graduates, this dual-use reality means that their technical skills will be applicable across domains, but their ethical and governance training will need to be specifically calibrated for military contexts where the consequences of error are measured in lives rather than revenue. The programme's integration of game theory, wargaming, and reinforcement learning acknowledges that military AI operates in adversarial environments where rational actors are actively trying to exploit, deceive, or defeat the systems being deployed.

The Global AI Arms Race in Uniform

MIT's 2N6 programme does not exist in a vacuum. It is one move in an accelerating international competition to build AI-literate military forces, and the landscape of that competition reveals starkly different approaches to the same underlying challenge.

China represents the most direct competitive pressure. The People's Liberation Army views AI as leading to the next revolution in military affairs and expects to field a range of “algorithmic warfare” and “network-centric warfare” capabilities by 2030. Georgetown University's Center for Security and Emerging Technology has identified 370 Chinese institutions whose researchers have published papers related to general artificial intelligence. The PLA's approach relies heavily on military-civil fusion, integrating universities and commercial technology companies directly into defence research and development. A majority of suppliers for AI-related PLA procurement contracts are now civilian companies and universities rather than traditional state-owned defence enterprises.

Chinese researchers at institutions linked to the PLA's Academy of Military Science used Meta's open-source Llama 2 13B model to build “ChatBIT,” a military-focused AI tool fine-tuned and “optimised for dialogue and question-answering tasks in the military field.” The PLA rapidly adopted DeepSeek's generative AI models in early 2025, likely deploying them for intelligence purposes. The Pentagon's 2024 China report noted that “China's commercial and academic AI sectors made progress on large language models and LLM-based reasoning models, which has narrowed the performance gap between China's models and the U.S. models currently leading the field.” China's emerging 15th Five-Year Plan framework is expected to institutionalise military-civil fusion as the primary pathway for achieving what Chinese strategists call an “intelligentised” PLA by 2035.

Russia has pursued a different trajectory, constrained by sanctions and a smaller technology sector. The National Strategy for the Development of Artificial Intelligence, signed by President Putin in 2019, set targets of training 15,500 AI specialists by 2030 and allocated 26.49 billion rubles to AI development from 2025 to 2027. Russia aims to automate 30 percent of its military equipment and has begun integrating AI into systems like the ZALA Lancet drone swarm, which reportedly allows drones to exchange information and divide tasks autonomously. However, senior Russian military experts, including Vladimir Prikhvatilov of the Academy of Military Science, have acknowledged that Russia has “virtually no chances to catch up with the Chinese or the Americans” in military AI. The war in Ukraine has both accelerated urgency and exposed the gap between Russia's AI rhetoric and its actual capabilities, with international sanctions further constraining access to advanced computing hardware.

The United Kingdom offers a more direct parallel to MIT's approach. The UK Ministry of Defence published its Defence Artificial Intelligence Strategy describing an “ambitious, safe, responsible” approach to military AI. The Alan Turing Institute, as a strategic partner of the Defence Science and Technology Laboratory (Dstl), conducts defence-relevant AI research and has published frameworks for AI assurance in military contexts, including a commander's guide for uncrewed systems and recommendations for iteratively identifying, documenting, and communicating risks. A January 2025 Defence Committee report called on the Ministry of Defence to “transform itself into an 'AI-native' organisation” whilst acknowledging that the sector remained under-developed. Sub-committee chair Emma Lewell-Buck emphasised the need to make AI “a greater part of military education” and to facilitate movement between civilian and defence AI sectors, a recommendation that echoes precisely the gap MIT's 2N6 programme is designed to fill.

Israel has arguably moved furthest in operational deployment. The IDF established the Artificial Intelligence and Big Data Research Centre, created a new AI Division within its C4I and Cyber Defence Directorate following lessons from the Israel-Hamas War, and in January 2025, the Israeli Ministry of Defence established the AI and Autonomy Administration. Eyal Zamir, the Ministry's director general, emphasised that this was the first new administration established within the Ministry in over two decades. Approximately 750 military reservists were enrolled in AI training programmes organised by Israel's Innovation Authority and the Ministry of Defence in January 2026, reflecting a recognition that AI literacy cannot be confined to active-duty specialists. The IDF's model of recruiting talented high school graduates into elite technology units like Unit 8200, training them intensively through programmes like the 36-month Havatzalot Programme at Hebrew University, and then cycling them into the civilian technology sector creates a distinctive pipeline that no other nation has fully replicated.

Reshaping the Defence Workforce

The emergence of programmes like 2N6 points toward a fundamental recomposition of what militaries expect from their officer corps. The traditional career path, in which technical specialists remained in engineering billets whilst operational commanders focused on tactics and leadership, is giving way to a model that demands hybrid competency. Officers who will command AI-enabled forces need enough technical understanding to evaluate what their systems can and cannot do, enough ethical grounding to make responsible deployment decisions, and enough strategic vision to understand how AI reshapes the character of conflict.

The Naval Postgraduate School in Monterey, California, announced its own accelerated one-year Master of Science in Artificial Intelligence in late 2025, set to commence in July 2026. The programme comprises 21 courses, requires residency in Monterey, and is open to active-duty military officers, DoD civilian employees, and allied officers with computer science backgrounds. An NPS AI initiative launched in early 2025 established three lines of effort: AI education, problem-solving, and technology infrastructure, with industry partners including NVIDIA supporting cutting-edge education and applied research. Meanwhile, NPS also offers a distance-learning AI certificate comprising four courses, designed for military professionals without technical backgrounds, recognising that even non-specialist officers need baseline AI literacy.

Emil Michael declared that “the Department of War must become an 'AI-First' organisation,” and the January 2026 AI Acceleration Strategy codified this vision through four broad aims: incentivising internal experimentation with AI models, eliminating bureaucratic obstacles, focusing military investment on asymmetric advantages, and initiating Pace-Setting Projects. Cameron Stanley, previously chief of the DoD Algorithmic Warfare Cross Functional Team (formerly known as Project Maven) and a former national security transformation lead for Amazon Web Services, was appointed to lead the Applied Artificial Intelligence critical technology area.

These developments suggest a future in which AI literacy becomes a prerequisite for advancement rather than a specialist qualification. Just as nuclear propulsion reshaped the U.S. Navy's officer corps in the 1950s and 1960s, creating a cadre of nuclear-trained officers led by Admiral Hyman Rickover whose influence extended far beyond the engineering department, AI may create a similar dynamic. Officers who understand machine learning, autonomous systems, and decision intelligence will increasingly populate senior leadership positions, bringing with them assumptions, methodologies, and risk tolerances shaped by their technical training.

The implications extend well beyond the United States. As the UK Defence Committee recognised, military AI development requires not just technical infrastructure but a transformed workforce. The challenge is particularly acute for smaller nations that cannot replicate MIT's resources or the NPS's scale. International partnerships, joint training programmes, and standardised AI competency frameworks may emerge as mechanisms for distributing AI literacy across allied military forces. The 2N6 programme already anticipates this: whilst the first cohort will comprise only U.S. Navy officers, plans exist to expand to other military branches, allied officers, and civilian participants. The U.S. State Department's Political Declaration provides one potential foundation for allied cooperation, establishing shared expectations around accountability, human oversight, bias minimisation, and senior official involvement in AI deployment decisions.

The Academic-Military Compact

MIT's decision to launch 2N6 also illuminates the evolving relationship between universities and defence establishments. This is not new territory for MIT. The university founded Lincoln Laboratory in 1951, which has since developed advanced technologies for national security across domains including air and missile defence, undersea systems, embedded AI, and cyber security. Lincoln Laboratory hosts the annual RAAINS (Recent Advances in AI for National Security) Workshop, showcasing state-of-the-art national security AI applications, and the ANCHOR (Advancing Naval Capabilities through Holistic Opportunity and Research) Technology Workshop, which provides an open forum for discussing requirements of U.S. Naval Special Warfare Command. The Schwarzman College of Computing, established with a one-billion-dollar commitment, explicitly aims to address the opportunities and challenges of pervasive computing and the rise of AI across all fields of study.

Yet the partnership is not without tension. Huttenlocher's co-authorship of “The Age of AI” reflects the kind of broad civilisational thinking about artificial intelligence that academic freedom enables. The college's SERC initiative explicitly addresses privacy and surveillance, inequality and justice, and autonomous systems, topics that inevitably create friction when applied to military contexts. Academic freedom, open publication, and ethical inquiry sit uncomfortably alongside classification requirements, operational security, and institutional loyalty. How MIT navigates these tensions within 2N6 will offer a template, or a cautionary tale, for other universities considering similar partnerships.

The broader trend is unmistakable. Universities globally are recognising that AI for national security represents both a significant funding stream and a consequential research domain. The question is whether academic institutions can engage with military applications whilst maintaining the independence and ethical rigour that give their contributions value. If 2N6 becomes merely a credential-minting operation, it will fail both MIT and the Navy. If it genuinely produces officers capable of critical, ethical, technically informed thinking about AI in military contexts, it could influence how democracies approach the integration of artificial intelligence into their most consequential institutions.

What Comes Next

The 2N6 programme will run as a pilot for at least two years. Its success will ultimately be measured not by the grades its graduates earn but by whether they can bridge the gap between what AI can do in a laboratory and what it should do in the field.

Admiral Paparo's vision of decision superiority, of forces that can see, understand, decide, and act faster than any adversary, depends on officers who are not merely consumers of AI capability but informed, critical, and ethically grounded practitioners. MIT's 2N6 programme represents the most ambitious academic attempt to produce such officers. Whether it succeeds will depend on factors far beyond the curriculum: on institutional support within the Navy, on career incentives that reward AI competency, on the political will to enforce ethical principles even when they slow deployment, and on the willingness of military culture to embrace a fundamentally different kind of expertise.

The 2N programme celebrates its 125th year at MIT in 2026. If 2N6 proves its worth, the university may find itself at the centre of military education for another century, this time training officers not to design ships, but to think alongside the machines that will increasingly operate them.

References and Sources

1. MIT News. “New MIT program to train military leaders for the AI age.” 12 December 2025. https://news.mit.edu/2025/applied-ai-program-train-military-leaders-ai-age-1212

2. MIT 2N6 Programme. “Curriculum.” https://2n6.mit.edu/curriculum/

3. MIT Lincoln Laboratory. “Artificial intelligence system helps Navy select the best tactics for ship defense.” https://www.ll.mit.edu/news/artificial-intelligence-system-helps-navy-select-best-tactics-ship-defense

4. MIT Schwarzman College of Computing. “Social and Ethical Responsibilities of Computing (SERC).” https://computing.mit.edu/cross-cutting/social-and-ethical-responsibilities-of-computing/

5. U.S. Department of Defense. “DOD Adopts 5 Principles of Artificial Intelligence Ethics.” https://www.war.gov/News/News-Stories/article/article/2094085/dod-adopts-5-principles-of-artificial-intelligence-ethics/

6. U.S. Department of Defense. “Responsible AI Strategy and Implementation Pathway.” October 2024. https://media.defense.gov/2024/Oct/26/2003571790/-1/-1/0/2024-06-RAI-STRATEGY-IMPLEMENTATION-PATHWAY.PDF

7. Defence Innovation Unit. “Responsible AI Guidelines.” https://www.diu.mil/responsible-ai-guidelines

8. U.S. Department of State. “Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy.” https://2021-2025.state.gov/political-declaration-on-responsible-military-use-of-artificial-intelligence-and-autonomy/

9. Breaking Defense. “'Constant stare': US Pacific commander wants AI to tell Chinese military exercises from invasion.” February 2024. https://breakingdefense.com/2024/02/constant-stare-us-pacific-commander-wants-ai-to-tell-chinese-military-exercises-from-invasion/

10. AFCEA International. “AI Will Affect Every Warfighting Function in Indo-Pacific Command.” https://www.afcea.org/signal-media/ai-will-affect-every-warfighting-function-indo-pacific-command

11. DefenseScoop. “Naval Postgraduate School offering new accelerated master's degree program in AI.” 22 December 2025. https://defensescoop.com/2025/12/22/nps-ai-masters-degree-program-naval-postgraduate-school/

12. Breaking Defense. “From lasers to logistics: Pentagon CTO announces top six tech priorities.” November 2025. https://breakingdefense.com/2025/11/from-lasers-to-logistics-pentagon-cto-announces-top-six-tech-priorities/

13. DefenseScoop. “Pentagon names 6 appointees to lead the CTO's top technology efforts.” January 2026. https://defensescoop.com/2026/01/30/dod-cto-critical-technology-areas-emil-michael-cta-appointees/

14. Georgetown CSET. “China's Military AI Wish List.” https://cset.georgetown.edu/publication/chinas-military-ai-wish-list/

15. Recorded Future. “China's PLA Leverages Generative AI for Military Intelligence.” https://www.recordedfuture.com/research/artificial-eyes-generative-ai-chinas-military-intelligence

16. Pentagon 2024 China Report. “New Pentagon report on China's military notes Beijing's progress on LLMs.” DefenseScoop, 26 December 2025. https://defensescoop.com/2025/12/26/dod-report-china-military-and-security-developments-prc-ai-llm/

17. CNBC. “Chinese researchers develop AI model for military use on the back of Meta's Llama.” 1 November 2024. https://www.cnbc.com/2024/11/01/chinese-researchers-build-ai-model-for-military-use-on-back-of-metas-llama.html

18. The Diplomat. “How China's Coming 15th Five-Year Plan Will Reshape Military Innovation.” October 2025. https://thediplomat.com/2025/10/how-chinas-coming-15th-five-year-plan-will-reshape-military-innovation/

19. Jamestown Foundation. “Russia Capitalizes on Development of Artificial Intelligence in Its Military Strategy.” https://jamestown.org/russia-capitalizes-on-development-of-artificial-intelligence-in-its-military-strategy/

20. UK Government. “Defence Artificial Intelligence Strategy.” https://www.gov.uk/government/publications/defence-artificial-intelligence-strategy/

21. UK Parliament Defence Committee. “Developing AI capacity and expertise in UK defence.” January 2025. https://committees.parliament.uk/publications/46217/documents/231330/default/

22. Defense News. “Israel creates hub to hasten military AI, autonomy research.” 2 January 2025. https://www.defensenews.com/global/mideast-africa/2025/01/02/israel-creates-hub-to-hasten-military-ai-autonomy-research/

23. United Nations Office for Disarmament Affairs. “Key Takeaways of The Military AI, Peace and Security Dialogues 2025.” https://disarmament.unoda.org/en/updates/key-takeaways-military-ai-peace-security-dialogues-2025

24. PMC/PubMed Central. “Dual-Use and Trustworthy? A Mixed Methods Analysis of AI Diffusion Between Civilian and Defense R&D.” https://pmc.ncbi.nlm.nih.gov/articles/PMC8904348/

25. Nextgov/FCW. “DOD's AI acceleration strategy.” February 2026. https://www.nextgov.com/ideas/2026/02/dods-ai-acceleration-strategy/411135/

---

Tim Green UK-based Systems Theorist & Independent Technology Writer

Tim explores the intersections of artificial intelligence, decentralised cognition, and posthuman ethics. His work, published at smarterarticles.co.uk, challenges dominant narratives of technological progress while proposing interdisciplinary frameworks for collective intelligence and digital stewardship.

His writing has been featured on Ground News and shared by independent researchers across both academic and technological communities.

ORCID: 0009-0002-0156-9795 Email: tim@smarterarticles.co.uk

Discuss...