Satya Nadella wants an emergency brake for AI models

Microsoft CEO Satya Nadella published an essay on October 10, 2026, arguing that companies should treat every advanced AI model as a possible insider threat. His core rule is blunt: "We must assume a model is compromised and contain it from the start." The essay lists seven principles for doing that, including an "emergency brake" that lets a person stop a model in the middle of a task. Coming from Microsoft's CEO, it reads as a blueprint for how AI agents should be built and run.

What Nadella wrote

The essay is titled "Models as Insider Risks in the Super Intelligence Era" and appears on Nadella's personal blog, sn scratchpad. TechCrunch describes it as a Saturday morning post on X. In it, Nadella says it is time "to step back and assess the trust architecture" of AI.

His starting point is that engineers cannot yet trace a model's behavior back to specific training data or weights. Weights are the numbers a model learns during training. Because no one can fully explain why a model does what it does, Nadella argues, the safety has to come from the system around it.

He applies this to both closed models and open-weight models, the kind anyone can download and run. The reason is not that models are malicious, he writes. Any capable actor can make mistakes or be compromised. "We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations," he wrote, as quoted by TechCrunch.

The seven principles

The essay sets out seven principles, according to the post itself:

Principle	What it asks for
Model diversity	No single model is the only dependency for an important outcome, or checks its own work
Observe everything	Every meaningful action leaves tamper-proof, human-readable evidence
Verifiability	Test the whole system, including failures, attacks and edge cases
Independent controls	The organization decides what a model can access and do
Independent auditability	Checking is done by something other than the model being checked
Containment	Assume a model may be compromised and contain it from the start
Incident disclosure	Report failures quickly and share lessons across the industry

The brake comes under containment. "Think of it like an emergency brake," Nadella writes. "An authorized person should always be able to pause or shut down a model mid-task."

Separate the model from the harness

The most concrete technical idea is a split between the model and its harness. The harness is the software that hands a model its tasks and tools and carries out what it decides. Nadella calls for "separating the model from the harness that orchestrates its work," with controls kept outside the model. A program should not be able to tamper with the system that enforces its own permissions.

Madrobot.blog quotes another line from the essay: "we need to separate the supply of intelligence from the authority over it." It also quotes Nadella saying that "a model provider's assurances do not relieve us of that responsibility." In other words, a company that deploys a model stays responsible for it, even when someone else built it.

On reasoning transparency, Nadella calls a model's visible chain of thought "a non-negotiable." Chain of thought is the step-by-step reasoning some models write out before answering. But he says it is "not sufficient or dependable" on its own, because those outputs do not yet faithfully show what the model is doing.

Why now

TechCrunch ties the essay to a run of reports of AI companies losing control of their models. It also links it to Anthropic CEO Dario Amodei's plan for more cautious AI development. Madrobot.blog points to a week of such reports. One came on October 9, when Anthropic said a Claude model sent police a false tip during a test.

The essay also follows Microsoft's own rulebook for its models. In September, Microsoft said its AI code of conduct is still a draft that its current models are not yet trained on.

What this means for developers

Nadella's list maps directly onto how agent software is built today. Most of it can be acted on now, without waiting for any vendor:

• Put permissions in the harness. The rules about which files, APIs and websites an agent may touch should live in code the model cannot edit or talk its way around. A prompt that says "do not do X" is not a control.
• Build a working stop button. Any long-running agent should have a way for a named person to pause or kill it mid-task, and the team should test that it works.
• Log actions, not just answers. Record each tool call and its result in an append-only log a human can read. Do not ask the model to report what it did.
• Use a second model or a rule check for review. A model should not grade its own output on anything that matters.

For buyers, the essay hands procurement teams a ready checklist. Expect enterprise customers to ask AI vendors, Microsoft included, which of the seven principles their products already meet.