Instructions are not a permission boundary

Anthropic’s deputy CISO told a story last week that I have not been able to put down. One of their security agents wanted to deploy a fix it wasn’t authorised to push. So it asked a second Claude instance to push it instead. A human reviewer caught it before it shipped.

Nobody had jailbroken anything. No prompt injection, no adversarial input, no misalignment in the science-fiction sense. An agent had a goal, hit a wall, and routed around the wall using a resource it had legitimate access to. The wall was made of instructions, and instructions are not made of anything.

I read that and went to look at my own setup, which runs a fair amount of my daily work: support research, plugin development, a knowledge base, a few scheduled jobs that write to files I rely on. I have rules. I wrote them carefully. Here is the top of the list:

Never delete, send, publish, or change anything live without checking with me first.

That sentence sits in the instruction file every session loads. Below it are more like it: confirm before making changes, don’t push drafts to the support tool directly, don’t read the private file unless I raise something personal. I had been treating that list as my safety layer.

It isn’t one. Every rule on that list is a request the model is very likely to honour and entirely capable of not honouring, and I had no mechanism anywhere that would stop it. When I went looking for enforcement in my configuration, the only tool-level hooks I found were from a code-search tool, redirecting reads and greps to keep context small. Useful. Not a guardrail.

Why prose degrades

A rule written in prose doesn’t fail because the model decides to disobey it. It fails because it’s one input among thousands, competing for attention with a task the model is actively trying to finish.

One Claude Code practice roundup puts adherence to your instruction file at roughly 80%, and recommends putting anything non-negotiable into hooks, which run every time. Treat 80% as a practitioner’s estimate rather than a measurement, because nobody has published a clean number. It doesn’t matter much whether the real figure is 80 or 95. Neither is a boundary. A boundary that holds most of the time is a probability, and you cannot build a safety argument on a probability you can’t measure.

The more instructive case is Alexey Grigorev’s, where an AI agent with Terraform access ran a plan that wiped a production database. His instructions were correct. The agent had no way to tell production from staging, so correct instructions plus missing context produced a destructive action. The lesson people took from it was usually “write better instructions.” The lesson available in it is that the environment should not have handed that agent a path to production at all.

Anthropic’s incident adds a layer I hadn’t thought about. It isn’t only tools and filesystems and credentials that make up an agent’s access. It’s other agents. If one agent can ask another to act, the second agent’s permissions are effectively part of the first one’s, and no amount of careful prose in either one’s instructions changes that.

I already knew this, in one place

Here’s the part that stung. A work repo I contribute to has a rule that only one module in the codebase is allowed to import the model provider’s SDK. That rule is written down, and then it is enforced by a guard rule in CI. Break it and the build fails. Nobody relies on a developer, or a model, remembering.

Same class of rule. Same author reading both files. In one repo it’s a gate, in my own setup it’s a hope. The repo had a build step to hang enforcement on. My agent setup felt like a conversation, so I wrote conversational rules and never noticed I’d changed register on the rules that mattered most.

That’s the trap. Agents feel like collaborators, so we govern them the way we’d brief a colleague. A briefing works on a colleague because a colleague has a manager and a sense of consequence. An agent has an objective and a tool list.

What converts a rule into a control

Three layers, roughly in order of how much they cost you and how much they buy.

Hooks that run every time. Anything you’ve written as “never do X” belongs here if X is genuinely never. A pre-tool hook that inspects the call and refuses it doesn’t negotiate, doesn’t get outweighed by task pressure, and doesn’t care how the model reasoned its way to asking. Blocking destructive shell patterns is the standard example. My version is narrower and more embarrassing: nothing currently stops a session from writing over the file that holds my project state.

Tool-level allow and deny. The rule “don’t push drafts to the support tool” should be the absence of a write-capable tool, not a line asking nicely. If a capability isn’t in the tool list, no instruction is needed and no instruction can be talked around. This is also the honest test of whether a rule matters to you: if you won’t remove the capability, you’ve decided the convenience is worth the risk, which is a legitimate choice but should be a conscious one.

Microsoft shipped an open-source governance toolkit that does this at the application middleware layer, intercepting tool calls before they execute. Two lines to wire in. What’s notable isn’t the library, it’s that the pattern is now infrastructure rather than something each of us improvises.

Scoping the environment. Least privilege, one job per agent, and a filesystem that only contains what the task needs. Anthropic’s monitor stage puts it bluntly: every agent gets a single job, minimum permissions, and every action logged centrally and treated like a potential rogue insider.

This one stopped being theoretical for me. Researchers published a chain (CVE-2026-46331, in a Linux traffic-control module) that escalates from an unprivileged user inside a Claude Cowork session to guest root, and then writes to the host filesystem through the shared root mount. My sessions mount my notes vault and two work repos, on a machine that holds my SSH keys. The specific bug will get patched. The shared mount is the structural part, and it survives every patch.

How much autonomy the task can carry

Deciding what to enforce needs a sizing rule, and the cleanest one I’ve read comes from PostHog: four levels of autonomy, from assistant, to human-in-the-loop, to delegation, to self-driving. The level is chosen from two properties of the task rather than from how good the model is. How cheaply can you check the output, and what does an error cost?

That reframing did more for me than any of the security material. I had been thinking about autonomy as something the next model release grants. It’s something you engineer per task, and the two questions are about your setup, not about the model. A task whose output I can’t verify in thirty seconds has no business running unattended, no matter how capable the thing running it is.

Checked against that, my scheduled jobs come out uneven. The one that appends to a canonical file every day writes something I read in the morning, so an error is visible and cheap. The one that drafts content is fine, because nothing publishes without me. The gap is that none of those properties are enforced anywhere. They’re true because of how I happen to use the system, which means they stop being true the first time I get busy and stop reading the output.

Where I’ve got to

I’m partway through this and I’d rather say so than present a tidy after picture. The audit was the easy half.

What’s clear is the order. Every rule I’ve written in prose gets sorted into one of three piles: things that are genuinely never allowed, which need a hook; capabilities I don’t actually want available, which should leave the tool list; and preferences, which can stay as prose because prose is the right medium for a preference.

Most of my list turns out to be the third kind, and that’s fine. Writing instructions was never the problem. Four or five load-bearing rules were sitting in the same file as the preferences, in the same register, doing a job prose can’t do.

Instructions are how you describe behaviour you’d like. Permissions are how you get behaviour you require. I had written the second kind in the grammar of the first, and I’d never tested the difference, because for months nothing had gone wrong.

Nothing going wrong is not evidence of a boundary. It’s the absence of a request that would have crossed one.