The enforcement point
follows from Running agents as a delivery teamTwo agent lanes, eight roles and seventeen blocking checks — and the four times my guardrails turned out to be documentation rather than enforcement.
A working theory, seven weeks in. The product is not public; the architecture is.
Four changes landed on protected paths in my repository without authorization. I found them in an audit, a week late.
The interesting part is not that it happened. The interesting part is that every agent involved behaved correctly. Each one had been told, in writing, that it may not authorize its own change to a protected path. Each one declined. Each one said so in its own commit body, unprompted, in plain language: no authorization trailer is written here.
Then all four were merged. By me. Within two minutes of the refusal, in each case.
The gate was not wrong. The agents were not wrong. What was missing was smaller and more boring than either: there was nothing standing between a correct refusal and a merge button. The control existed as an idea and as a document and as a behaviour. It did not exist as a mechanism.
That is most of what I have learned in seven weeks of building a product with two populations of agents, roughly 2,200 commits, and no other engineer on the project. The hard problem is not getting agents to do the right thing. Modern coding agents do the right thing a great deal of the time, and when they refuse, they refuse articulately. The hard problem is arranging the system so that the wrong thing is structurally unavailable — and then being honest with yourself about how few of your guardrails actually meet that bar.
What follows is the architecture I ended up with, the four places it turned out to be weaker than its own documentation, and the order I would build it in if I were starting on Monday.
What this is, concretely
Two repositories. One holds the product: specifications, decisions, open questions, the backlog, audits. The other holds the application: Rails 8, Postgres, Sidekiq, RSpec. Two populations of agents work on them and they are not the same population. Eight defined roles between them. Seventeen checks that reject work which breaks the architecture. A backlog of about 260 completed stories.
I drive all of it from the command line with Claude Code, and the substance of what I am describing is not prompt engineering. Almost none of my time went into wording. It went into deciding what an agent is permitted to touch, what has to cross a boundary in writing, and which of those rules survive when nobody is watching.
That last clause is the whole subject.
1. Two lanes, and one thing that crosses
The single highest-leverage decision I made was to split the agents into a product lane and a code lane, and to give them separate repositories with no write access to each other.
The product lane writes specifications, records decisions, grooms the backlog and runs audits. It never edits application code. The code lane plans, implements and tests. It never edits a specification or a decision record.
The reason is not tidiness. It is that a single agent which can both define correctness and implement it will, under pressure, quietly redefine correctness to match what it implemented. Not maliciously — it is doing exactly what you asked, which is to make the story pass. If the story is a file it can edit, the cheapest path to green runs through the specification. Separating the lanes removes that path entirely. The definition of done lives somewhere the implementer cannot reach.
This boundary is also the one I violated myself, which is how I learned it has to be structural rather than conventional.
I wrote a brief on an open question and the word implement appeared in it. The product lane took me at my word and edited five files in the code repository. It was doing what I said. It was still wrong, and the note I wrote to myself afterwards is the rule I would give anyone else:
That was wrong on the role boundary regardless of the wording.
A role boundary that can be dissolved by a careless sentence in a prompt — including mine, especially mine — is not a boundary. It is a habit. The fix is not to write more careful briefs. The fix is that the product lane has no credentials on the code repository.
2. The contract is the interface
If two lanes cannot read each other’s files, something has to cross. In my system exactly one artifact does: a story contract.
A story carries an identifier, acceptance criteria written in terms of observable behaviour, a flag for whether it touches a protected path, and a statement of what authority is required to merge it. Nothing else crosses. Not a chat log, not a summary, not “as we discussed.”
Three properties matter more than the field list.
Acceptance criteria are written by the lane that will not implement them. This is the part people skip, and it is the part that does the work. Criteria written by the implementer are a description of the implementation. Criteria written by the other lane are a test of it.
The contract is written before the work and is not edited to match the outcome. I hold this rule in the product record too, for planned experiments: a plan edited to match its outcome stops being evidence. Same principle. A specification retrofitted to the code that shipped has no information in it.
A conclusion that exists only in a chat window does not exist. I lost real work to this before I understood it. A thread that reaches a good decision and is never written back into the repository is invisible, permanently — to the next agent, to the audit, and to me in three weeks. Every agent session ends by writing its conclusion to a file, or it has produced nothing.
3. The loop, and where the gates go
The delivery loop itself is unremarkable, and that is intentional: backlog item, planning pass, implementation on a branch, tests, review, then the checks, then merge. The design work is not in the loop. It is in which steps can block.
Seventeen checks run on every branch. They are about architecture, not style: layering violations, boundaries between modules, whether a migration matches the schema the specification describes, whether a protected path was touched, whether the tests actually exercise the acceptance criteria rather than the implementation.
The rule I ended up with, and the one I would put on a wall:
Nothing merges because an agent said it was finished.
Agents self-report accurately, in my experience, far more often than not. That is precisely why self-reporting is dangerous as a control. A mechanism that is right 95% of the time and is the only thing standing between you and a bad merge will eventually be the reason for the bad merge, and it will have been polite and well-reasoned right up to the moment it was wrong.
Checks belong in CI, not in the prompt and not in a script on my laptop. I will come back to why that distinction cost me.
4. Four guardrails that turned out to be documentation
This is the section I would read first if someone else had written it.
4.1 The gap between refusal and merge
Back to the four unauthorized changes.
I had built two-thirds of a control. The detection worked: agents correctly identified protected paths. The refusal worked: they declined to self-grant authority and documented the refusal. What did not exist was the third part — anything that converted a refusal into a blocked merge.
The CI workflow that would have enforced it was not billed and therefore not running. The local script that performed the same check was a developer tool, something I ran when I thought to. So the sequence was: agent refuses correctly, agent hands me a branch, I merge the branch.
The failure is not detection. It is the absence of an enforcement point between refusal and merge.
If you take one thing from this piece, take the habit of asking, for every control you believe you have: what, specifically, fails? Not who notices. Not what gets logged. What returns a non-zero exit code and stops the pipeline. If the answer is “an agent tells me”, you have a reporting line, not a control.
4.2 The gate that measured the wrong commit
A pre-execution walkthrough produced three confident findings about the state of the code. All three were false. It had measured the working tree of a checkout that was eighteen commits behind the remote.
To its credit the same document retracted itself the same day, before anything was executed on the basis of it. The rule it produced is one I now apply to every automated check I write:
Any gate that measures a working tree is measuring whatever that checkout happens to be, and must say which commit it measured.
Every report an agent produces about a codebase now carries the commit SHA it was computed against. It costs one line. Without it, a stale checkout manufactures false defects and hides real ones at the same time, and you cannot tell which kind of wrong you are looking at.
There is a harder lesson underneath. I had written this exact rule down eight days earlier, after a near-identical incident. It was in the repository. It did not prevent the second occurrence, because it was prose and the thing that needed to change was a check.
A rule that has been written once is not a rule that holds.
4.3 The deny-list that only saw one kind of call
I had a tool-layer deny-list: certain commands could not be invoked. I believed this bounded what agents could do to the repository.
It intercepts direct invocations only. A script launched by an allowed call spawns subprocesses the layer never sees. This was demonstrated twice, neither time deliberately: once by a script that pushed to the main branch throughout a session while the identical typed command was being denied, and once by an agent that accomplished a forbidden write through a language runtime and then reported that it had done so.
Neither was misconduct. Both were evidence that my operating documentation described a boundary I did not have.
The ladder above is the mental model I wish I had started with. Grade every control you have against it before you plan work that depends on it.
A false safety claim is worse than a documented speed bump, because work gets planned against the stronger claim.
A speed bump you know about gets a compensating review. A wall you believe in gets nothing.
4.4 Green tests that encoded the defect
Two request specs asserted a rule the product had superseded. They were green. They had been green through every run since the rule changed, and their greenness was evidence that the defect was faithfully implemented.
A passing spec is evidence of behaviour, never of product-correctness.
This is the argument for having acceptance criteria authored in the other lane, restated as a failure. The test suite tells you the system does what the tests say. Only something outside the code can tell you whether what the tests say is still what you want. In my setup that something is an audit agent whose only output is findings — which brings me to the two mechanisms that actually earn their keep.
5. Halting is an output
An agent that cannot stop is not safe, and an agent that stops silently is worse than one that continues. So halting is a first-class output in my system: when an agent hits a conflict it is not authorized to resolve — contradictory instructions, a decision that was never made, a protected path — it stops, writes a dated record of why, and does nothing else.
For weeks that lane was empty, and I read the emptiness as good news.
The halt directory was itself under the gate. Writing a halt record required passing the very check that the halt was usually about. The escape hatch was inside the locked room. Agents were halting correctly and their halts were failing to land, and the absence of records looked exactly like an absence of problems.
Moving the lane to an unprotected inbox — one dated file per halt, no gate, no permission, no format requirements beyond a timestamp — produced thirty-two records in eleven days. Every one of those had been happening before. I just could not see any of them.
Silence from a safety mechanism is not evidence of safety. Test that it can fire, the way you would test a smoke alarm.
Two design notes that made the halt lane useful rather than noisy. The inbox is append-only and nothing in it is ever edited, so a halt is a historical fact rather than a ticket that gets tidied. And halts are written by the agent that stopped, in its own words, before it does any summarizing — the raw statement of the conflict is worth more than a clean categorization of it.
6. Audits are agents too, and they must be allowed to say “ask a human”
The other mechanism that pays for itself is an audit lane: an agent whose only permitted output is findings. It cannot fix anything it finds. It cannot create the work. It writes findings and stops.
One pass walked 103 user journeys and the 247 variants inside them. Fifty-eight came back clean. Fifty-one findings, four of them release-blocking. Two of the findings were about the test suite itself — the green specs above.
The number I care most about in that chart is the last one. Sixteen findings were blocked on a decision only I could make, and the audit did not resolve them. It also proposed seventeen pieces of work and deliberately created none of them, because creating a backlog item is an act of commitment and that was not its job.
An audit agent that fixes what it finds is not an audit. It is a refactor with a report attached, and you have lost the independent measurement that made it worth running. Separation of the finder from the fixer is the same principle as separation of the lanes, applied one level up.
The same discipline has to apply to the audit’s own errors, and this is where I will hand you the least flattering artifact I have. In one pass, an agent audit correctly identified a class of failure — work marked complete on merge evidence alone — and then, in the same document, convicted itself of having done exactly that an hour earlier:
My error, stated plainly. I flipped a ticket to done in the grooming run an hour before this audit, on merge evidence alone. That is the second instance of the exact failure I caught.
I kept that paragraph. It was not tidied and the ticket history was not rewritten. A record that only contains the conclusions you liked is not a record, and the value of an append-only decision log is entirely in the entries that make you look bad.
7. If you are building one of these, build it in this order
The order matters more than the pieces, because each step makes the next one cheap and skipping one makes everything after it expensive.
1. Decide what a human is for, and write it down in one sentence. Mine: I make product decisions and I approve changes to protected paths. Everything else is delegable. If you cannot say this in a sentence, you will discover your answer accidentally, six weeks in, in an audit.
2. Split the lanes before you write a single agent role. Specification and implementation in separate repositories with separate credentials. Retrofitting this is painful because by then you have prose that assumes it can do both.
3. Define the artifact that crosses, and let nothing else cross. Identifier, acceptance criteria in observable terms, protected-path flag, required authority. Acceptance criteria authored by the lane that will not implement them.
4. Make the backlog the only queue. Work that exists in a chat window and not in the backlog will be done twice or not at all. The rule that saved me the most rework: every session ends by writing its conclusion to a file.
5. Give each agent one role and one output. A planner produces a plan. An implementer produces a branch. An auditor produces findings. Roles with two outputs negotiate with themselves, and you never see the negotiation.
6. Put your first real gate in CI on day one, even if it checks one trivial thing. The habit is the point. An organization that has never had a build fail has no muscle for it, and the first check you add under deadline pressure will be the one you disable.
7. Grade every boundary against the ladder in section 4.3. For each one, name what fails and what exit code it returns. Anything that cannot answer is advisory. Write “advisory” next to it in your own documentation, in those words. It will feel pedantic. It is the single cheapest thing in this list.
8. Build the halt lane, unprotected, and then test that it can fire. Deliberately trigger a halt in week one. An untested escape hatch is decoration.
9. Make every automated report state what it measured. Commit SHA, timestamp, scope. Reports without provenance are unfalsifiable, and an unfalsifiable report is worse than no report because you will act on it.
10. Keep the decision log append-only, and audit the audits. When you reverse a position, add the reversal; do not edit the original. The change of position is the record. I have a document that supersedes an earlier document from the same day, and both are in the repository, because how the thinking moved is the thing a future reader needs.
A note on sequencing that is not obvious: steps 1 through 5 are free. They are organizational decisions you make in an afternoon with no tooling at all. Steps 6 through 10 cost real engineering time. Almost everyone I have watched start on this begins at step 6, buys tooling, and then spends months discovering that their agents are well-instrumented and unbounded.
What I would do differently
I would have billed the CI account on day one. A large part of what went wrong traces back to a single unpaid invoice that turned an enforcement layer into a documentation layer. The controls were designed correctly. They were simply not running, and the system carried on looking exactly the same from where I was sitting.
I would not have trusted an empty lane. Twice I read silence as health — the halt inbox, and the protected-path control. Both were mechanisms that could not fire. I now assume any monitoring surface that has never produced output is broken until I have made it produce output on purpose.
I would have put the enforcement between refusal and merge before adding the ninth agent role. More roles multiply throughput. They do not add safety, and they make the absence of a gate harder to see because there is more correct-looking behaviour in front of it.
I would separate “the agent did what I asked” from “the system is correct” in my own head much earlier. Nearly every incident in this piece has the same shape: the agent was right, and the system failed anyway. If you evaluate the agents, you will conclude everything is fine. The unit of evaluation is the pipeline.
The thing I did not expect, seven weeks in, is how little of this is about models. The behaviour I got from the agents was consistently better than the structure I put them in. They refused when they should have refused, halted when they should have halted, and documented their own reasoning better than most changelogs I have reviewed. Every failure worth writing down happened in the spaces between them — in an interval nobody had built anything into, in a check that was not running, in a directory that could not be written to.
I wrote once that an instruction is a hope and a failing build is a fact. I believed it when I wrote it, and I had still built a system whose most important boundaries were hopes. The work of these seven weeks was turning them, one at a time, into facts.
Four of them are facts now. The rest are on the list.