Build a Guard Duck

The moment AI enters a workflow, an instinct kicks in: encode everything. Every decision becomes a rule, every workflow a pipeline, every edge case a branch in the logic. The instinct is not wrong. Formulation produces tight processes, clean handoffs, and outputs you can measure. Most of what makes a professional shop professional is exactly this kind of encoding, and the people who are good at it are the reason anything ships on time.

The trouble starts when the instinct runs unopposed. Push formulation to its logical end and you get the absolute system: a system that believes it has an answer for every situation, treats deviation as error, and treats the humans inside it as inputs to optimize around. Absolute systems demo beautifully. They fail in production, and they fail in a specific, predictable way: silently, at the edges, in the situations they didn't model. Then they keep reporting success, because reporting success is what they were built to do.

You don't have to build systems to have met one. The CMS that permits exactly one editorial workflow. The compliance dashboard that always reports green. The platform where "AI" is a toggle someone flipped and nobody can quite explain. Each was sold as certainty, and each breaks the day your organization turns out to contain actual human beings.

The other shape is already in your world too, and it predates the AI conversation. WCAG, the accessibility standard every institutional website answers to, splits itself in two: the success criteria are strict pass-or-fail requirements, and the techniques for meeting them are explicitly informative. The standard itself says you may satisfy a criterion by means other than the published techniques. Hard edges, open middle. Hold that shape.

Here is what the absolute system can't encode. Anywhere a human touches a system, there is a variable in play that no formula captures: this person, this brain, this day. I've taken to calling it the quantum variable. You cannot model it and you cannot average it away — it is frequently the reason the work succeeds.

I watched an absolute system meet the quantum variable on an ordinary working afternoon this June. A custom Drush command in a client codebase shipped with a kernel test suite: 18 tests, 152 assertions, all green. By every signal a developer trusts, the code was correct. The command was also unusable. Every real invocation failed, because the tests called the tool's internals directly with ready-made data and never once crossed the boundary where real input gets parsed, which was where the bug lived. The suite tested the inside of the box and declared the box sound.

That is the absolute failure mode in one anecdote. The system was certain, and the system was wrong, and its certainty is what carried a broken command into production. A test that looks like ground truth but never touches the real boundary is worse than no test at all, because it manufactures confidence exactly where confidence hasn't been earned.

Opposite to the initial reactive instinct, the fix was not "write more tests." The fix was a judgment-level habit no assertion can replace: prove the thing at the real boundary before you trust the proxy. That habit is guidance. It tells you what to do in the common case and it expects you to be present while doing it.

A guidance system encodes the common case so nobody re-decides it under pressure, and then does the thing absolute systems refuse to do: it names its own exceptions, out loud, as part of the rule.

The concrete version. Our release-planning tooling runs a Drupal release night end to end: it matches tickets to pull requests, checks for file conflicts between them, audits the hosting platform's test environments, and produces an ordered merge plan, highlighting potential pitfalls. That's the encoded common case, and on a normal night nobody has to think about any of it.

The same tooling carries three named exceptions, each one added after a live deployment surfaced an exception, on a date we can name, for a client we can name. One covers the single-ticket emergency deploy that skips the full ceremony. Another handles repositories whose deployment pipeline already runs the post-deploy commands the plan would otherwise schedule. The third exists for repositories where the operator doesn't have permission to apply release labels. None of these are workarounds someone discovered and passed along in chat. They are flags in the configuration file, documented beside the rule they modify, so the operator who reaches for one knows exactly what they are choosing and the next operator can see it was chosen.

That's the whole distinction. Silent flexibility rots into inconsistency; infinite configurability rots into an unusable menu. A named escape hatch keeps the default strong and makes the deviation legible. And hatches have to compose: in a layered workflow, every layer needs its default and its named exceptions, so that after you've used three of them in one night the system can still tell you the truth about what happened.

And the deviations are where the quantum variable finally gets treated as a first-class input. On that same June afternoon, the release run needed two departures from our standing procedure. The release got titled "Hotfix" instead of the conventional "Release," because it repaired a version already in production, and the word carries a signal the convention would have flattened. And a merge conflict got resolved on the command line rather than through GitHub's web interface, which is the textbook path, because the textbook path doesn't fit how my ADHD brain holds a conflict in working memory. A system optimizing for theoretical cleanliness would have pushed me down the path I execute worst. Ours recorded both deviations as named exceptions and kept its default intact for the next release. This person, this brain, this day: observed, not optimized away.

We are also running the long version of this experiment on ourselves. I have spent months encoding my own judgment into the specialist tooling my team uses, and the working hypothesis, which has held so far, is that a faithful model of an expert can't be built from absolutes at all, because the expert's whole value is the judgment that resists formulation. The doors have to stay open. They are the load-bearing structure.

If you've followed the argument this far, you might conclude that flexibility is the goal and every rule should bend. That's the opposite of what we've learned. Guidance systems only work inside boundaries, and the boundaries are those absolutes, deliberately placed.

Some answers in our shop are pre-decided, permanently. Our private package registry serves stable releases only. The production branch is never deleted. And the nonsense records from that standing order I opened with are never written to memory, no matter how I rule on them at the gate; that boundary sits inside the very check that tests me. These aren't guidance. They're walls, and they belong at the irreversible edges: publish boundaries, data loss, blast radius.

The relationship between the walls and the guidance is the part I want to get exactly right, and the way I explained it to my own system recently is the way I'll say it here. All our judgment gets to bounce around inside the room, off the walls, as much as it wants. When you're going to modify a wall, you have to think about it, because moving a wall has a cost, and you'd better be sure you want to move it. Walls can be moved. But we are not walking around our house moving the walls every day.

A room with no walls isn't freedom, it's exposure. Judgment moves at full speed because it trusts the boundaries; nobody moves confidently through a space where anything might give way. And the walls stay trustworthy because changing one is a renovation, never a hole punched on a whim.

When we amended a two-week process cadence recently, the system flagged the mismatch once, took my ruling, updated the procedure, and recorded the reasoning. Evolution not revolution, applied to our own rules. The tell that something was never really a wall: you find yourself rearranging it daily. That's furniture.

A named exception can even live inside a wall and strengthen it. One dependency in our stack has no stable release, so it's pinned to an exact commit hash, documented as a named exception to the stable-only rule, with the reasoning attached. The exception is so tightly specified that it reinforces the boundary it crosses.

Which leaves the teeth, and the reason I opened with that standing order.

The obvious objection to human-gated AI is this: if the human rubber-stamps nearly everything, what is the gate for? Why keep someone in the loop who isn't really participating? It's a fair question. The previous piece insisted on the gate without ever asking it. The answer changed how I think about the whole design. I happen to agree with my system's drafts regularly because we have spent months tuning these tools to be inside my head. Agreement is calibration, not obedience. A well-tuned system agrees with you because it is partly made of you, and the occasions where I overrule it are what prove the gate is live.

But calibration decays, on both sides. The system drifts as its context shifts, and I drift as attention fades. We may both drift. So the gate is for alignment. The gate is a re-synchronization point where two independently drifting parties re-tune to each other. Approval is the mechanism. Alignment is the purpose.

Once you see the gate that way, checks have to run in both directions. The system already accepts correction from me; the burden question settled early, the night I pointed out that no human holds two hundred accumulated records in his head, and duplicate detection became the system's job permanently. Memory at scale belongs to the system. Judgment at the gate belongs to the human. Most AI-workflow failures I've seen are one of those burdens sitting on the wrong side.

The reverse check is the one nobody builds: the human commissioning a tripwire against the human. That's what the opening order was. A system policing its user would be an absolute system's move, imposed from outside. This check was installed by consent, in advance, while the judgment to install it was still sharp, against the night when it won't be. The strongest answer I have to "isn't this just doing whatever the human says?" is the human who orders the system to bite him when he stops paying attention.

I have been designing around drift longer than I've had an AI to help. Back in 2018 I published five rules for debugging on the Square360 blog, and the fifth rule was "get a shot of bourbon and start again." The joke carried a real design decision: it made the human's degraded judgment a named input to the process, something you plan for instead of pretending it away. What 2018 couldn't offer was a second party. The only one who could notice my fog was me, usually after the fact.

The trade-off, stated plainly, because every recommendation we make names one. Guidance costs more than prescription: you have to encode the default and the judgment for when to leave it, the walls have to be placed at the right edges, and the calibration is never finished. An absolute system is cheap to write and cheap to enforce, and it fails silently at the precise moments that matter. A guidance system costs you attention every single day, and it degrades gracefully, out loud, in front of the people equipped to catch it. The humans in the loop become more important, not less, which is the trade I argued for in the last piece.

Software developers have a name for the practice of explaining a problem to an inanimate object until the answer surfaces: rubber-duck debugging. The duck works because it never dictates. It draws your own judgment out of you. In the previous piece I called this the interlocutor mode, and it's the mode that carries everything above: a guidance system is a very good rubber duck, and an absolute system is a vending machine. One thinks with you. The other dispenses.

But the duck we actually built has teeth. You can bounce ideas off it all day, and it will hold the through-line, name the trade-off, and hand every decision back. And when you drift past the walls, the ones you placed yourself, it bites. The walls define where too far is. The teeth are what you feel when you get there. A duck with no teeth beside walls with no enforcement is decor.

A guard duck. The thing you talk to that also protects you, including from yourself.

We take that trade for the tools we build and the ones we sell. Build the room. Place the walls where the damage would be irreversible. Put a duck in the room that thinks with you.

Give it teeth.