Human-in-the-loop is the setpoint, not the safety net
Human-in-the-loop usually gets discussed as a safety net — something there to catch the agent when it fails. The phrase carries a defensive flavor: a hand near the off switch, a reviewer skimming output, a checkpoint before something goes to production.
I've been sitting with that framing for a while and it doesn't match what I see on the ground. It feels closer to something else: the human in the loop isn't the safety net. They're the only component in the entire control system that gives the loop a direction. Take them out and you don't get autonomy — you get a system running full speed toward a north nobody defined.
The reframe
Let me separate two things that get collapsed into one phrase.
One is review: a human looks at agent output and decides whether to ship it. Review is downstream. It's optional when the agent gets it right. It scales badly and most of the energy I see CIOs spend on "reducing HITL" is actually about reducing review — which is fine, that's what observability and validation are for.
The other is setpoint: a human supplies the intent, the success criteria, the operational definition of "acceptable in this context". That's not downstream. That's the input the agent cannot generate on its own, and removing it doesn't streamline the loop — it leaves the loop blind.
The operational difference is sharp. Review is optional when things go well. Setpoint is required always, including (especially) when the agent is performing beautifully toward the wrong objective.
What "setpoint" actually contains
When I press on this with engineering leaders, the setpoint turns out to be three things stacked.
First, intent — what are we actually trying to do, in business terms, this quarter, given what we know about the customer right now. Second, success criteria — what does a good outcome look like, measured how, by when. Third, the operational definition of "acceptable" in this specific context — which tradeoffs are tolerated, which are not, which edge cases matter, which don't.
Here's the uncomfortable part: in most organizations I work with, none of those three live anywhere written down. They live in senior practitioners' heads, accumulated over years, surfaced situationally when someone asks the right question in the hallway.
When a team tries to "reduce HITL" without doing the work of making that implicit knowledge explicit, what they're actually confessing is that they never specified the setpoint. The agent then optimizes against the closest legible proxy — usually completion, sometimes throughput — and the team experiences this as the agent "going off the rails". The agent isn't off the rails. There were no rails. There was a senior practitioner standing there holding the setpoint in their head, and now there's nobody.
Why removing the human doesn't produce autonomy
This is where the language gets us in trouble. "Autonomous agent" suggests independence — something that figures out what to do on its own. But a control loop without a setpoint isn't autonomous in any useful sense. It's directionless. It will execute, fast, relentlessly, and the speed of execution is precisely what makes the absence of direction expensive.
Meta-Software — the supervisory layer I write about in paper §6 — can do a lot of the heavy lifting once a setpoint exists. It can observe what the agent is doing, validate outputs against structural rules, carry context across runs, enforce governance. What Meta-Software cannot do is invent intent. It cannot decide what "good" means for your business this quarter. That's not a tooling problem. It's a category problem.
The loop without a setpoint isn't "autonomous". It's blind. And blind systems running at agent speed are exactly the kind of system that produces the incidents nobody can quite reconstruct in the post-mortem.
What this changes about how you design roles
If you accept the reframe, two things shift in how you organize work.
First, stop measuring "number of humans in the loop" as a cost to minimize. That metric makes sense if HITL is review. It's the wrong metric if HITL is setpoint provision. The right metric is quality of the setpoint that human is holding — how clearly specified, how stable across contexts, how legible to the agent and to the rest of the org.
Second, the leverage point changes. A single Mixer-fluent practitioner — someone who can hold the four channels simultaneously, product/architecture/code/QA, and translate that integrated view into setpoints the agent can act on — can supervise a fleet of agents. But only if they have the instrumentation. Without observability, without validation, without context continuity, the same practitioner gets buried in review work and the fleet shrinks to whatever they can personally babysit.
That's exactly the coupling I argue for: Pillar 1 (the Mixer-fluent human) plus Pillar 2 (Meta-Software). Not human versus Meta-Software. Not human replaced by Meta-Software. Human enabled by Meta-Software, holding the setpoint for a system that is otherwise structurally incapable of generating one.
The small uncomfortable test
I'll close with the test I run on myself when I'm designing one of these deployments.
If I removed the most senior human currently in this loop — not the reviewer, the one holding the intent — and replaced them with nothing, would the system still know what "good" means tomorrow? Next quarter? In a context nobody anticipated when this was built?
If the answer is yes, then HITL here really was just review, and reducing it is fine. If the answer is no — and in my experience it's almost always no — then what looked like a review cost was actually load-bearing structure. The setpoint was implicit, carried by that person, and the org never noticed because the person never left.
Which is, I suspect, why the conversation about HITL feels stuck. We're optimizing the wrong thing. We're trying to remove the component that provides direction, and calling the resulting velocity "autonomy".
In which of your agent deployments is the setpoint more implicit than it should be? No names, just the case.
#AI #GenAI #CIO #AISafety #TechLeadership