Every team selling AI agents says there's a human in the loop. Ask what that human is there to do. Only one of the three answers should worry you.
A supervisor on the floor presses the microphone button and nothing happens.
Our agents traced the cause and stopped. The fix was two lines of text for the user. They could have drafted those words themselves, but what a product says to its users is a decision we keep for people. So they laid out the options, recommended none, and waited.
The agents' note on the task, once a person had answered: "This interruption was not redundant."
That is a human in the loop at work. The phrase usually means less. "There's a human in the loop": you've heard it from every team that sells you AI agents, and you've probably said it yourself when a board member asked about risk. It promises a person somewhere in the process, and doesn't say what that person is doing. Researchers at Swansea and Bristol universities call this humanwashing: using a person's presence to imply a safety the system doesn't have.
The short version. A person inside an AI agent loop does one of three jobs: checking the work against what was already agreed, deciding what nobody has decided yet, or owning what ships. The technical companion article has the full data; it calls the first two verification and judgment, and treats sign-offs as ownership.
What is a human in the loop actually doing?
Think of a flight. The flight computers compare the plane's position to the flight plan constantly and correct course without being asked: that's checking. When the plan stops fitting, a storm ahead or a closed runway, someone decides what to do instead: that's judgment. And the captain is accountable for the flight: that's owning. Nobody pays a pilot to hold the controls for the whole flight. They are paid for the moments the autopilot can't handle.
Software has the same three jobs, at every step:
Notice the empty cells. Checking and deciding happen at every step; owning sits at the gates: approving work to start, signing off what ships, and answering when something goes wrong.
Most teams don't separate these. All three go to the same people, in the same meetings, under the same word: review. That's why "human in the loop" sounds like one job. It's three, heading in different directions: checking is going to the machines, deciding is the one nobody else can do, and owning stays where it is. Other frameworks split the loop by control point, approval, review and escalation, which tells you where the person sits. Decide, own, or check says what they do there, and which of the three is worth paying for. If your people spend their days checking and their spare minutes deciding, the ratio is backwards, and so is the invoice.
How often does a human in the loop actually change the outcome?
To see how the three jobs split in practice, we went back to the project from the opening, the same one behind the companion article: a voice-driven mobile assistant for frontline supervisors, built by a team of four with Rootstrap's multi-agent delivery system. The system records every check it runs and every time it stops to ask a person. We pulled two weeks of it, 28 August to 11 September 2026, 108 code changes.
In those two weeks the agents ran 976 checks, and the system stopped to ask a person 120 times. Here is how that splits across the three jobs:
The agents ran the checks themselves, fixing whatever failed before a person saw it. The one that needed a person needed hands: a bug had been reported on a phone, and confirming the fix meant holding the phone. The other 26 stops were things only a person can do, like granting access to a system. The number that matters most for anyone paying for this: what happened when the system did ask a person. Of the 55 stops closed in those two weeks, people agreed with what the agents had proposed in 45. In six they changed something, and all six were decisions, not corrections of an agent’s mistake.
When people were asked to approve, they approved. When they were asked to decide, they changed things, and their time was the scarce part: the microphone question waited 66 hours for an answer, and 65 stops were still waiting for a person when the two weeks ended. The bottleneck on this project was never review capacity. It was decision capacity.
That is what a multi-agent system looks like when it is used well: 976 checks nobody paid a person for, and people's hours spent on deciding and signing.
The question to ask: decide, own, or check?
Here is the test. It works on your own team as well as on any vendor making the human-in-the-loop claim: for every person in the loop, ask what they are there to do. Decide, own, or check.
Decide. Good. That's what people are for now, and it's where your delivery is most likely to be waiting. Remember the microphone: the question sat for 66 hours. Ask how fast those decisions get made, and who is allowed to make them.
Own. Also good, and don't let anyone optimize it away. Approvals and sign-offs look redundant as checks, and they are. As signatures, they're the part of the loop that should never shrink. An agent can verify a release. It can't answer for one. Regulators are heading the same way: Article 14 of the EU AI Act, which covers high-risk systems, asks for a person who can interpret the system's output, override it and stop it, not one who re-reads what a machine already read.
Check. Ask why. A person re-reading work a machine has already reviewed is not a second layer of safety. It checks a sample of what the machine checked in full, it drifts into approving whatever the machine approved (Article 14 names this automation bias), and it burns the hours the decisions were waiting for. You're paying for a bottleneck dressed as a safety net. Sometimes there's a reason, such as a regulator or a project where the agents' verification isn't trustworthy yet. "We review everything because we don't trust the agents" is not one. It means they haven't built the system that would let them stop.
One caveat, because you should hold us to it too: this is one project over two weeks, and the system called for a person by mistake at least three times. Taking people out of checking only works when the checks leave a trail and the rule that calls people back in is itself checked. Ask to see the trail.
So what are you paying people for? Not to check: that work is going to the machines, and it gets better without anyone being hired. You need people for the moments when a question has no agreed answer, which on this project was half of all the stops. You need a name on what ships. And you need people to run the system itself, because deciding what it checks, when it stops and whom it asks is engineering work nobody has automated.
Which raises the obvious question: if the decisions are the job, why not make them yourself? You could. The question is whether you know what each answer costs. The autopilot flies the plane, and the pilot is paid to understand why it stays in the air, so they can tell when the autopilot is wrong. The same goes for what a failed payment should say to a customer, what happens to a person's data when they leave, or when it is safe to cut a corner and when it isn't. Product managers, designers, engineers and security people are paid to know what each answer costs.
Staff for judgment. Sign for what ships. Stop paying people to check.
If you're evaluating a team right now, we wrote up four questions to ask before you hire one. This is the answer underneath all four. The goal was never a system that stops needing people. It's a system that knows exactly which moments still do, and puts the people there.
If you want to see how ours works, write to us.
Frequently asked questions
How can I tell if a vendor's human in the loop is real? Ask what each person is there to do: check the work against what was agreed, decide what nobody has decided, or own what ships. A human in the loop (often shortened to HITL) is real when the people are mostly deciding and owning. If they are mostly checking, the loop is slow and the safety is mostly appearance. Ask to see the record: what the system checked on its own, when it stopped to ask a person, and what the person did with the question.
What should the people on an AI delivery team actually be paid to do? Judgment and ownership. Judgment is making the call when nothing was agreed, and it takes expertise: knowing what each answer costs is what engineers, designers and security specialists are paid for. Ownership is the name on what goes out, because an agent can verify a release but can't answer for one. Someone also has to run the system itself, deciding what it checks and when it stops. Checking is the job to stop paying for: in our data, agents ran 976 checks and one more needed a person.
What is the difference between verification and judgment in an agent loop? Verification is confirming that work matches what was already agreed: tests, requirements, the team's conventions. Agents do it at scale; on one production project, 976 times in two weeks. Judgment is deciding what should happen when nothing was agreed or two agreed things contradict each other: what the product does when the requirements are silent, which version ships. That still needs a person, and the bottleneck was decision capacity, not review capacity.
