Human in the Loop: Which Decision Matters Most

The Wrong Loop: Why "Human in the Loop" Is Becoming Your Biggest Vulnerability

A Fortune 500 CISO told me last month his AI governance committee meets every two weeks to review model deployment requests. Average approval time: 11 days. Meanwhile, his security team's automated response system can quarantine a compromised endpoint in 0.3 seconds—but still routes the final "wipe and restore" decision through a manager who's in back-to-back meetings until Thursday.

He thought he was being careful. He was actually designing his own defeat.

The Asymmetry Is Getting Worse

Ben Thompson wrote something recently that I've been wrestling with for weeks. The economics of attack versus defense have always been asymmetric, but AI is making it grotesque. Attackers only need one exploit to land. Defenders have to be perfect, every time, forever.

That math creates enormous pressure to automate defense completely. And here's where it gets uncomfortable: the startups will do it. They'll automate the entire stack because they have to—they don't have the budget for humans and they're not encumbered by twenty years of compliance frameworks built around control attestations.

Regulated incumbents won't. Or can't. Or tell themselves they're being prudent when they're actually just slow.

I've sat in enough risk committees to know the script. "We need a human in the loop for accountability." Heads nod. Nobody asks which loop. Nobody asks what that human is actually deciding. The phrase itself becomes the control, like saying it three times makes you safe.

That caution—the thing that feels like protection—is becoming the opening attackers walk through.

Most "Human in the Loop" Is Security Theater

Let me be blunt about what I see in a lot of enterprise environments: humans rubber-stamping approvals they don't understand, on timelines that guarantee they won't look closely, for systems they last touched three years ago.

That's not a control. That's a liability with a badge.

I watched a financial services client route every API configuration change through a review board. Sounded rigorous. In practice? A junior engineer submits the change request at 4:45 PM on Thursday. A senior architect who's never seen the codebase clicks "approve" Friday morning because the ticket's been open for five days and IT has SLAs. The change deploys. Two weeks later they discover they've been leaking customer data to a logging service.

The human wasn't in the loop. The human was the bottleneck that made everyone believe there was oversight.

Here's the thing Thompson gets right: if your human review adds latency but not judgment—if the person can't actually evaluate what they're approving and has no authority to say no—you've built security theater. You've added cost and delay while creating the appearance of control.

And when the breach happens, that appearance becomes evidence of negligence.

The Binary Is a Trap

So automate everything? Let the models run free?

No. That's the other cliff.

The debate has collapsed into a false binary: automate-everything versus automate-nothing. All-in on AI or drown in human process. Pick your poison.

The real question isn't whether to keep humans in the loop. It's which loop.

I've survived enough technology disruption cycles to recognize the pattern. When electronic trading hit the NYSE floor in the 1990s, the argument wasn't "keep humans or go electronic." It was "which decisions require human judgment and which are better executed by machines that don't get tired, emotional, or distracted?"

The firms that thrived figured out the answer. Market makers who insisted on manually approving every trade got destroyed by latency. Firms that automated everything including risk limits blew themselves up spectacularly. The winners automated the repetitive, low-judgment, high-speed decisions and kept humans on the circuit breakers.

Same movie, different decade.

Audit Your Loops

Here's what I ask clients to do, and it makes people squirm:

Map every place you currently require human approval or review. Not the policy—the actual process. Then ask two questions:

1. What is this person actually deciding?

If the answer is "they're verifying the request meets policy" and the policy is codifiable, that's the wrong loop. Pull the human out. Automate it. You'll be faster and more consistent.

If the answer is "they're making a judgment call on something the system has never seen before" or "they're the person who goes to jail if this goes wrong," that might be the right loop.

2. What's the blast radius if this goes wrong?

Low consequence, high frequency, reversible? Automate it. Your humans are expensive and you're wasting them on decisions that don't matter.

High consequence, irreversible, novel? Keep a human there. Not as theater. As the last thing between you and a one-way door.

The Backwards Pattern

Here's what I see in most regulated firms: humans stapled to reversible decisions, automation creeping toward the irreversible ones.

Why? Because the low-value stuff is easy to staff and document. You can write a procedure. You can train someone in an afternoon. You can show the auditor your sign-off sheet.

The irreversible stuff—the decision to wipe a production database, the call to shut down a revenue-generating system because you think it might be compromised, the judgment that this transaction pattern is fraud not just unusual—that's hard to staff. It requires expertise, authority, accountability. It's expensive and messy and doesn't fit in a flowchart.

So firms automate it, or build decision trees that pretend judgment is just following branches, or route it to whoever's on call.

Then they put humans in the loop on the stuff that doesn't matter.

I watched a healthcare client require three-person approval for any change to their marketing website. Takes four days minimum. Meanwhile, their production database backup restoration process is fully automated and can be triggered by anyone with ops access.

Which door would you rather have guarded?

The Question Your Security Team Should Answer Monday

I don't know exactly where the human/machine boundary should be in your organization. It depends on your risk tolerance, your talent, your system complexity, a dozen factors I can't see from here.

But I know this: if you can't articulate which decisions require human judgment and why, you're not being careful. You're just being slow.

And in a world where attackers are automating everything, slow is the same as wrong.

So here's your homework. Walk into your security team's office Monday morning and ask:

"Show me our five highest-consequence irreversible decisions. Who makes them? How fast can they act? And what training did they get?"

Then ask:

"Show me the five places where humans approve things but couldn't actually explain what they're approving or stop it if they wanted to."

The gap between those two lists is where you're vulnerable.

Not because you kept humans in the loop. Because you put them in the wrong one.

Want to talk through where your humans should actually be? I work with finance and professional services firms trying to get this right before they learn the hard way. Let's talk.