Authority, Escalation and Human Control

When Does the AI Have to Stop and Ask?

Capability Is Still Not Authority

An AI system may be technically capable of proposing, scheduling or invoking an action. That does not mean it has authority to perform that action.

CAN DO ≠ MAY DO

This distinction has followed us from tools, through agents, permissions, planning and mission success. Now we make the control boundary explicit.

Five Different Questions

The Simplest Control Loop

The model may propose an action. The surrounding system decides whether that proposal can cross the execution boundary.

What Is Escalation?

Escalation means transferring a decision, uncertainty or required authorization to an authority outside the current autonomous scope.

That authority might be:

Escalation is not necessarily failure. It can be successful compliance with the system's control design.

Four Ways to Continue — One Way to Stop

Human-in-the-Loop

A human-in-the-loop design deliberately places a person inside the workflow at selected decision points.

The human is not merely watching. Their response changes what the system is authorized or instructed to do.

Human-on-the-Loop

Another design allows automated action within defined limits while a human supervises and can intervene.

The distinction matters because “a human is involved somewhere” does not tell us whether approval is required before action or whether oversight occurs while/after authorized actions proceed.

Human-out-of-the-Loop

Some bounded operations may execute without individual human review.

That does not mean the AI has unlimited autonomy. It means the system has delegated a specific class of decisions or actions in advance.

Delegated Authority

Authority can be scoped:

This is far clearer than “the agent is autonomous.”

Autonomy is meaningful only relative to a defined scope.

The Approval Gate

Suppose diagnosis indicates that restarting Apache is likely to restore service.

Until approval arrives, the proposed restart is not an authorized action.

Approval Should Be Specific

A vague “yes” can be dangerous if the proposed action has changed since the approval request.

The approval can be bound to the action, target, scope and time. If the plan later becomes “edit config and reboot host,” the old approval should not silently expand to cover it.

Approval Is Not a Permanent Permission Upgrade

One authorized action need not alter the agent's general permission envelope.

When Should the System Ask?

Useful escalation triggers can include:

Risk Can Change the Threshold

The same uncertainty can justify different behavior depending on consequence.

The exact thresholds are system and domain decisions—not universal numbers built into the transformer.

Uncertainty Is Not the Same as Probability

An agent may know that evidence is incomplete without possessing a calibrated numeric probability for every outcome.

That can be enough to trigger escalation.

Ask a Question or Ask for Permission?

These are different.

The first obtains missing mission information. The second requests authority for a defined action.

Escalation Should Carry Evidence

A good escalation does not merely say:

It can carry the decision context:

The human can then make a better decision without reconstructing the entire investigation.

Human Approval Can Also Be Wrong

Human control is not magical correctness. A person can misunderstand, approve the wrong target or overlook new evidence.

The system should still validate the action against current state and applicable controls.

Human authorization answers “may proceed.” It does not prove “this action will work.”

State May Change While Waiting

Suppose approval takes ten minutes. During that time another operator repairs the service.

The system should not blindly execute a stale plan merely because approval finally arrived.

Revalidation may be required:

Emergency Stop

A control architecture may include a way for an authorized human or supervisory system to halt further action.

Stopping an external operation already in progress may not always be possible. “Stop” itself has implementation limits.

Escalation in Multi-Agent Systems

An individual worker agent may not need direct access to a human. It can escalate to a supervisor or orchestrator.

The important question is whether the authority chain is explicit and preserved.

Delegation Cannot Manufacture Authority

If a supervisor agent lacks permission to modify production, delegating the request to another agent should not magically create that permission.

Relevant constraints and authorization context must propagate through delegation.

Escalation Can Be Part of Mission Success

Recall Page 38's mission statuses. A mission may correctly end with:

If the mission only authorized diagnosis, that is not incomplete obedience. It may be exactly the intended endpoint.

The AI Should Not Quietly Expand “Done”

If asked to diagnose, the system should not silently reinterpret success as “diagnose and repair.” If asked to draft, it should not silently reinterpret success as “draft and publish.”

Authority Is External to the Transformer

The transformer computes representations and generates outputs. It does not contain a magical hardware bit labeled AUTHORIZED.

Authority emerges from the surrounding control architecture:

The model may help interpret the situation, but enforceable controls belong in the system around it.

WITNESS: The Approval Chain Is Evidence

For consequential actions, WITNESS should be able to reconstruct:

This produces a chain:

Forensic reconstruction should be able to answer not only “What did the AI do?” but “Under whose authority did that action cross into the world?”

Absence of a Human Click Is Not Proof of No Human Control

A human may establish policy and delegated authority in advance. Conversely, a human clicking “approve” once does not mean every later action was individually reviewed.

HAAW therefore asks for the actual control mechanism, not the vague statement “there was a human in the loop.”

The Stop-and-Ask Decision