How AI's Actually Work — HAAW

Tools and Agents

When the Model Can Do More Than Generate the Next Token

Start With a Correction

The language model itself has not suddenly grown hands, opened a browser, queried a database, or executed a shell command. At its core it is still performing inference and generating output. The larger AI system can recognize specially structured model output and use it to invoke external capabilities.

LLM generates / selects structured output ↓ SURROUNDING SOFTWARE interprets that output ↓ TOOL performs an external operation

Four Pieces — Do Not Blend Them

Model

Processes context and produces tokens or structured outputs according to its learned parameters.

Tool

An external capability: search, calculator, database, code runner, file system, API, device, or other service.

Orchestrator

Software that presents tools to the model, validates requests, executes permitted calls, and returns results.

Agent Loop

A repeated cycle in which results from actions become new context for deciding what to do next.

A Tool Is Not Magic

A tool can be an ordinary program or service.

calculator(17, 24) database_query("Bessie RAM") web_search("current weather") read_file("manual.pdf") run_test("project") send_request(API, arguments)

What makes it an AI tool is not that the tool itself must be intelligent. It is that the AI system can invoke it as part of a model-directed workflow.

How Does the Model Know a Tool Exists?

The surrounding system can provide a description of available tools and their expected arguments as part of the model's usable context or control structure.

AVAILABLE TOOL: name: calculator purpose: perform arithmetic arguments: expression: string USER: "What is 17 × 24?"

The model can then produce output corresponding to a request to use that tool.

The Model Does Not Directly Execute the Function

MODEL OUTPUT / TOOL REQUEST { tool: "calculator", expression: "17 * 24" } ↓ ORCHESTRATOR validates request ↓ CALCULATOR SOFTWARE executes operation ↓ RESULT 408

The exact representation varies by system. The important architecture is the separation: the model proposes or selects the action; other software performs it.

The Result Comes Back

User asks question
↓
Model requests calculator
↓
Orchestrator executes calculator
↓
Tool returns 408
↓
Result becomes new model context
↓
Model generates response

Now the answer can be grounded in deterministic arithmetic rather than relying only on language-model continuation.

The Tool Result Is New Information

This connects directly to the Context page.

before tool: context = user question + existing instructions after tool: context = user question + existing instructions + tool request + tool result

The tool has not rewritten the model's weights. It has supplied information for the current inference sequence.

Search Is a Tool

USER QUESTION ↓ MODEL / ORCHESTRATION ↓ SEARCH REQUEST ↓ SEARCH SYSTEM ↓ RESULTS ↓ MODEL CONTEXT ↓ ANSWER

The model's training knowledge and current search results are therefore different sources of information. HAAW should preserve that distinction.

Database Access Is a Tool

Suppose the model needs the current inventory count.

MODEL: need current SKU quantity ↓ TOOL REQUEST: inventory_lookup(SKU-481) ↓ DATABASE: 137 units ↓ MODEL: "There are currently 137 units."

The number 137 came from the database result. The LLM provided language and workflow behavior around it.

Code Execution Is a Tool

A model can generate code without executing it. A code-execution tool changes the architecture:

MODEL generates program ↓ ORCHESTRATOR sends program to sandbox/runtime ↓ CPU executes instructions ↓ stdout / errors / files / test results ↓ results returned to model ↓ model can revise

This creates a powerful feedback loop: generate → execute → observe → correct.

File Access Is a Tool Boundary

An LLM does not automatically have access to every file on a machine.

MODEL │ │ requests permitted file operation ↓ FILE TOOL / SERVICE │ ├─ permission check ├─ locate file ├─ read requested content └─ return result ↓ MODEL CONTEXT

Access control belongs to the surrounding system, not to wishful thinking about what the model “knows.”

From Tool Use to an Agent Loop

A single tool call is not necessarily an agent. The word agent is used inconsistently, but a common architecture allows repeated model-directed steps toward a goal.

GOAL ↓ observe current context/state ↓ choose next action ↓ execute tool ↓ observe result ↓ update context/state ↓ choose next action ↓ ... ↓ stop / report result

Agent Does Not Mean Autonomous Creature

“Agent” is a system-design term, not proof of independent consciousness, desire, or self-created goals.

An agentic system can be software executing a loop under a user-supplied objective, available tools, permissions, stop rules and other constraints.

A Simple Agent Example

Goal: determine why a web service is unavailable.

GOAL: diagnose web service 1. ping / reachability tool → host reachable 2. service-status tool → httpd inactive 3. log-reading tool → configuration syntax error 4. configuration-check tool → line 83 invalid 5. report: "Service is stopped because the configuration fails validation at line 83."

This should feel familiar to a sysadmin. The interesting change is that the model can help choose which diagnostic operation to perform next based on results from the previous one.

The Model Is Still Generating Between Actions

Even in an agent loop, the LLM's core mechanism has not vanished.

context ↓ TOKENIZER → TOKEN / TOKEN ID → EMBEDDING LOOKUP → EMBEDDING VECTOR → TRANSFORMER ↓ structured action / next-token output ↓ external tool ↓ new context ↓ model inference again

The agent loop is built around model inference.

Planning Can Be Another Model Operation

A system may ask the model to decompose a larger objective:

GOAL: "Find the cause of the outage and prepare a report." possible decomposition: 1. inspect service status 2. inspect logs 3. inspect configuration 4. test suspected cause 5. summarize evidence

The plan is generated model output. It can be useful, incomplete, or wrong. Execution results provide opportunities to revise it.

Plans Are Not Obligations

A robust agent should be able to change course when observations contradict its plan.

PLAN says: check database next NEW EVIDENCE says: database is healthy; DNS resolution is failing GOOD LOOP: update state → revise next action BAD LOOP: blindly execute stale plan

Tools Create Consequences

Generating a sentence and executing an action are not equivalent.

Read

Search, inspect, retrieve, calculate.

Write

Create or modify a file, database record or configuration.

Communicate

Send a message, submit a form or call an external service.

Control

Start, stop or alter a process, machine or other system.

The more consequential the tool, the more important permissions, validation, isolation, confirmation and auditing become.

Permissions Belong in the Architecture

MODEL REQUEST ↓ POLICY / PERMISSION LAYER ↓ allowed? ── no → reject / request authorization │ yes ↓ validate arguments ↓ execute tool ↓ record result

A capable model should not automatically imply unlimited authority.

Human Approval Can Be a Synchronization Point

Some workflows deliberately stop before a consequential action.

model proposes: "Delete obsolete records" ↓ WAIT FOR HUMAN APPROVAL ↓ approved? ├─ no → stop └─ yes → execute permitted operation

That is a real dependency in the system's event flow: later action cannot proceed until the required authorization arrives.

Tools Can Fail

Unavailable

The service is down or unreachable.

Bad Arguments

The model requests an operation with malformed or inappropriate parameters.

Permission Denied

The requested operation exceeds authorized access.

Bad Result

The external source itself returns incorrect or stale information.

Misread Result

The tool works, but the model interprets the returned evidence incorrectly.

Loop Failure

The agent repeats, wanders, or fails to recognize that its goal is complete.

Stopping Is Part of Agent Design

An agent loop needs a reason to stop.

stop when: goal satisfied OR user intervention required OR tool reports terminal condition OR step / time / cost limit reached OR permission boundary reached OR confidence/evidence insufficient OR unrecoverable error occurs

Without stopping conditions, “keep trying” can become wasted computation or harmful repeated action.

State Between Steps

Multi-step work requires the system to preserve useful state:

goal observations tool results files / artifacts completed steps remaining tasks errors permissions current plan

Some state may live in the model's context; some can live in external storage or orchestration software. Again, the LLM is not the whole system.

Parallel Agents and Parallel Tools

Independent tasks can sometimes run concurrently or in parallel.

┌→ search source A ─┐ GOAL → dispatch ─┼→ search source B ─┼→ collect results → continue └→ query database ──┘

But Craig's earlier rule returns: if the next step needs all three results, the fast branches may wait for the slow one.

Parallel work removes some waiting. Dependencies put waiting back wherever later work requires unfinished results.

An Agent Can Still Hallucinate

Tools do not eliminate the language model's failure modes.

correct tool result ↓ model misinterprets it ↓ wrong conclusion OR model chooses wrong tool ↓ irrelevant result ↓ confident answer

Grounding, verification and careful orchestration remain important.

Agentic Does Not Mean Unbounded

USER GOAL + SYSTEM INSTRUCTIONS + AVAILABLE TOOLS + PERMISSIONS + CURRENT CONTEXT / STATE + STOP CONDITIONS ↓ AGENT LOOP

The surrounding boundaries are part of the agent system just as much as the model is.

WITNESS Just Got Busier

A tool-using agent creates a much richer event field than a single inference.

model inference ───────────────┐ tool selection ────────────────┤ permission check ──────────────┤ external request ──────────────┤ tool execution ────────────────┼→ AGENT EVENT FIELD tool result ───────────────────┤ context update ────────────────┤ next inference ────────────────┤ human approval ────────────────┤ artifact modification ─────────┘

For forensic reconstruction, the relationship among these events can matter as much as their timestamps.

What Would WITNESS Record?

goal / request available tool definitions model-requested action arguments authorization decision tool identity / version start and completion times tool output / error context returned to model subsequent model action files or external state changed human approvals stop condition

That record begins to answer a far richer question than “what tokens did the LLM generate?”

The Model Did Not Grow Hands

The LLM remains a model performing inference.

Tools provide external capabilities.

An orchestrator connects model output to permitted tool execution and returns results.

An agent loop repeatedly combines model inference, action, observation and updated state toward a goal.

So when we casually say:

"The AI searched the web, read the file, ran the test, and fixed the problem."

HAAW opens that sentence into a chain of distinct software and hardware events.

HAAW Checkpoint

Tool
An external capability that can perform an operation or retrieve information for the AI system.
Tool call
A structured request for an external capability to perform a specific operation with specified arguments.
Orchestrator
Software coordinating model inference, tool availability, validation, execution, permissions and returned results.
Agent
A variably used term; here, a system that can repeatedly select actions, observe results and continue toward a goal.
Agent loop
Observe state → choose action → execute → observe result → update state → repeat until a stop condition.
State
Information preserved across steps, such as goals, observations, results, plans, artifacts and completed work.
Permission boundary
A control determining whether a requested external action is authorized.
Human-in-the-loop
A design in which specified actions require human review, input or approval before proceeding.
Tool result
Information returned by an external operation and made available for subsequent system processing.
Stop condition
A rule or event that ends or pauses an agent loop.
Central lesson
The LLM does not need to perform every operation itself. A larger system can convert model-directed structured outputs into controlled external actions and return their results for further inference.