How AI's Actually Work — HAAW

Inside a Reasoning Step

What Happens in the Machine While a Problem Is Being Worked — We have described reasoning behavior from the outside. Now slow one generation step down and follow the numbers through the machine.

Freeze the Machine

Suppose the model has been asked: “What is 17 × 24? Work it out by breaking 24 into easier parts.”

The context might already contain generated text such as:

17 × 24 = 17 × (20 + 4) =

Now freeze time. The machine must generate what comes next.

What Physically Exists Right Now?

Weights

Learned numerical tensors resident in accelerator memory or otherwise available to computation.

Token Sequence

The prompt and any generated tokens so far, represented as token IDs.

KV Cache

Per-sequence attention state retained from earlier processed tokens.

Runtime State

Buffers, scheduling information, generation settings, and other execution state.

There is no stored English object saying “Craig's multiplication method.” There are numbers and software operations.

Step 1 — The Current Sequence Is Numerical

visible text "17 × 24 = 17 × (20 + 4) =" ↓ tokenizer token IDs [ ... numerical identifiers ... ] ↓ embeddings + positional handling matrix of vector representations

By the time the transformer operates, the text has become numerical representations. The exact token boundaries depend on the tokenizer.

Step 2 — A Transformer Layer Receives Activations

Call the incoming representation matrix X.

X = current token representations X ↓ normalization ↓ self-attention ↓ residual addition ↓ normalization ↓ MLP ↓ residual addition ↓ updated representations

Then another transformer layer receives those updated values. This repeats through the model's stack.

Step 3 — Attention Builds Q, K and V

Q = XWQ K = XWK V = XWV

X contains current activations. WQ, WK and WV are learned weight matrices. GPU hardware performs enormous numbers of multiply-and-add operations to produce new tensors.

Already we can see the distinction: training created the W matrices. This particular multiplication problem creates X, Q, K, V and other request-specific activations.

Step 4 — Attention Changes What Information Can Influence the Current Position

QKᵀ ↓ attention scores ↓ causal mask ↓ softmax ↓ weighted combination of V ↓ attention output

For the position being processed, attention can combine information from permitted earlier positions in the sequence. Different heads can form different learned relationships.

HAAW should be careful here: we can inspect attention computations, but an attention weight alone is not a complete explanation of why the model “reasoned” a particular way.

Step 5 — The MLP Transforms the Representation Again

attention-updated representation ↓ learned linear transformation ↓ nonlinear activation / gating ↓ another learned transformation ↓ new activation values

The exact MLP design varies by architecture, but again the physical work is tensor arithmetic using learned parameters.

Step 6 — Residual Paths Preserve and Add Information

old representation ─────────────┐ + new transformation ─────────────┘ ↓ updated representation

Residual connections let transformations modify an existing representation rather than forcing each layer to recreate everything from scratch.

Step 7 — Do It Again. And Again.

Layer 1 → changed activations Layer 2 → changed activations Layer 3 → changed activations ... Layer N → final hidden representation

The model's learned computational structure is expressed through the cumulative effect of many transformations.

There is no point where HAAW needs to insert a mysterious “reasoning module.” The reasoning behavior emerges from the trained network operating on the current sequence.

Step 8 — Turn the Final Representation Into Vocabulary Scores

final hidden representation ↓ output projection / language-model head ↓ logits ↓ scores across vocabulary

The machine has not yet produced an English word. It has produced numerical scores for possible next tokens.

Step 9 — Select One Token

candidate token relative score ------------------------------------ " " ... "(" ... "340" ... "17" ... "+" ... other tokens ... generation procedure ↓ one next token selected

This table is conceptual, not a claim about the exact tokenization or scores of any particular model.

And Here Is the Trick

The selected token is appended to the sequence.

BEFORE: 17 × 24 = 17 × (20 + 4) = NEXT TOKEN: ... AFTER: 17 × 24 = 17 × (20 + 4) = ... ↓ THIS UPDATED SEQUENCE BECOMES INPUT STATE FOR THE NEXT GENERATION STEP

The machine has changed its own immediate working context by generating a token.

Reasoning Can Unfold Through the Generation Loop

problem + work so far ↓ transformer computation ↓ next token ↓ append ↓ problem + slightly more work ↓ transformer computation ↓ next token ↓ append ↓ ... repeat ...

A multi-step answer can therefore involve repeated sequential passes, while each individual pass contains enormous parallel tensor computation.

Two Different Kinds of “Step”

Human-Readable Step

“Break 24 into 20 + 4.” One conceptual step in an explanation.

Machine Generation Step

One next-token computation involving the transformer stack, logits, selection, and append.

These are not necessarily one-to-one. A single human-readable reasoning step can require many generated tokens and therefore many model generation steps.

The Model Does Not Manipulate Our Sentence Directly

"break 24 into 20 + 4" ↓ tokens ↓ vectors ↓ distributed numerical transformations ↓ new token probabilities

Our sentence is a human-readable description. Inside the transformer, the operative state consists of numerical representations—not a little symbolic sentence being passed from neuron to neuron.

Does It Actually “Know” 24 = 20 + 4?

From behavior we may observe that the model reliably uses that relationship. Mechanically, what we can say is that its learned parameters and current activations produce outputs consistent with the relationship.

HAAW discipline: useful human words such as knows, understands, and reasons describe behavior at a higher level. Underneath, we can continue tracing numerical state and computation.

Where Did Training Enter This Step?

Training is not happening now. Its result is present in the weights.

PAST — TRAINING examples → loss → gradients → optimizer ↓ learned weights NOW — INFERENCE new problem + learned weights ↓ activations ↓ transformer computation ↓ next-token scores ↓ generated solution

Training changed the matrices. Inference is now using the matrices.

Where Is the Reusable Procedure?

Not in one obvious location that says:

IF multiplication_is_hard: split_second_number() multiply_parts() add_results()

A transformer can display behavior resembling that algorithm without storing it in this conventional program form. The capability is represented through learned parameter structure and the activation dynamics produced when a particular input runs through it.

Could We Watch It?

Some machine events are directly instrumentable:

Tokens

Input and generated token IDs can be recorded.

Activations

Selected internal tensors can sometimes be captured or analyzed.

Attention / Cache

Attention-related values and KV state may be observable depending on access and implementation.

Hardware Events

Kernel execution, memory movement, timing, utilization, and device activity can be measured.

But recording values is not the same as understanding their causal meaning. That gap is exactly where interpretability—and eventually WITNESS—becomes interesting.

The Whole Step, Mechanically

Current problem + generated work
↓
Token IDs
↓
Embeddings / current representations
↓
Attention + MLP + residual transformations through many layers
↓
Final hidden representation
↓
Logits across vocabulary
↓
Next-token selection
↓
Append token to sequence / update KV cache
↓
Repeat with changed working context

So What Is Happening While It “Reasons”?

The prompt and prior generated work exist as a token sequence and numerical state.

Learned weights repeatedly transform activations through attention, MLPs, residual paths, and other architecture-specific operations.

The final representation produces scores for possible next tokens.

One token is selected and appended.

That token changes the context for the next computation.

Repeated generation can construct intermediate work and eventually an answer.

At the physical level, the “reasoning step” is not magic: it is stored weights, transient numerical state, memory movement, tensor operations, token selection, and repetition.

HAAW Checkpoint

Reasoning step
A useful high-level description of part of a problem-solving process; it should not be confused with one specific hardware or token operation.
Machine generation step
One autoregressive next-token computation followed by token selection and sequence update.
Activation
A transient numerical value or tensor created while the current input is processed.
Attention
A learned computation that combines information from permitted sequence positions using query, key, and value representations.
MLP
A learned feed-forward transformation inside a transformer block.
Residual connection
A path that adds a transformed representation back to an earlier representation.
Logit
A numerical score associated with a possible output token before conversion or selection procedures.
Working context
The current sequence and associated runtime state influencing subsequent generation.
Distributed representation
Information represented through patterns across many numerical dimensions rather than necessarily one explicit symbolic location.
Interpretability
The study of how internal model states and computations relate to model behavior.
Central lesson
Training shaped the weights; during reasoning-like inference, those weights repeatedly transform the current numerical state, generate another token, and thereby change the state used for what comes next.