HAAW — Training

The Great Divide

Inference uses the matrices. Training changes the matrices.

Inference

Supply input. Use the learned weights. Compute activations and logits. Generate output.

Training

Supply training examples. Measure model error. Compute how parameters contributed to that error. Adjust the weights. Repeat.

This distinction is foundational.

The model that answers a prompt is the result of an earlier optimization process in which enormous numbers of parameter values were repeatedly adjusted.

Before Training: A Model Structure Without Learned Skill

The architecture specifies the kinds of operations the model can perform: embeddings, attention, feed-forward transformations, normalization, output projection, and so on.

But architecture alone does not supply useful trained parameter values.

MODEL ARCHITECTURE
├── layers
├── dimensions
├── attention structure
├── parameter tensors
└── computation graph / operations

INITIAL PARAMETER VALUES
└── initialized numerically before learning

Training begins with parameter values that have not yet become the final learned weights.

What Is Training Trying to Do?

At a high level, training tries to find parameter values that make the model perform better according to a defined objective.

For a language model trained on next-token prediction, the model is repeatedly asked to assign useful probability to the token that actually follows the preceding sequence in training data.

Training sequence:
The sky is blue

input context → "The sky is"
target token → "blue"

The real training corpus and tokenization are vastly more complex than this tiny example, but the principle is visible.

The Forward Pass

The first part looks familiar because inference also performs a forward computation.

Training tokens enter model

↓

Embeddings are produced

↓

Transformer layers compute activations

↓

Output layer produces logits

↓

Logits imply a distribution over possible next tokens

This direction—from input through the network toward an output—is the forward pass.

But Training Knows the Target

During training, the system has target information derived from the training example.

Model prediction:
blue → 0.41
clear → 0.17
dark → 0.08
...

Training target:
blue

The training system can therefore measure how well the model's output agrees with the desired target.

Loss: Turn Error Into a Number

A loss function converts model performance on the training objective into a numerical value suitable for optimization.

model output + target
    │
    ▼
LOSS FUNCTION
    │
    ▼
LOSS VALUE

For next-token language modeling, cross-entropy loss is commonly used. Conceptually, it penalizes the model when it assigns insufficient probability to the correct target token.

Loss gives training something numerical to minimize.

“The answer was bad” is not enough for an optimizer. Training needs a mathematical objective whose change can be related to changes in the parameters.

The Central Question

The model may contain billions of adjustable parameters.

If the loss is too high, which parameters should change?

And in which direction?

LOSS = some numerical value

parameter w₁ → change up or down?
parameter w₂ → change up or down?
parameter w₃ → change up or down?
...
parameter wₙ → change up or down?

This is where gradients enter the story.

Gradient: Which Way Changes the Loss?

For each trainable parameter, we want information about how a small change in that parameter would affect the loss.

Mathematically, this is expressed through derivatives. Across many parameters, those derivatives form a gradient.

Loss = L
Parameter = w

∂L / ∂w

roughly asks:
"If w changes slightly, how does L change?"

The gradient does not simply say “this weight is wrong.” It describes local sensitivity of the loss to parameter changes.

Backpropagation

The model is a long composition of mathematical operations. We already followed information forward through those operations.

Backpropagation works through the computational relationships in the reverse direction, efficiently applying the chain rule of calculus to determine gradients for trainable parameters.

FORWARD
input → layer 1 → layer 2 → ... → output → loss

BACKWARD
gradients ← layer 1 ← layer 2 ← ... ← loss derivatives

Backpropagation does not run the model “backward” to produce reverse language.

It propagates derivative information backward through the computation graph so training can determine how parameters contributed to the loss.

The Chain Rule Is Doing Serious Work

Suppose one value affects another, which affects another, which finally affects the loss.

w → x → y → L

The chain rule lets us connect those dependencies:

dL/dw = dL/dy × dy/dx × dx/dw

Real transformer training involves enormous tensor operations rather than this tiny scalar example, but the mathematical idea scales: derivative information flows through the network's composed operations.

Now the Optimizer Can Act

Gradients provide information about how parameter changes relate to the loss. An optimizer uses that information to update the parameters.

A deliberately simplified gradient-descent update looks like:

w_new = w_old − η × gradient

η = learning rate

The learning rate controls the scale of the update. Too large can make optimization unstable; too small can make progress impractically slow.

Modern training commonly uses more sophisticated optimizers than plain gradient descent, but this equation exposes the central mechanism.

Craig's Numerical Analysis Connection

Training is not one heroic calculation that discovers the answer.

It is an iterative numerical process.

choose parameter values
    ↓
evaluate model behavior
    ↓
measure objective / loss
    ↓
calculate gradients
    ↓
adjust parameters
    ↓
evaluate again
    ↓
repeat, repeat, repeat...

That puts AI training firmly in the world of numerical methods: iterative improvement, finite precision, step sizes, stability, convergence behavior, approximation, and very large collections of numbers.

One Training Step

1. Select / prepare training batch

↓

2. Forward pass

↓

3. Compute loss

↓

4. Backpropagate gradients

↓

5. Optimizer updates parameters

↓

6. Continue with more training data

↺

At modern scale, that loop is distributed, parallelized, optimized, checkpointed, and repeated across vast quantities of data and computation.

A Tiny Matrix View

Return to a generic linear transformation:

Y = XW

During inference:

X changes with input
W stays fixed
Y is computed

During training:

X enters
W is used
Y contributes to eventual loss
gradient for W is calculated
W is updated

After enough training, the final learned W is saved as part of the model.

Training Requires More Memory Than Just the Weights

Inference already required weights, activations, caches, and buffers. Training adds substantial additional state.

Parameters

The model weights being learned.

Activations

Forward-pass intermediate values needed for computation and, often, gradient calculation.

Gradients

Derivative information associated with trainable parameters.

Optimizer State

Many optimizers maintain additional numerical state used to determine parameter updates.

This is one reason training a large model can demand substantially more accelerator memory than merely running that model for inference.

Activations Matter Twice

During inference, activations are intermediate results on the way toward output.

During training, forward-pass information can also be needed to calculate gradients during backpropagation.

FORWARD PASS
activations created
    │
    ▼
loss
    │
    ▼
BACKWARD PASS
stored / recomputed information helps calculate gradients

Training systems can trade memory against extra computation—for example, by recomputing some activations rather than retaining all of them.

Why Multiple GPUs Become Essential

Large-scale training can require more model state, activation memory, optimizer state, and computation than one accelerator can provide.

So the multi-GPU ideas we already studied return in a more demanding form.

training batch
    │
    ▼
DISTRIBUTED TRAINING SOFTWARE
    │
    ├──→ GPU 0
    ├──→ GPU 1
    ├──→ GPU 2
    └──→ GPU N

gradients / parameters / activations / optimizer state
may require coordinated communication

At that point, interconnect bandwidth and synchronization are again part of the mathematical machine.

Training Data Does Not Become a Filing Cabinet

It is tempting to imagine training as copying documents into millions or billions of little storage locations inside the model.

That is not a good physical picture.

Training repeatedly changes numerical parameters according to an optimization objective.

The resulting model can encode statistical patterns and relationships in its parameter structure, but ordinary weights are not organized as a readable archive of source documents.

This does not mean models can never reproduce memorized material. It means the basic training mechanism is parameter optimization, not document filing.

When Training Ends

Eventually, a chosen model state is saved.

Training updates parameters

↓

Evaluation / selection

↓

Model checkpoint saved

↓

Weights stored persistently

↓

Later loaded for inference

Now we have returned to the beginning of our HAAW inference story: numerical model data sitting on persistent storage, waiting for software and hardware to load and execute it.

The Circle Closes

TRAINING DATA
    ↓
forward passes
    ↓
loss
    ↓
backpropagation
    ↓
optimizer updates
    ↓
TRAINED WEIGHTS
    ↓
saved model
    ↓
loaded by inference runtime
    ↓
PROMPT → TOKENS → TENSORS → TRANSFORMER → LOGITS → TOKEN

The inference machine we spent so many pages opening is the downstream result of this earlier optimization process.

Training Changes the Matrices

The phrase is deliberately simple, but now we know what sits behind it.

A forward pass measures what the current model does.

A loss function turns performance into an optimization objective.

Backpropagation calculates gradients through the model's operations.

An optimizer uses those gradients to update trainable parameters.

The cycle repeats until the resulting parameter state becomes the trained model we later use for inference.

The weights did not appear by magic. They are the surviving numerical result of an enormous iterative computation.

HAAW Checkpoint

Training: The optimization process that adjusts trainable model parameters using data and a defined objective.
Forward pass: Computation from model input through the network to model output and, during training, toward a loss value.
Target: The training signal representing what the model is being trained to predict or accomplish for an example.
Loss function: A mathematical function that converts model performance on the training objective into a numerical value to optimize.
Gradient: Derivative information describing how the loss changes locally with respect to trainable parameters.
Backpropagation: The efficient application of the chain rule through the computation graph to calculate gradients for parameters.
Optimizer: The algorithm that uses gradients and often additional state to determine parameter updates.
Learning rate: A parameter controlling the scale of optimization updates.
Training step: A cycle involving a batch, forward computation, loss, backward computation, and parameter update.
Checkpoint: A saved state of model parameters—and, during training, potentially other training state—from which work can be evaluated, deployed, or resumed.
Training versus inference: Training changes learned parameters. Standard inference normally holds those parameters fixed and uses them to process new input.

Next HAAW Page: Backpropagation — Following the Error Back Through the Network