Data & Training

A model doesn't start out knowing anything. It starts as random numbers — billions of parameters set to noise. Training is the process that turns that noise into something useful. It takes months, costs millions, and when it's done the model is frozen. What it learned during training is all it will ever know — until the next training run.

The Raw Material — Data

Before training begins, you need data — enormous amounts of it. A frontier language model is trained on trillions of tokens of text: web pages, books, code, scientific papers, forums, documentation. The Common Crawl dataset alone — a snapshot of a large portion of the public web — contains petabytes of text.

Data quality matters as much as quantity. Raw web data is noisy — spam, duplicates, low-quality content, toxic material. A significant portion of AI development work happens before training starts: filtering, deduplicating, and cleaning the dataset. Garbage in, garbage out applies to AI just as it does to any database.

Think of training data like a log archive: You have petabytes of raw logs. Before you can analyze them usefully you need to filter out noise, deduplicate entries, normalize formats, and remove corrupted records. The AI data pipeline does the same thing — at a scale measured in trillions of records rather than billions.

Three Training Phases

Phase 1

Pre-training

The model reads the entire dataset and learns to predict the next token in a sequence. No right answers are given — just patterns in the data. This is where most of the compute and cost lives.

Phase 2

Fine-tuning

The pre-trained model is further trained on curated, high-quality examples of the specific behavior wanted — following instructions, answering questions, writing code. Smaller dataset, less compute.

Phase 3

RLHF 📖 Glossary

Reinforcement Learning from Human Feedback. Human raters compare model outputs and score them. The model learns to produce responses humans prefer. This is what makes Claude helpful rather than just fluent.

How Pre-training Works

Pre-training is the core of the process. The model sees a sequence of tokens — words, parts of words — and tries to predict what comes next. It gets scored on how well it predicted. The score drives an adjustment to the model's weights. Repeat billions of times.

This is called gradient descent. The model is always moving its weights in the direction that reduces its prediction error. Over billions of steps across trillions of tokens, the weights converge toward values that encode genuinely useful knowledge about language, facts, reasoning, and structure.

Nobody programs the model to know that Paris is the capital of France, or that a function should return a value, or that a polite email ends with a closing. It infers all of that from patterns in the data.

The sysadmin analogy: Imagine tuning a system by running millions of benchmarks and adjusting one parameter slightly after each run, always in the direction that improves the score. That's gradient descent — except the system has billions of parameters and the benchmark is "predict the next word correctly." After enough iterations, the parameters settle into values that work remarkably well.

What RLHF Actually Does 📖 Glossary

A model that has only done pre-training and fine-tuning is fluent but unpredictable. It might answer a question helpfully, or it might generate something plausible-sounding but wrong, or offensive, or unhelpful.

RLHF is the alignment step. Human raters are shown pairs of model responses and asked to pick the better one. Those preferences train a separate reward model — a model that has learned to score responses the way a human rater would. The main model is then trained to maximize that reward score.

The result is a model that not only knows things but behaves in ways humans actually find useful — it follows instructions, stays on topic, acknowledges uncertainty, and declines harmful requests. That behavior came from RLHF, not from pre-training.

The Training Pipeline 📖 Glossary

Raw data collection — web crawl, books, code, papers

Data cleaning — filter, deduplicate, normalize, remove toxic content

Tokenization — text converted to token IDs the model can process

Pre-training — billions of gradient descent steps across the full dataset

Fine-tuning — curated examples shape specific behaviors

RLHF — human feedback trains the reward model; model learns to maximize it 📖 Glossary

Evaluation & safety testing — red-teaming, benchmarks, capability assessment

Deployment — weights frozen, model moves to inference infrastructure

When Training Ends — The Frozen Model

When training completes, the model weights are frozen. The model is now a static artifact — a very large file of numbers that encodes everything it learned. It will not learn anything new from conversations with users. It will not remember what you told it last week.

This is one of the most important things to understand about AI: inference is not learning. When you send a prompt, the model is executing — running the frozen weights against your input to generate a response. No weights change. Nothing is retained after the conversation ends.

New knowledge requires a new training run — which is why models have a knowledge cutoff date, and why major AI labs release new model versions rather than continuously updating a single model.