What Can the Model See?
For one inference operation, the model can operate on information that has been represented and supplied within its usable input context—not automatically on everything stored elsewhere in the computer, service, account, or Internet.
A chat application may know that old conversations, files, databases, tools, or user information exist. That does not mean all of those things are simultaneously inside the model's current context.
Four Ideas People Commonly Blend Together
Weights
Learned model parameters produced by training. They are reused across many different prompts.
Context
Information supplied or represented for the current inference sequence.
KV Cache
Temporary numerical attention state retained from already-processed sequence positions to make decoding efficient.
Persistent Memory / Storage
Information retained outside the immediate inference state and potentially retrieved for later use.
These can interact, but they are not interchangeable names for one mechanism.
Weights: What Training Left Behind
We already established:
↓
parameter updates
↓
trained weights
↓
saved model
The weights encode learned statistical structure in numerical parameters. During ordinary inference they are normally used rather than rewritten by the current conversation.
So when a model answers a general question using patterns learned during training, that is different from reading the answer out of the current context.
Context: What Is Present for This Inference
Suppose the current model-ready sequence conceptually contains:
APPLICATION INSTRUCTIONS
RELEVANT EARLIER CONVERSATION
RETRIEVED INFORMATION OR TOOL RESULTS
CURRENT USER MESSAGE
STRUCTURAL MARKERS
After tokenization and model-specific preparation, this information contributes to the sequence being processed.
This is much closer to what we mean by what the model can see right now.
Context Is Not Necessarily the Whole Conversation
The interface may display a long conversation, but the serving system still has to decide how relevant information is represented for a particular model call.
│
├── earlier turn
├── earlier turn
├── earlier turn
├── ...
└── current message
│
▼
CONTEXT PREPARATION
│
▼
MODEL-READY CONTEXT
Depending on the product, model, and situation, earlier information might be supplied directly, summarized, retrieved selectively, represented through another mechanism, or omitted.
The Context Window
A model architecture and serving system support a finite amount of sequence information for an inference operation.
This usable range is commonly discussed as the context window and is usually measured in tokens rather than words or characters.
instructions | prior content | retrieved data | user input | generated tokens
The exact limit and accounting rules depend on the model and service. Input and generated output can both consume sequence capacity.
Why Tokens Matter Again
Context capacity is not usually described as “twenty pages” or “one hundred paragraphs.”
It is fundamentally tied to the token sequence the model processes.
Two pieces of text with the same number of words can consume different numbers of tokens.
Context Is Working Information, Not Training
If Craig tells the model during a conversation:
and that information is available in the active context, the model can use it when answering a later question in that inference sequence.
That does not mean the model's trained weights have just been rewritten to permanently contain that fact.
NOT THE SAME AS
TRAINING → modifies learned parameters
Then What Is the KV Cache?
The KV cache is where language about “memory” becomes especially slippery.
During transformer attention, keys and values are computed for sequence positions. During autoregressive generation, retaining appropriate key/value tensors allows later decode steps to reuse those earlier computations.
│
▼
transformer attention computation
│
├──→ changing activations
└──→ retained K/V state
for later decode steps
The KV cache is therefore related to the already-processed sequence, but it should not be imagined as a little text transcript or semantic notebook stored inside VRAM.
The KV Cache Is Not the Conversation Written in GPU Memory
The KV cache contains numerical key/value tensors used by attention. It is derived from processed sequence state. It is not simply a second copy of the conversation stored as readable sentences.
The runtime can use that numerical state to avoid recomputing attention information for previous positions during each new decode step.
Retrieval: Bring Outside Information In
Suppose the system has access to a large collection of documents.
The entire collection does not have to be placed into every prompt.
This general pattern is often called retrieval-augmented generation, or RAG, when retrieval supplies external information to a generative model.
A Tool Works Similarly at the Boundary
If an AI system can use a search engine, calculator, database, file service, or another external tool, the model itself does not thereby become that tool.
│
▼
EXTERNAL TOOL
│
▼
tool produces result
│
▼
result is represented for subsequent model processing
Again, information outside the model becomes useful to inference when the surrounding system makes it available through an appropriate path.
Persistent Memory Is Another Layer
A product may store selected information so it can be used again in a future conversation or request.
│
▼
PERSISTENT EXTERNAL STORAGE / MEMORY SYSTEM
│
│ later retrieval / selection
▼
future request context
│
▼
MODEL INFERENCE
The exact mechanism is product-specific, but conceptually this differs from both the model's trained weights and the temporary KV cache.
Now We Can Untangle “Remember”
Learned During Training
Information is reflected statistically in the trained model parameters.
Present in Context
Information is available in the current model input sequence.
Retained in KV Cache
Numerical attention state from already-processed sequence positions is available for efficient continued decoding.
Retrieved From Storage
An external system preserved information and later supplied relevant material for a new inference.
In ordinary conversation people may call all four “remembering.” Technically, they describe different mechanisms.
What Happens When Context Gets Large?
More context means more sequence state to process and manage.
Depending on architecture and implementation, longer context can increase prefill work, KV-cache memory requirements, and other computational costs.
│
├──→ more input to process
├──→ more attention state
├──→ larger KV-cache demand
└──→ additional runtime / memory pressure
Large context windows are therefore not merely a user-interface feature. They have consequences down in the machinery we have already studied.
Context Reaches All the Way Down to VRAM
This is where several HAAW pages reconnect.
Something that looks like “just more text” at the interface becomes tokens, tensors, computation, and memory usage lower in the stack.
What the Model Cannot Automatically See
Unless the surrounding system provides an appropriate access path, the current inference should not be imagined as automatically seeing:
Every File
Files existing on storage are not automatically model context.
Every Old Conversation
Stored conversation history is not automatically identical to the active sequence.
The Whole Internet
Web access requires retrieval or tool mechanisms; the live Internet is not simply contained in the context window.
All Machine Memory
RAM and VRAM contain many things. Presence somewhere in memory does not make information model input.
“Can See” Has a Technical Meaning
For HAAW, we can now use careful language:
The weights provide learned model state.
The context provides information for the current inference.
The KV cache preserves useful numerical state from already-processed sequence positions during generation.
Retrieval and persistent memory systems can bring information from outside the immediate inference into a future context.
Once these boundaries are visible, “the AI remembers” stops being one mysterious mechanism and becomes several understandable pieces of machinery.
HAAW Checkpoint
- Context
- Information represented and made available for the current model inference.
- Context window
- The finite usable sequence capacity available to a model/inference configuration, generally measured in tokens.
- Weights
- Learned parameters created by training; they are distinct from the current input context.
- Conversation history
- Previously exchanged content that may be stored by an application; it influences inference only through whatever context or retrieval mechanism the system provides.
- KV cache
- Temporary numerical key/value attention state retained from processed sequence positions to support efficient autoregressive decoding.
- Retrieval
- Selecting information from an external source so relevant material can be supplied to the model for inference.
- RAG
- Retrieval-augmented generation: a general pattern in which retrieved external information is provided to a generative model.
- Persistent memory
- Information retained outside immediate inference state and potentially selected or retrieved for later use.
- Technical “seeing”
- Information being available through the model's current usable input/state—not merely existing somewhere in the surrounding computer or service.