What is a context window, and why did it forget my name
A model has no memory between requests. Watch your app resend the whole chat every turn, fill the window, and drop the very turn that held your name.
One request is the model's whole world
A model keeps nothing between requests. Everything it can see for this answer arrives inside this one request, and it has to fit the window.
Step 1 of 6
Fourteen turns into a chat about your database, it asks your name. You told it in the first message. It did not forget; it was never sent that message this time. Step through the scene above to watch your name go in, get resent thirteen times, and fall off the front.
The request is the memory
A language model keeps nothing between requests. Every turn, your application bundles the system prompt, the entire conversation so far, and the new message, and sends the whole thing again. Turn 14 ships turns 1 to 13 with it, replies included. The context window is the limit on that bundle: the most tokens one request may contain, input and output together.
{
"system": "You are a helpful assistant.",
"messages": [
{ "role": "user", "content": "Hi, I'm Priya. Question about indexes..." },
{ "role": "assistant", "content": "Hi Priya! ..." },
{ "role": "user", "content": "..." }
]
}
That array grows on every turn, and so does the bill, because every token in it is charged again each time it is sent. A long conversation is not expensive once at the end. It gets more expensive on every single turn.
Something has to go, and your app chooses what
Push the bundle past the window and the model does not shorten it for you. The request comes back as an error: prompt too long, no reply. So the application trims first, and the usual rule is drop from the oldest end. The oldest end is turn 1. Turn 1 is your name.
// the crude version most apps start with
while (countTokens(messages) > WINDOW - RESERVED_FOR_REPLY) {
messages.shift(); // drop the oldest turn. Whatever it held.
}
The fix is not a bigger window; that only moves the day this happens. The fix is deciding what must survive and carrying it on purpose: a line in the system prompt, or a summary your app writes and re-sends in place of the turns it dropped. Pinning the system prompt alone does not save your name, because your name was never in it.
Two things worth knowing on sight. Being inside the window is not the same as being read: a published study found that an instruction buried in the middle of a long context was recalled worse than one at either end, and in some cases worse than giving the model nothing at all. And a pasted file is not a special kind of input that lives somewhere else; it is more tokens in the same bundle, competing with the conversation for the same room. The lesson lets you type your own name into turn one, watch the bar fill, and find the exact turn on which it dies.
Common questions
- Does the model remember earlier conversations?
- No. A request is the model's whole world; nothing carries over from the last one. When a chat product seems to remember, it is the application re-sending history, or writing notes into the prompt, not the model holding on to anything.
- What happens if I send more than the window holds?
- The request is refused with an error saying the prompt is too long, and no reply is generated. Nothing is truncated for you. Any trimming you have seen was a decision your application made to avoid that error.
- Will a bigger context window fix this?
- It delays it. A larger window means more turns fit before something has to go, and it does nothing about the two other costs. Every turn is billed for everything before it, and things buried in the middle of a long context are measurably worse recalled than things at the ends.
- Is the reply counted against the window too?
- Yes. The model's own output has to fit inside the same limit, and on the next turn that reply is part of the history you send back, so it counts twice over the life of a conversation.
Keep reading
Temperature vs top-p, watch the sampling change
The model hands over odds; code after it draws a token. Watch temperature reshape those odds without reordering them, and top-p cut the tail out of the draw.
Blocking vs non blocking in Node, watch the thread pool
One readFileSync freezes every user on your server. Watch the thread stop on one line, then hand the read to a worker and answer everyone else meanwhile.
401 vs 403, the difference in one request
401 means the server doesn't know who you are; 403 means it does, and the answer is no. Watch one request hit both gates.