# Context windows

> Everything the model sees on one call — prompt, history, documents and its own output — has to fit.

Concept · Models & prompting · https://levelcode.org/concepts/context-windows

A model has no memory between API calls. Everything it can use on a call must be in that call's **context window**: the system prompt, tool definitions, conversation history, any documents you paste in, **and the output it generates**. All of it is measured in [tokens](https://levelcode.org/concepts/tokens), and the window has a hard maximum, from tens of thousands to over a million tokens depending on the model.

```text
┌──────────────── context window ────────────────┐
│ system prompt │ tools │ history │ documents │ ▸ output (max_tokens) │
└───────────────────────────────────────────────────┘
```

If the input plus the `max_tokens` you reserve for the answer exceeds the window, the request fails, or the answer gets cut off when output hits its limit.

## Bigger isn't free

- **Cost and latency grow with input.** You pay for every input token on every call. A chat that resends a 50,000-token history each turn is expensive unless you use [prompt caching](https://levelcode.org/concepts/prompt-caching).
- **Attention gets diluted.** Models are better at using information near the start and end of a long context than information buried in the middle ("lost in the middle"), and quality on details drops as context grows. A focused prompt usually beats a stuffed one.
- **Long conversations accumulate junk:** old tool results, superseded plans, wrong guesses. They all compete for attention.

## Managing it

- **Retrieve instead of stuffing.** Put the few relevant passages in the context, not the whole corpus. That's what [RAG](https://levelcode.org/roadmaps/ai-engineer/rag) and [chunking](https://levelcode.org/concepts/chunking) are for.
- **Put stable content first.** Instructions and reference documents go at the top, which also makes them cacheable. Put the question at the end, close to where the model starts writing.
- **Trim or summarize history.** Keep the last few turns verbatim, replace older ones with a summary, and drop bulky tool output once it's been used.
- **Reserve output space deliberately.** Set `max_tokens` to what the task needs, and check the stop reason so you can tell a finished answer from one that hit the limit.
- **For agents, keep state outside the window.** Write notes, files or task lists the model can re-read, instead of relying on everything staying in context.

Treat the window like RAM: large but finite, with every byte paid for on every call.