Context window
The maximum number of tokens a model can take in at once, counting the prompt, any documents and its own reply.
Also: context length, context
A model can only attend to the tokens in its context window; anything beyond is invisible to it. GPT-3 had 2,048 tokens in 2020, GPT-4 offered up to 32,000 in 2023, and Google's Gemini 1.5 Pro reached one million and then two million in 2024. A million tokens is several long novels or a large codebase. Longer windows cost more memory and compute, because of attention and the KV cache, and models do not always use distant parts of a long context reliably.