What Is an AI Model Context Window
The context window is the amount of text a language model "sees" in a single request: your prompt, chat history, uploaded documents, and the model's reply. Whether an entire codebase, a long contract, or only a short chat fits in one API call depends on window size. Below is what a context window is, how it is measured in tokens, and why it is one of the main parameters when choosing a model.
Definition in plain terms
A context window is the maximum number of tokens a model can process in one request. A token is neither a word nor a character: it is a text fragment after tokenization. In Russian and English, one token is roughly 3-4 characters or part of a word; "programming" may take 2-3 tokens.
The model does not "remember" past sessions on its own. Everything it knows at answer time is what you passed in the current request within the window. If the dialogue or document exceeds the limit, older fragments are cut off or must be compressed with separate techniques (RAG, summarization, chunking).
How the window works in practice
In a typical API request, context includes:
- System prompt - model instructions (role, response format, constraints).
- Message history - previous user and assistant turns in the chat.
- Attachments - text from PDFs, code from a repo, knowledge-base snippets.
- Model output - on many models, the reply also counts toward the total limit for one call.
The scheme is simple: input + output ≤ context window size (some models cap input and output separately, but the logic is the same - there is a ceiling per call).
Example for a 128K window:
| Component | Approximate size |
|---|---|
| System prompt | 500-2,000 tokens |
| Chat history (20 messages) | 5,000-15,000 |
| Uploaded document | 80,000 |
| Model reply | up to 8,000-16,000 |
If the sum exceeds the limit, the API returns an error or the platform automatically trims the start of history - depending on the client.
Input context and output context
Developers often confuse two ideas:
- Input context - how many tokens the model accepts (prompt + history + files).
- Max output tokens - upper bound on reply length per call (a separate API parameter, e.g.
max_tokens).
A model with a 1M window may accept almost a million input tokens, but the reply may still be capped at, say, 8K-64K tokens per call. Long reports sometimes need several sequential requests or streaming with continuation.
On some pricing tiers, long-context requests (above a threshold, e.g. 200K input) cost more - reflecting heavier inference load.
Why a large context window matters
Large context helps when one request must hold a lot of related information:
- Document analysis - contracts, reports, research dozens of pages long without prior compression.
- Code work - several project files, stack traces, and edit history in one prompt.
- Long dialogues - support and consulting where the full thread matters, not just the last 10 messages.
- Agentic scenarios - an agent accumulates observations, tool calls, and intermediate results in one session.
A small window (4K-32K) is enough for short tasks: classification, form field extraction, paragraph translation, a simple chatbot. Enterprise scenarios with documents and code usually target 128K and above.
Typical sizes in 2026
By mid-2026 the market splits into several tiers:
| Tier | Window size | Example models | Typical tasks |
|---|---|---|---|
| Compact | 8K-32K | Light local models, older APIs | Chat, classification |
| Standard | 128K-200K | GPT-5.5, Claude Sonnet 5, Gemini 3.1 Pro | Code, documents, agents |
| Extended | 1M-2M | GPT-5.6, Claude Fable 5, Gemini 3.5 Flash | Large repos, corpora |
Context grows faster than "useful" length: a model may technically accept a million tokens, but quality on the longest tail can drop (the "lost in the middle" effect - the model uses information from the middle of long context less well). So a large window is not a substitute for thoughtful architecture, but an expansion of capability.
Limits and workarounds
Even with a 2M window, not everything belongs in one prompt:
- Cost - input is billed by tokens; a million tokens per request quickly raises production cost.
- Latency - long context takes longer on GPU.
- Quality - relevant facts are better supplied explicitly than "buried" in the middle of 500 pages.
Practical patterns when the window is tight or for savings:
- RAG (Retrieval-Augmented Generation) - search relevant chunks in a vector store and inject only those into the prompt.
- Summarization - compress old messages or document sections with a separate model call.
- Chunking - split text into parts and aggregate results.
- Sliding window - only the last N messages plus a short summary of the past go into history.
In production, teams often combine: RAG for knowledge-base facts + moderate chat history + a 128K-1M model for "heavy" requests.
How to pick window size for your task
| Task | Recommended minimum | Comment |
|---|---|---|
| FAQ bot, intent classification | 8K-16K | Short history, documents via RAG |
| Copilot for one repo | 128K-1M | Depends on codebase size |
| Legal / compliance review | 200K+ | Long PDFs, cross-references |
| Multimodal documents | 1M+ | Text + images consume more tokens |
Before choosing a model, estimate real input size: count tokens for a typical request (via tiktoken, the API tokenizer, or the provider's count_tokens), and leave 20-30% headroom for the reply and history growth.
Summary
The context window is the language model's "working memory" for one request. It is measured in tokens and includes the prompt, history, attachments, and often space for the reply. Large windows (128K-2M) enable analysis of long documents and codebases without constant trimming, but they do not replace RAG, summarization, and cost control. When choosing a model, look not only at the spec number but also at long-context quality, long-context pricing, and the real size of your typical requests.
Frequently asked questions
How is a token different from a word?
A token is a unit of text after the model's tokenization. One word may be one or several tokens; punctuation and spaces count too. On average for Russian and English, 1000 tokens ≈ 750-900 words or 3000-4000 characters, but the exact count depends on language and model. For estimates, use the specific provider's tokenizer, not Word's word count.
What happens if text is longer than the context window?
It depends on the platform: the API may return "context length exceeded", trim the start of history (oldest messages go first), or offer compression. The model does not "read on" beyond the window - information outside the limit does not exist for it. Fixes: RAG, summarization, chunking, or a model with a larger window.
Does more context always mean better answers?
No. A large window gives the ability to pass more data, but does not guarantee the model uses it all equally well. On very long inputs, accuracy on facts from the middle often drops, and cost and latency rise. It is better to pass relevant context (via search or a structured prompt) than the entire corpus "just in case."
How do I count how many tokens my document will use?
Use official tools: tiktoken for OpenAI-compatible models, Anthropic tokenizer, Google AI Studio for Gemini, or count_tokens in the provider SDK. Rough rule: 1 page of text (≈500 words) is on the order of 650-800 tokens; code and tables may yield more tokens per character due to special symbols.
Do I need a 1M-token model for a regular chatbot?
For a typical chatbot with short replies and FAQ - no, 32K-128K plus RAG over a knowledge base is enough. A 1M+ window makes sense for large PDF analysis, whole repositories, long agentic sessions, or when data cannot be split upfront without losing coherence. First measure real request sizes in production - often 128K is enough with the right architecture.