GUIDE深度 · Deep dive
Context Budget: Input, Output and System Prompt
The context window is a shared budget for input+output. System prompts, tool schemas, retrieved chunks, history and model output all consume tokens; filling input to 100% causes truncation/errors. Leave 20-40% for output and retrieve/summarize long documents.
Allocation
System/tool prompts: fixed overhead.
Retrieved chunks: only relevant blocks.
Output: leave 20-40%.
When over budget
Summarize/trim history first.
Process in chunks and aggregate.
Reduce tool schemas/examples.
FAQ
Is 128k exactly 128000 tokens?
Usually approximate and shared by input+output; check model docs.
Does output count?
Yes; reserve room when generating long text.