CORE GUIDE
Attention budget
A large context window does not create unlimited useful attention: every extra token competes for model focus, latency and cost, so context must be budgeted around task-relevant evidence.
Mental model
Treat attention as a scarce allocation problem. Context capacity answers how much can fit; attention budget asks which instructions, evidence and state deserve to occupy the model's limited effective focus for this decision.
Why it matters
Teams often respond to weak answers by adding more documents, history and examples. That can make the prompt longer while making the signal weaker. Attention budgeting forces an explicit trade-off between relevance, redundancy and cost, and connects foundation-level context mechanics to production context engineering.
01
Budget context by decision value
Start from the decision the model must make, then reserve context for non-negotiable instructions, current task state and the smallest evidence set that can change that decision. Remove duplicate background, compress old conversational material and retrieve supporting detail on demand. Measure whether each added block improves the target outcome enough to justify its tokens and latency.
02
Example: a support agent carries an entire account history
A support agent receives thirty pages of customer history, product documentation and prior tool traces for every request. Refund-policy evidence is present but buried. The team instead keeps current policy, the active order, recent turns and a retrieval pointer to older history. Answer quality improves because important evidence occupies a larger share of the working context.
Common failure modes
- Equating maximum context-window size with useful task context.
- Adding more retrieved documents without measuring whether they displace higher-value evidence.
- Keeping duplicated history because removing information feels riskier than diluting attention.
Engineering heuristics
- Reserve budget first for authority, current state and decision-changing evidence.
- Prefer selective retrieval and compression over permanent prompt accumulation.
- Evaluate context blocks by marginal outcome gain, not by how informative they look in isolation.
Takeaways
- 01Context capacity is not the same as effective attention.
- 02More tokens can reduce signal even when they fit.
- 03Attention budgeting is a practical bridge from context-window mechanics to context engineering.
Reading evidence
This records actions you actually took; it does not claim mastery, proficiency, or certification.
Used in
This Concept is reused across these canonical learning paths.
Related concepts from the Knowledge Graph
These relationships come from the canonical graph, not a separate Guide taxonomy.