The context budget
Read · 1 of 3
Every token competes
A model can only read so much at once: its context window. Agents fill it fast, because every tool result stays in the conversation.
A fuller context isn't a better one. Old, irrelevant results distract the model, slow it down and cost money. Deciding what goes in is called context engineering, and it's one of the biggest levers on agent quality.
The simplest strategy that works: always keep the original goal, keep the most recent turns, and drop the oldest middle when you're over budget.
Read this lesson as text
Keep the goal and the newest turns; drop the middle when you must.
Every token competes
A model can only read so much at once: its context window. Agents fill it fast, because every tool result stays in the conversation.
A fuller context isn't a better one. Old, irrelevant results distract the model, slow it down and cost money. Deciding what goes in is called context engineering, and it's one of the biggest levers on agent quality.
The simplest strategy that works: always keep the original goal, keep the most recent turns, and drop the oldest middle when you're over budget.
Newest first
sizes = [50, 300, 200, 100, 80]
budget = 300
kept = [0]
total = sizes[0]
for i in range(len(sizes) - 1, 0, -1):
if total + sizes[i] > budget:
break
kept.append(i)
total += sizes[i]
print(sorted(kept), total)Message 0 is the goal and always stays. What does this print?
- [0, 3, 4] 230
- [0, 1] 350
- [0, 2, 3, 4] 430
Show the answer
[0, 3, 4] 230 Start with the goal (50). Walk back from the newest: 80 fits (130), 100 fits (230), 200 would make 430, so stop. Keep messages 0, 3 and 4.
Exercise: Trim to budget
Input: the token budget, then n, then n lines id tokens (oldest first). The first message is the goal.
Always keep the goal. Then keep messages from the newest backwards while they fit; stop at the first one that doesn't.
Print the kept ids in their original order on one line, then tokens=T dropped=D.