Memory across chats
Remembering a peanut allergy, days later.
Every conversation starts empty. Long-term memory is just tools: one to save a fact, one to search what was saved, usually with embeddings.
- model calls
- 1
- tool calls
- 0
- tokens in
- 229
- tokens out
- 38
- time
- 2.2s
Task I'm vegetarian and allergic to peanuts. Coding marathon tonight, suggest a snack I can order.
Model call 1. It read 229 tokens and wrote 38. It asked for a tool, so it isn't done.
save_memory({"fact":"Vegetarian and allergic to peanuts"})
search_products({"query":"snack"})
stop_reason: tool_use
Context window
229 tokens sent
- Your messages21
- Tool definitions146
- System prompt62
0.02% of the 10,48,576-token window. Every call sends all of it again.
Context window229 tokens sent
- System prompt62
- Tool definitions146
- Your messages21
Days later, a brand-new chat. The model's context is empty again, but the memory store isn't:
- model calls
- 1
- tool calls
- 0
- tokens in
- 221
- tokens out
- 17
- time
- 2.7s
Task Order me something to munch during tomorrow's hackathon.
Model call 1. It read 221 tokens and wrote 17. It asked for a tool, so it isn't done.
recall({"query":"diet allergy preferences snacks"})
stop_reason: tool_use
Context window
221 tokens sent
- Your messages13
- Tool definitions146
- System prompt62
0.02% of the 10,48,576-token window. Every call sends all of it again.
Context window221 tokens sent
- System prompt62
- Tool definitions146
- Your messages13
What should an agent save to long-term memory?