Agent Memory

An agent needs memory to know what it has already done in a task, and sometimes to remember useful facts between tasks.

Two Kinds of Memory

Remember: the model itself does not remember anything between requests. Every kind of memory is something your code stores and sends back.

Short-Term Memory

During a task, every step (the model's thoughts, tool calls and results) is added to the message list. That list is the agent's working memory.

The problem: long tasks create a lot of text, and the context window is limited. Tool results like web pages can be very long.

A common fix is to summarise older steps:

steps = [
    "Searched 'Karachi to Lahore train' and found 4 trains.",
    "Read Green Line details: departs 4 pm, Rs 6,000 economy.",
    "Read Karakoram Express: departs 3 pm, Rs 5,500 economy.",
    "Read Tezgam: departs 6 pm, Rs 4,800 economy.",
]
MAX_STEPS = 2

if len(steps) > MAX_STEPS:
    old, recent = steps[:-MAX_STEPS], steps[-MAX_STEPS:]
    summary = f"Summary of {len(old)} earlier steps: found trains, Green Line Rs 6,000."
    steps = [summary] + recent

for s in steps:
    print("-", s)

- Summary of 2 earlier steps: found trains, Green Line Rs 6,000.
- Read Karakoram Express: departs 3 pm, Rs 5,500 economy.
- Read Tezgam: departs 6 pm, Rs 4,800 economy.

In a real agent, the summary is written by the model itself ("Summarise these steps in 2 sentences, keep all prices").

Long-Term Memory

Long-term memory survives after the task ends. The simplest version is a JSON file:

import json, os

MEMORY_FILE = "memory.json"

def remember(key, value):
    data = {}
    if os.path.exists(MEMORY_FILE):
        with open(MEMORY_FILE) as f:
            data = json.load(f)
    data[key] = value
    with open(MEMORY_FILE, "w") as f:
        json.dump(data, f)

def recall():
    if not os.path.exists(MEMORY_FILE):
        return {}
    with open(MEMORY_FILE) as f:
        return json.load(f)

remember("language", "Roman Urdu")
remember("city", "Hyderabad")
print(recall())
os.remove(MEMORY_FILE)   # clean up for this example

{'language': 'Roman Urdu', 'city': 'Hyderabad'}

At the start of a new task, the agent's code loads these facts and adds them to the system prompt: "Known about this user: prefers Roman Urdu, lives in Hyderabad."

Memory as a Tool

Another approach is to give the agent tools such as save_note(text) and search_notes(query). The model decides what is worth remembering and looks it up when needed. For many notes, search uses embeddings, like the RAG idea in the Chatbots tutorial.

Memory Rules

• Save only useful facts: preferences, decisions, results, not every message.
• Let users see and delete what is remembered about them.
• Never store secrets like passwords or card numbers in agent memory.
• Expire old facts: a price from last year may be wrong today.

Exercise