Lesson 11: Memory

Give your agent persistent memory that spans all conversations for a user.

Topics Covered

• History vs Memory: Two different mechanisms, often confused.
• User Memories: Facts extracted and stored by user_id.
• Automatic Extraction: The LLM decides what's worth remembering.
• Cross-Session Persistence: Memories survive script restarts and new sessions.
• Memory Injection: How stored facts reach the LLM.

History vs Memory

These terms get confused. In Agno, they're distinct mechanisms:

History (Lesson 10) replays the literal conversation. Every user message and assistant response from the current session gets prepended to the context. It's tied to session_id—different session, different history.

Memory extracts facts from conversations and stores them by user_id. The LLM analyzes what you say and decides what's worth remembering: preferences, personal info, past decisions. These facts persist across all sessions for that user.

Think of it this way:

• History is a transcript of what was said
• Memory is notes about what was learned

You can use both together. History gives continuity within a conversation. Memory gives continuity across conversations.

How Memory Works

1. User says something ("I'm allergic to shellfish")
2. After responding, Agno asks the LLM: "Are there facts worth remembering here?"
3. LLM extracts: "User is allergic to shellfish"
4. Fact gets stored in Postgres, linked to user_id
5. Next session (even with a new session_id), memories load into context
6. Agent knows about the allergy without being told again

The extraction is automatic. You don't define what's memorable—the LLM figures it out based on context.

The Code

11-memory.py

"""
Lesson 11: User Memory

Extracts and stores facts about the user (preferences, info). Unlike session
history, memories persist across ALL sessions for the same user_id. The LLM
automatically identifies facts worth remembering and retrieves them later.

Run:    uv run 11-memory.py
Try:    "I love Italian food" → Ctrl+C → re-run → "What food do I like?"

Observe in Phoenix (http://localhost:6006):
- Memory extraction calls after user messages
- Stored memories injected into system prompt
- New session_id but same user memories

Reset:  uv run tools/reset_data.py
"""

import os
from dotenv import load_dotenv
from phoenix.otel import register
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.db.postgres import PostgresDb

load_dotenv()

register(project_name="11-memory", auto_instrument=True, batch=True, verbose=True)

db = PostgresDb(db_url="postgresql+psycopg://ai:ai@localhost:5532/ai")

agent = Agent(
    name="Assistant",
    model=OpenAIChat(id=os.getenv("OPENAI_MODEL_ID")),
    instructions="You are a helpful assistant. Be concise.",
    db=db,
    user_id="demo-user",              # fixed user = memories persist
    enable_user_memories=True,        # extract and store facts about user
    add_history_to_context=True,
    num_history_runs=5,
    add_datetime_to_context=True,
    markdown=True,
)

agent.cli_app(stream=True)

What's New

User identification:

user_id="demo-user",

Links memories to this user. Different user_id = different memory store. In production, this would be your user's actual ID.

Memory extraction:

enable_user_memories=True,

Tells Agno to run memory extraction after each interaction. The LLM analyzes the conversation and stores relevant facts.

Notice we still have add_history_to_context=True. History and memory work together—history for this session's transcript, memory for long-term facts.

Try It

uv run 11-memory.py

> I love Italian food and I'm learning Spanish
Got it! Italian food lover learning Spanish.

> My daughter's name is Emma
That's a lovely name!

> Ctrl+C (stop script)

Now restart—new session, but same user:

uv run 11-memory.py

> What do you know about me?
Based on what you've told me: you love Italian food, you're learning Spanish, and your daughter's name is Emma.

The agent remembers facts from the previous session. It doesn't have the conversation history (that was a different session), but it has the extracted memories.

Observe in Phoenix

Open http://localhost:6006 and look at traces for 11-memory.

You'll see additional LLM calls that weren't in previous lessons:

• Memory extraction: After your message, a separate call asks the LLM what facts to remember
• Memory retrieval: Before responding, existing memories are loaded and injected

Look at the system prompt in any trace—you'll see stored memories included alongside your instructions.

What Gets Stored

Agno creates a memories table in Postgres:

-- View stored memories for a user
SELECT * FROM agno_memories WHERE user_id = 'demo-user';

Each row is a fact the LLM decided was worth keeping. The extraction prompt guides what's memorable—typically preferences, personal info, decisions, and context that might be useful later.

History + Memory Together

This agent uses both:

Mechanism	Tied To	Contains	Loaded When
History	session_id	Full message transcript	Same session continues
Memory	user_id	Extracted facts	Any session for this user

Within a session, the agent has both the conversation transcript (history) and accumulated facts (memory). Across sessions, only memory persists—unless you use a fixed session_id like in Lesson 10b.

Memory Limitations

Memory extraction costs extra LLM calls. Each interaction triggers an extraction check, which adds latency and token usage.

The LLM decides what's memorable, which means:

• It might miss things you consider important
• It might remember things you'd rather it forget
• Quality depends on the model's judgment

For production systems, you might want explicit memory management—letting users view, edit, or delete stored memories.

Key Concepts

Concept	This Lesson
user_id	Links memories across sessions
enable_user_memories	Turns on automatic fact extraction
Memory extraction	LLM decides what's worth storing
Memory injection	Stored facts added to context

What's Next

The agent now remembers conversations (history) and learns facts about users (memory). In Lesson 12, we give it the ability to take actions—calling functions, querying databases, and interacting with external systems.