Memory & RAG Foundations
Understand how memory is represented, structured, and utilized to bridge the gap between static model parameters and dynamic, lifelong agent experiences.
Short-Term Memory
Inside-TrialGoverns the immediate execution trace of the agent during a single interaction task. Typically implemented directly inside the LLM context window using scratchpads, reflection logs, or in-context planning stacks.
Long-Term Memory
Cross-TrialTranscends individual execution tasks to persist experiences across different environments, sessions, or tasks. Implemented via external databases (vectors, graphs, or relational databases) to coordinate lifelong learning.
Architectural Comparison: LLM Memory vs. Agent Memory vs. RAG
| Dimension / Feature | LLM Internal Memory | Agent Persistent Memory | Retrieval-Augmented Gen (RAG) |
|---|---|---|---|
| Architecture Residency | Model Hidden Activations & KV Cache (Transient) | External State Database/Index Layer (Persistent) | External Static Document Index / Vector Store |
| Persistence Span | Transient (Lost at context turn or session end) | Lifelong / Session-transcending (Cross-trial persistence) | Static / Read-only (Rarely modified by agent actions) |
| State Evolution | Fixed weights, dynamic KV activations during forward pass | Evolving state: writes, consolidates, forgets dynamically | Static index updates (requires batch indexing pipelines) |
| Computational Overhead | Very high (quadratic context window scaling) | Low (selective vector queries, abstracts history) | Medium (dense vector search & re-ranking) |
| Primary Cognitive Role | Local context tracking & sequence completion | Identity consistency, skill caching, long-term experience | Factual retrieval and hallucination grounding |
Retrieval-Augmented Generation (RAG) paradigms
Modular RAG
Decomposes the traditional retrieval pipeline into dedicated, independent functional modules: document parsing, indexing, retrieval, re-ranking, and context filtering. This allows developers to plug-and-play specialized algorithms at each stage of the context path.
Graph RAG
Structures external knowledge as entity-relation graphs rather than isolated text chunks. By running graph traversals, concept clustering, or PageRank-based walks, it retrieves context-aware relational sub-graphs, enabling multi-hop question answering.
Agentic RAG
Fuses retrieval into an active planning loop where the agent policy decides when to retrieve, what queries to issue, and how to verify retrieved content. It replaces static lookup with iterative querying, self-correction, and tool routing.
Context Engineering & Pruning
Context engineering treats the model's finite token window as a highly constrained computational resource. It designs layout templates, prompt hierarchies, and dynamic KV cache adapters to maximize information utilization while minimizing retrieval latency.