Middle-Out: temporal compression for AI episodic memory systems

The problem

AI assistants forget things they should know and remember things that don’t matter.

This is not a retrieval problem. It’s a compression problem. Every AI memory system eventually has to decide what to keep and what to discard, and most of them make that decision badly.

Both approaches assume memory value decays monotonically with age. That assumption is wrong.

The U-shaped retention curve

Human episodic memory doesn’t decay monotonically. Psychologists call it the reminiscence bump: people disproportionately retain memories from early in their lives and from the recent past, while the middle period collapses into generalized schemas.

The information-theoretic argument

Shannon entropy measures how surprising a piece of information is given what you already know.

Foundational memories have high entropy. They define the pattern; they were unpredictable from what came before them because they were creating a new baseline. The first time you made a particular architectural decision, the first time a failure mode surfaced, the founding insight. Novel. Pattern-setting. High entropy.

Recent memories have high entropy. They’re updating or challenging the established pattern. The most recent events are most surprising because they’re the ones you haven’t integrated yet.

Middle memories have low entropy. They’re instances of an established pattern. Predictable from what came before. The 47th time you encountered the same failure mode and applied the same fix. The 12th session where a settled design just worked as expected. Compressible.

Middle-Out exploits this asymmetry.

The algorithm

Middle-Out operates in three stages, applied to the middle band of a temporal episodic memory.

graph LR
    A[Memory Timeline] --> B[Band Detection]
    B --> C[Foundation Band]
    B --> D[Middle Band]
    B --> E[Active Band]
    D --> F[Semantic Clustering]
    F --> G[Abstractive Summarization]
    G --> H[Compressed Memory]
    C --> I[Final Memory Store]
    H --> I
    E --> I

Stage 1: Band detection. Divide the memory timeline into three bands:

• Foundation: memories with high retrieval frequency (still actively shaping behavior, regardless of age). Never compress.
• Active: the last N sessions (rolling window). Never compress.
• Middle: everything else. Compression candidates.

Stage 2: Semantic clustering. Within the middle band, cluster memories by semantic similarity. Most episodic memory systems generate embeddings on write; those embeddings are already there. Cluster them. HDBSCAN works well because it doesn’t require prespecifying cluster count. Each cluster represents a recurring theme: debugging the same failure mode, implementing the same class of solution, tracking the same evolving decision.

Stage 3: Abstractive summarization. For each cluster, generate one summary memory. Preserve: key decisions, outcomes, gotchas, and what changed. Discard: process details, intermediate states, failed attempts, redundant instances. Replace N memories with 1.

The compression ratio is determined by cluster count. Ten clusters over five hundred memories is 50:1 compression on the middle band.

The context window connection

Liu et al. (2023) showed that language models pay less attention to information in the middle of long contexts, the “lost in the middle” problem.² Models perform significantly better when relevant information appears at the beginning or end.

Implementation

The reference implementation runs against an episodic memory layer that stores structured memories with embeddings generated on write. The decay tagging system maps directly to Middle-Out bands:

The three-stage algorithm runs as a scheduled job. After each compression run, the retrieval miss rate (how often a relevant memory was compressed away) feeds back into the cluster granularity parameter. The system gets better at compression over time by measuring what it lost.

What this is not

Middle-Out is not a general compression algorithm. It’s specific to episodic memory systems with temporal structure and semantic embeddings. Don’t apply it to file storage. (Richard tried that. Different problem.)

Open questions

The cluster granularity hyperparameter is the main remaining design question. Too few clusters over-compresses and loses distinct memories that happened to be semantically adjacent. Too many clusters under-compresses and wastes the effort. The self-improving feedback loop (retrieval miss rate to cluster count adjustment) is the working answer, but the optimal initial value and learning rate are still being calibrated.

Adaptive band boundaries based on retrieval frequency are implemented but not yet validated over a full year of memory data. The hypothesis is that the Foundation band will stabilize around 5-15% of total memories for a well-established system.