> ## Documentation Index > Fetch the complete documentation index at: https://docs.rkat.ai/llms.txt > Use this file to discover all available pages before exploring further. # Memory and compaction > Automatic context compaction and semantic memory for long-running conversations Two systems handle conversations that exceed model context limits: 1. **Context compaction** -- automatically summarizes conversation history when the context window fills up, preserving recent turns and discarding older ones. 2. **Semantic memory** -- indexes discarded messages so the agent can retrieve past context on demand via the `memory_search` tool. Together they allow agents to maintain coherent multi-turn sessions that exceed any single model's context limit. ## What this guide is for Use this guide when you want to understand or configure: * long-horizon conversation behavior * semantic recall via `memory_search` * compaction thresholds * the relationship between compaction and semantic memory ## Feature flags Compaction and semantic memory have separate build-time and runtime switches. | Feature flag | Cargo feature | What it enables | | --------------------- | ---------------------- | -------------------------------------------------------------------------------------- | | Compaction strategy | `session-compaction` | `DefaultCompactor` in `meerkat-session` | | Memory store backend | `memory-store` | CLI convenience feature enabling the memory store backend and session wiring | | Memory session wiring | `memory-store-session` | `HnswMemoryStore`, `memory_search`, and memory integration with the agent/session path | The CLI's default features include the memory backend/session wiring, but **not** `session-compaction`. For no-default/custom source builds, enable the memory features explicitly when you need `memory_search`. ```bash theme={null} ./scripts/repo-cargo build -p rkat --features session-compaction,memory-store ``` At runtime, `AgentFactory::memory(true)` enables the semantic memory path (`HnswMemoryStore` + `memory_search`). Compaction is controlled independently by whether the binary is built with `session-compaction` and whether a compactor is wired into the session path. ```rust theme={null} let factory = AgentFactory::new(store_path) .builtins(true) .memory(true); // enable semantic memory // Per-request override: let mut build_config = AgentBuildConfig::new("gpt-5.6-sol"); build_config.override_memory = ToolCategoryOverride::Enable; ``` That means these cases are all valid: * compaction enabled, semantic memory disabled * semantic memory enabled, compaction unavailable * both enabled together ## CompactionConfig Controls when and how compaction runs. | Field | Type | Default | Description | | ------------------------------- | ------------- | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `auto_compact_threshold` | `u64` | model-aware; raw struct default `100_000` | Token high-water mark. If config leaves it implicit, `AgentFactory` uses four-fifths of the selected catalog model's context window. An explicit config or per-build override wins. | | `max_request_bytes` | `Option` | `None` | Optional host request-body cap. Byte compaction triggers at four-fifths of the stricter available cap: this value or an exact provider-reported cap. | | `recent_turn_budget` | `usize` | `4` | Maximum number of recent complete conversational turns to retain. A turn starts only at a conversational User message; typed injected context stays attached to that turn. Capacity recovery may retain fewer turns. | | `max_summary_tokens` | `u32` | `4096` | Maximum tokens the LLM may produce for the compaction summary. | | `min_turns_between_compactions` | `u32` | `3` | Minimum turns that must elapse between consecutive compactions (loop guard to prevent runaway compaction). | ## How compaction triggers Compaction is checked at every turn boundary, just before the next LLM call. The decision flow: The first turn always skips compaction. Ordinary compaction attempts respect `min_turns_between_compactions`. Current-history or request-byte capacity recovery bypasses this cost guard. Compaction triggers if ANY of: * `last_input_tokens >= auto_compact_threshold` (input tokens from the last LLM response), OR * `estimated_history_tokens >= auto_compact_threshold` (a conservative current-history estimate: text and serialized structure use a bytes/4 heuristic, while images and video use media-aware estimates), OR * the measured request budget for this boundary crosses the threshold on its input side. This is the same measurement the pre-dispatch preflight performs - fully hydrated request messages, the exact visible tool definitions, and the effective output reserve - so tool schemas and hydrated media, which the provider counts but a transcript-only estimate cannot see, are visible to the trigger. It is available whenever the active model declares a context window, OR * request bytes cross four-fifths of the effective byte cap. An exact provider-lowered witness is preferred; otherwise an explicitly configured `max_request_bytes` is compared with the conservative transcript request estimate. ## What happens during compaction When compaction triggers: Emitted with input/estimated token counts and message count. Without a curator, a provider-safe projection of the current history plus the compaction prompt is sent to the LLM with no tools and `max_summary_tokens` as the response limit. Media and reasoning are projected for summarization, and an oversized source projection is bounded to a head/tail handoff excerpt. With a curator, this LLM call is skipped. **On ordinary failure**: a CompactionFailed event is emitted and the session is not mutated (safe failure). **If the summarization request itself exceeds provider capacity**: the built-in path uses a deterministic mechanical handoff summary with zero provider usage so the oversized session can still make progress. This is not used for curator failures. **On success**: `DefaultCompactor::rebuild_history` produces new messages: * Every unkeyed System message and the latest version of each keyed System prompt are preserved verbatim and in source order. Superseded keyed versions are discarded. * A User-channel summary message is injected with the typed `CompactionSummary` transcript role and the rendered prefix `[Context compacted]`. * Up to `recent_turn_budget` complete turns are retained. The compactor shrinks that set when request-body pressure requires more room. * All other messages become `discarded`. If semantic memory is enabled, the runtime pairs the validated transcript rewrite with a scoped memory batch. The in-memory store publishes the pair synchronously; the durable HNSW store stages the batch under the exact rewrite identity for session-runtime commit. The session messages become authoritative only after the memory projection accepts the same rewrite. A store rejection, transcript validation error, or runtime handoff refusal preserves the original history. Compaction usage is recorded against the session and budget. A **CompactionCompleted** event is emitted with summary token count and before/after message counts. ### Host-supplied summary curation `AgentBuildConfig.compaction_curator_override` accepts an `Arc`. When present, the curator receives the typed `CompactionWindow` and produces the summary instead of the summarization LLM call. Summary usage is recorded as zero. The runtime does not fall back to an LLM if the curator fails or returns an empty summary; it emits a typed `CompactionFailed` event and preserves the original history. The curator does not control trigger timing, retained/discarded provenance, memory indexing, or transcript commit. Those remain runtime-owned and are validated before mutation. The compactor sends this prompt to the LLM: > You are performing a CONTEXT COMPACTION. Your job is to create a handoff summary so work can continue seamlessly. > > Include: > > * Current progress and key decisions made > * Important context, constraints, or user preferences discovered > * What remains to be done (clear next steps) > * Any critical data, file paths, examples, or references needed to continue > * Tool call patterns that worked or failed > > Be concise and structured. Prioritize information the next context needs to act, not narrate. ## Memory indexing after compaction When both a `Compactor` and a `MemoryStore` are wired into the agent, discarded messages are indexed into semantic memory before compacted history is committed. If the memory store rejects indexing, Meerkat preserves the original history, emits a `CompactionFailed` event, and skips that compaction attempt instead of dropping the only authoritative copy of the discarded text. For each discarded message: * The producer carries `message.indexable_content()` into a typed `MemoryIndexRequest`. The store owns the include/exclude decision; the producer does not use an empty-string convention. Compaction summaries and injected context are excluded from ordinary semantic recall. * The request carries `MemoryMetadata` containing the session ID, the typed source handle (the offset range of the source message), and a timestamp. This means previously discarded conversation content becomes searchable via the `memory_search` tool. ## The `memory_search` tool When memory is enabled, the agent gains a `memory_search` tool. ### Tool definition | Property | Value | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | `memory_search` | | Description | Session-scoped semantic recall: searches conversation content that was compacted away earlier in the current session, including turns compacted before a resume. | ### Parameters Natural language search query describing what you want to recall. Maximum number of results to return. Capped at 20. ### Response format Returns a JSON array of result objects: ```json theme={null} [ { "content": "The text content of the memory entry", "score": 0.85, "source_range": { "start": 3, "end": 4 } } ] ``` The text content of the memory entry. Similarity score from 0.0 (no match) to 1.0 (exact match). Typical useful matches are above 0.7. The half-open `[start, end)` offset range of the source message(s) the entry was indexed from. Memory is scoped to a single session; results do not carry a `session_id`, and there is no cross-session recall. ## Memory store implementations Uses: * **hnsw\_rs** (v0.3) for approximate nearest-neighbor search with cosine distance. * **SQLite** for persistent metadata and text storage. Storage layout: `{store_path}/memory/memory.sqlite3` Key characteristics: * **Embedding**: Bag-of-words TF with hash-based dimensionality reduction (4096-dimensional vectors, L2-normalized). Each word is hashed to a bucket and its presence increments that dimension. * **Persistence**: Data survives process restart. `open()` runs a one-time in-place schema migration (indexed `session_id` projection column + durable point-ID allocator table, inside one `BEGIN IMMEDIATE` transaction) and an idempotent heal of legacy rows; it scans and embeds **nothing**. A scope's HNSW graph is built lazily on first use (search / index / enumerate / drop), so opening the store no longer pays for every session in the realm. * **Scoping**: One HNSW index per session owner, built on demand from the scope's durable rows. * **Lifecycle**: `drop_scope(owner)` deletes a scope's durable rows all-or-nothing and drops its live index (dropped point IDs are never reused); `enumerate_scoped(scope, request)` pages raw scope rows in durable-id order with optional `source_range`-overlap and `indexed_after` filters (a zero `limit` is rejected with a typed error). * **Score conversion**: HNSW cosine distance (0 = identical, 2 = opposite) is converted to a 0..1 similarity score: `score = 1.0 - (distance / 2.0)`. * **Thread safety**: Point IDs are allocated transactionally from the durable allocator table (never reused, collision-free across concurrent store instances). Insertions, scope drops, and lazy scope loads are serialized via a `Mutex`; the scoped-index map sits behind a `RwLock` for concurrent searches. * **Parameters** (`HnswParams` defaults): `max_nb_connection = 16`, `max_layer = 16`, `ef_construction = 200`, `ef_search = 200`. In-memory substring matching -- not suitable for production. * Stores entries in a `Vec` behind a `RwLock`. * Search: lowercases both query and content, counts matching words, scores as `matching_words / total_query_words`. * No persistence -- data is lost when the process exits. ## How memory gets wired When the `memory-store-session` feature is compiled in and memory is enabled: 1. An `HnswMemoryStore` is opened at `{store_path}/memory/`. 2. The `memory_search` tool is added to the agent's tool set. 3. A `DefaultCompactor` is attached only if `session-compaction` is also enabled. 4. The embedded `memory-retrieval` companion skill is available in the skill inventory when its capability gate is satisfied. It is not automatically preloaded; callers or the agent can activate it through the normal typed skill paths. ## Examples ```bash theme={null} # Build with memory + compaction features ./scripts/repo-cargo build -p rkat --features session-compaction,memory-store # Enable the richer tool preset so memory_search is available rkat run --tools full "Analyze this large codebase..." ``` ```rust theme={null} use meerkat::{AgentFactory, AgentBuildConfig}; use meerkat_store; let realm = meerkat_store::realm_paths("team-alpha"); let factory = AgentFactory::new(realm.root.clone()) .runtime_root(realm.root) .project_root(".") .builtins(true) .memory(true); let mut build_config = AgentBuildConfig::new("gpt-5.6-sol"); // Memory + compaction are wired automatically when: // 1. The factory has memory(true) // 2. The binary is compiled with session-compaction + memory-store-session features let agent = factory.build_agent(build_config, &config).await?; ``` ### Custom CompactionConfig ```rust theme={null} use meerkat_core::CompactionConfig; use meerkat_session::DefaultCompactor; let compactor = DefaultCompactor::new(CompactionConfig { auto_compact_threshold: 50_000, // Compact earlier max_request_bytes: Some(9_000_000), // Optional host request-body cap recent_turn_budget: 6, // Keep more recent turns max_summary_tokens: 8192, // Allow longer summaries min_turns_between_compactions: 5, // More spacing between compactions }); ``` ## See also * [Configuration: memory and compaction](/concepts/configuration) - config file settings * [Architecture](/reference/architecture) - how compaction fits into the agent loop