Skip to main content
Two systems handle conversations that exceed model context limits:
  1. Context compaction — automatically summarizes conversation history when the context window fills up, preserving recent turns and discarding older ones.
  2. Semantic memory — indexes discarded messages so the agent can retrieve past context on demand via the memory_search tool.
Together they allow agents to maintain coherent multi-turn sessions that exceed any single model’s context limit.

What this guide is for

Use this guide when you want to understand or configure:
  • long-horizon conversation behavior
  • semantic recall via memory_search
  • compaction thresholds
  • the relationship between compaction and semantic memory

Feature flags

Compaction and semantic memory have separate build-time and runtime switches.
The default CLI build includes memory backend/session wiring and effective session compaction. Its default mob feature pulls in meerkat-mob, which enables session-compaction on the shared facade and session crates through Cargo feature unification. The CLI’s own session-compaction alias is not listed directly in its defaults. For no-default/reduced builds that omit this dependency path, enable that alias explicitly when you need DefaultCompactor, and enable the memory features when you need memory_search.
At runtime, AgentFactory::memory(true) enables the semantic memory path (HnswMemoryStore + memory_search). The effective facade session-compaction feature installs DefaultCompactor independently of that memory toggle. Low-level core embeddings can instead supply a Compactor; facade builds use the factory/config pipeline rather than direct AgentBuilder::compactor injection (on the facade builder it makes try_build fail). Since 0.9.4 a Compactor implements prepare_for_summarization(messages, budget) with no default: budget is the SummarizationSourceBudget of the model that writes the summary (None when it declares no context window), and the returned SummarizationSource states how much of the history it carries. Returning SummarizationSource::whole(meerkat_core::materialize_latest_system_prompt_versions(messages)) keeps the 0.9.3 behaviour. Since 0.9.9 the CompactionContext a Compactor decides from also carries projected_input_tokens (an InputTokenProjection: the last answered request’s count plus the estimate of the rows appended since) and last_context_refusal_boundary_index (a recorded context refusal or bounded request; see How compaction triggers); a custom Compactor may read them or ignore them, and None measures as before. A host that swaps a durable snapshot of the session into a live agent calls Agent::reload_compaction_cadence after the swap, so the agent adopts the cadence the snapshot carries.
That means these cases are all valid:
  • compaction enabled, semantic memory disabled
  • semantic memory enabled, compaction unavailable
  • both enabled together

CompactionConfig

Controls when and how compaction runs.

How compaction triggers

Compaction is checked at every pre-LLM boundary: just before each LLM call, including each call of a tool loop. The decision flow:
1

Skip the first boundary

The session’s first LLM call never compacts.
2

Cadence check

Ordinary compaction attempts respect min_turns_between_compactions, counted in pre-LLM boundaries since the last compaction or failed attempt. Current-history or request-byte capacity recovery bypasses this cost guard.A history crossing counts as recovery only while compacting can still get the history under the threshold. The session records the history estimate each rewrite leaves (last_rewrite_history_tokens in its compaction cadence). When that is still at or above the threshold, the rows a rewrite keeps (the System rows, the in-flight request, the recent turns and the summary) cross it on their own, so the crossing waits for the cadence instead of paying for a summary at every boundary, and a warning names the remedies: raise auto_compact_threshold above that floor, omit it so it derives from the model window, or shrink the System rows. Such a crossing bypasses the cadence again only when the request no longer fits the model window and the history added since that rewrite is at least the overage. A model that declares no context window has no overage, so there such a crossing always waits for the cadence, however much the history grew, and only a request-byte crossing still bypasses it.A request-byte crossing keeps its bypass. With an exact provider cap the rebuilt request must shrink and fit that cap, or the turn fails as irreducible instead of compacting again. With the transcript estimate, a rewrite keeps the System rows, the summary and at most two retention budgets of verbatim rows, which stay under a byte threshold measured in megabytes unless the System rows alone approach the provider’s request cap.
3

Threshold evaluation

Compaction triggers if ANY of:
  • last_input_tokens >= auto_compact_threshold (input tokens from the last LLM response), OR
  • the projected input of this request crosses auto_compact_threshold: the provider’s count for the last answered request plus an estimate of only the rows appended since (a tool result, for one). The count lags by exactly those rows and the estimates below under-read dense content such as JSON, so the projection keeps the exact count and estimates only the growth. It bypasses the cadence when the growth made the crossing (the last answered request was under the threshold) and the last rewrite, if any, got the history under the threshold. Its anchor is persisted with the session, so an agent rebuilt from its store projects its first request too; a committed rewrite (a compaction or a host’s transcript edit) and a fork at a point drop it until the next answer, and the answer to a request that carried the run’s tool results shortened (see below) sets none, OR
  • estimated_history_tokens >= auto_compact_threshold (a conservative current-history estimate: text and serialized structure use a bytes/4 heuristic, while images and video use media-aware estimates), OR
  • the measured request budget for this boundary crosses the threshold on its input side. This is the same measurement the pre-dispatch preflight performs - fully hydrated request messages, the exact visible tool definitions, and the effective output reserve - so tool schemas and hydrated media, which the provider counts but a transcript-only estimate cannot see, are visible to the trigger. It is available whenever the active model declares a context window, OR
  • request bytes cross four-fifths of the effective byte cap. An exact provider-lowered witness is preferred; otherwise an explicitly configured max_request_bytes is compared with the conservative transcript request estimate, OR
  • the last request did not fit: the provider refused it for not fitting the model’s context, or it carried the open run’s own tool results shortened. A refused run still fails, and the refused request reported no usage; a shortened result stays whole in the transcript. The agent records either in the session’s compaction cadence (committed with the run’s boundary, so an agent rebuilt from the store sees it too), and the next check whose compaction covers the whole transcript (the next run’s first) compacts it once, bypassing the cadence. The attempt clears the fact whether it commits or not, and so does an answered request that carried every result whole (the transcript then fits).

Compaction a host asks for

A host can ask a live session to compact now, without a rebuild and without changing any threshold: session/compact over JSON-RPC (mob/compact_member for a mob member), Session.compact() in the SDKs, or MeerkatMachine::request_compaction in Rust. The session’s runtime registration records the request, and the next check consumes it: the check before the next model request, in the running turn once its tool results are in, or in the next turn. That check compacts without the decision flow above (the cadence and the thresholds); what happens during the compaction is unchanged, so a turn in flight keeps its request and newest round, and a check with nothing to fold compacts nothing. A session’s first model request has nothing to compact either: a request made before it is consumed there and compacts nothing. A check that consumes the request and compacts nothing says so: it publishes requested_compaction_skipped with the run and the reason (first_model_request or nothing_to_fold) instead of compaction_started. A nothing_to_fold skip records a compaction attempt like any other, so the ordinary trigger may then wait up to min_turns_between_compactions checks. Two requests before a check give one compaction. The request is never persisted, and a session with no live runtime is refused rather than loaded for it. A session whose agent has no compactor never consumes a request: it stays requested.

What happens during compaction

When compaction triggers:
1

Emit CompactionStarted event

Emitted with input/estimated token counts and message count.
2

Produce the summary

Without a curator, a provider-safe projection of the current history plus the compaction prompt is sent to the summarizing model (the session’s active model after any model fallback, or the admitted controller) with no tools and max_summary_tokens as the response limit. Media and reasoning are projected for summarization: each image or video in a user row or tool result becomes a placeholder such as [image (image/png) omitted from this summary request], while the rows the rewrite keeps carry their media unchanged. The source may take two bytes per token of the summarizing model’s context window left after max_summary_tokens, capped by the model’s declared input ceiling, less the prompt, and never less than 4 KiB. For a model that declares no window it may take a quarter of auto_compact_threshold. A larger source is trimmed (large tool results first, then the other context, user requests last), and every cut text is marked for the summarizer. If even the fully trimmed source does not fit, only the newest user requests that fit are sent, under a note that says how many of them it carries. With a curator, this LLM call is skipped.
3

Handle result

On ordinary failure: a CompactionFailed event is emitted and the session is not mutated (safe failure).If the summarization request itself exceeds provider capacity: the built-in path can use a deterministic mechanical handoff summary with zero provider usage for recognized capacity errors, but only when current history has no protected, unsummarized assistant transcript observations with source SpokenUnmeasured. If those observations remain, the compaction attempt emits CompactionFailed and preserves the original history rather than mechanically discarding them. This fallback is not used for curator failures.On success: DefaultCompactor::rebuild_history produces new messages:
  • Every unkeyed System message and the latest version of each keyed System prompt are preserved verbatim and in source order. Superseded keyed versions are discarded.
  • A User-channel summary message is injected with the typed CompactionSummary transcript role and the rendered prefix [Context compacted].
  • Up to recent_turn_budget complete turns are retained. The compactor shrinks that set when request-body pressure requires more room.
  • All other messages become discarded.
4

Project discarded memory

If semantic memory is enabled, the runtime pairs the validated transcript rewrite with a scoped memory batch. The in-memory store publishes the pair synchronously; the durable HNSW store stages the batch under the exact rewrite identity for session-runtime commit.
5

Commit the rewrite

The session messages become authoritative only after the memory projection accepts the same rewrite. A store rejection, transcript validation error, or runtime handoff refusal preserves the original history.
6

Record usage and emit completion event

Compaction usage is recorded against the session and budget. A CompactionCompleted event is emitted with the summary token count, the before/after message counts, and summary_source: who wrote the summary (provider_call, host_curator or mechanical_fallback) and, for a provider call, how much of the history its source carried (whole, trimmed with the last kind of text cut, or requests_only with how many of the user requests it kept), and estimated_history_tokens_after: the history estimate the rewrite left, in the unit of estimated_history_tokens. With the durable memory store the runtime finalizes the projection after the run’s boundary commit, so the event follows the run’s run_completed on the session’s event stream, which an RPC client receives through session/stream_open; a turn’s own session/event stream has ended by then and does not carry it.

Host-supplied summary curation

AgentBuildConfig.compaction_curator_override accepts an Arc<dyn CompactionCurator>. When present, the curator receives the typed CompactionWindow and produces the summary instead of the summarization LLM call. Summary usage is recorded as zero. The runtime does not fall back to an LLM if the curator fails or returns an empty summary; it emits a typed CompactionFailed event and preserves the original history. The curator does not control trigger timing, retained/discarded provenance, memory indexing, or transcript commit. Those remain runtime-owned and are validated before mutation.
The compactor sends this prompt to the LLM:
You are performing a CONTEXT COMPACTION. Your job is to create a handoff summary so work can continue seamlessly. Include:
  • Current progress and key decisions made
  • Important context, constraints, or user preferences discovered
  • What remains to be done (clear next steps)
  • Any critical data, file paths, examples, or references needed to continue
  • Tool call patterns that worked or failed
Images and videos shown as bracketed placeholders have their content omitted from this summary request. Their absence here says nothing about what the agent could see. Do not infer the agent’s ability to see them from these placeholders, and do not invent their contents; keep what the surrounding recorded text says about them and any references needed to act on them. Be concise and structured. Prioritize information the next context needs to act, not narrate.

Memory indexing after compaction

When both a Compactor and a MemoryStore are wired into the agent, discarded messages are indexed into semantic memory before compacted history is committed. If the memory store rejects indexing, Meerkat preserves the original history, emits a CompactionFailed event, and skips that compaction attempt instead of dropping the only authoritative copy of the discarded text. For each discarded message:
  • The producer carries message.indexable_content() into a typed MemoryIndexRequest. The store owns the include/exclude decision; the producer does not use an empty-string convention. Compaction summaries and injected context are excluded from ordinary semantic recall.
  • The request carries MemoryMetadata containing the session ID, the typed source handle (the offset range of the source message), and a timestamp.
This means previously discarded conversation content becomes searchable via the memory_search tool. With the durable HNSW store, a compaction’s staged batch becomes visible once the runtime boundary that commits its rewrite is acknowledged: the live agent finalizes it at that boundary, or at its startup when the session is revived. A session no agent will host again can still owe one, for a mob member whose stopped run compacted and that is rotated or deleted before it wakes. PersistentSessionService::settle_owed_compaction_projections finalizes such a session’s owed batches cold (no actor is built and no model is called) and answers the records it published (OwedCompactionSettlement::Settled), or Live when the session is hosted in the process, whose agent owns the finalization. A mob’s rotation onto a fresh session settles the session it leaves this way, and MobHandle::settle_member_owed_compaction_projections lets a host settle a seated member’s session, and read its discards, before it ends that session.

The memory_search tool

When memory is enabled, the agent gains a memory_search tool.

Tool definition

Parameters

string
required
Natural language search query describing what you want to recall.
integer
default:"5"
Maximum number of results to return. Capped at 20.

Response format

Returns a JSON array of result objects:
string
The text content of the memory entry.
number
Similarity score from 0.0 (no match) to 1.0 (exact match). Typical useful matches are above 0.7.
object
The half-open [start, end) offset range of the source message(s) the entry was indexed from. Memory is scoped to a single session; results do not carry a session_id, and there is no cross-session recall.

Memory store implementations

Uses:
  • hnsw_rs (v0.3) for approximate nearest-neighbor search with cosine distance.
  • SQLite for persistent metadata and text storage.
Storage layout: {store_path}/memory/memory.sqlite3Key characteristics:
  • Embedding: Bag-of-words TF with hash-based dimensionality reduction (4096-dimensional vectors, L2-normalized). Each word is hashed to a bucket and its presence increments that dimension.
  • Persistence: Data survives process restart. open() runs a one-time in-place schema migration (indexed session_id projection column + durable point-ID allocator table, inside one BEGIN IMMEDIATE transaction) and an idempotent heal of legacy rows; it scans and embeds nothing. A scope’s HNSW graph is built lazily on first use (search / index / enumerate / drop), so opening the store no longer pays for every session in the realm.
  • Scoping: One HNSW index per session owner, built on demand from the scope’s durable rows.
  • Lifecycle: drop_scope(owner) deletes a scope’s durable rows all-or-nothing and drops its live index (dropped point IDs are never reused); enumerate_scoped(scope, request) pages raw scope rows in durable-id order with optional source_range-overlap and indexed_after filters (a zero limit is rejected with a typed error).
  • Score conversion: HNSW cosine distance (0 = identical, 2 = opposite) is converted to a 0..1 similarity score: score = 1.0 - (distance / 2.0).
  • Thread safety: Point IDs are allocated transactionally from the durable allocator table (never reused, collision-free across concurrent store instances). Insertions, scope drops, and lazy scope loads are serialized via a Mutex; the scoped-index map sits behind a RwLock for concurrent searches.
  • Parameters (HnswParams defaults): max_nb_connection = 16, max_layer = 16, ef_construction = 200, ef_search = 200.

How memory gets wired

When the memory-store-session feature is compiled in and memory is enabled:
  1. An HnswMemoryStore is opened at {store_path}/memory/.
  2. The memory_search tool is added to the agent’s tool set.
  3. A DefaultCompactor is attached only if session-compaction is also enabled.
  4. The embedded memory-retrieval companion skill is available in the skill inventory when its capability gate is satisfied. It is not automatically preloaded; callers or the agent can activate it through the normal typed skill paths.

Examples

Custom CompactionConfig

See also