- Context compaction — automatically summarizes conversation history when the context window fills up, preserving recent turns and discarding older ones.
- Semantic memory — indexes discarded messages so the agent can retrieve past context on demand via the
memory_searchtool.
What this guide is for
Use this guide when you want to understand or configure:- long-horizon conversation behavior
- semantic recall via
memory_search - compaction thresholds
- the relationship between compaction and semantic memory
Feature flags
Compaction and semantic memory have separate build-time and runtime switches.The default CLI build includes memory backend/session wiring and effective
session compaction. Its default
mob feature pulls in meerkat-mob, which
enables session-compaction on the shared facade and session crates through
Cargo feature unification. The CLI’s own session-compaction alias is not
listed directly in its defaults. For no-default/reduced builds that omit this
dependency path, enable that alias explicitly when you need DefaultCompactor,
and enable the memory features when you need memory_search.AgentFactory::memory(true) enables the semantic memory path
(HnswMemoryStore + memory_search). The effective facade
session-compaction feature installs DefaultCompactor independently of that
memory toggle. Low-level core embeddings can instead supply a Compactor;
facade builds use the factory/config pipeline rather than direct
AgentBuilder::compactor injection (on the facade builder it makes
try_build fail). Since 0.9.4 a Compactor implements
prepare_for_summarization(messages, budget) with no default: budget is the
SummarizationSourceBudget of the model that writes the summary (None when
it declares no context window), and the returned SummarizationSource states
how much of the history it carries. Returning
SummarizationSource::whole(meerkat_core::materialize_latest_system_prompt_versions(messages))
keeps the 0.9.3 behaviour. Since 0.9.9 the CompactionContext a Compactor
decides from also carries projected_input_tokens (an
InputTokenProjection: the last answered request’s count plus the estimate
of the rows appended since) and last_context_refusal_boundary_index (a
recorded context refusal or bounded request; see
How compaction triggers); a custom Compactor may
read them or ignore them, and None measures as before. A host that swaps a
durable snapshot of the session into a live agent calls
Agent::reload_compaction_cadence after the swap, so the agent adopts the
cadence the snapshot carries.
- compaction enabled, semantic memory disabled
- semantic memory enabled, compaction unavailable
- both enabled together
CompactionConfig
Controls when and how compaction runs.How compaction triggers
Compaction is checked at every pre-LLM boundary: just before each LLM call, including each call of a tool loop. The decision flow:1
Skip the first boundary
The session’s first LLM call never compacts.
2
Cadence check
Ordinary compaction attempts respect
min_turns_between_compactions,
counted in pre-LLM boundaries since the last compaction or failed attempt.
Current-history or request-byte capacity recovery bypasses this cost guard.A history crossing counts as recovery only while compacting can still get
the history under the threshold. The session records the history estimate
each rewrite leaves (last_rewrite_history_tokens in its compaction
cadence). When that is still at or above the threshold, the rows a rewrite
keeps (the System rows, the in-flight request, the recent turns and the
summary) cross it on their own, so the crossing waits for the cadence
instead of paying for a summary at every boundary, and a warning names the
remedies: raise auto_compact_threshold above that floor, omit it so it
derives from the model window, or shrink the System rows. Such a crossing
bypasses the cadence again only when the request no longer fits the model
window and the history added since that rewrite is at least the overage.
A model that declares no context window has no overage, so there such a
crossing always waits for the cadence, however much the history grew, and
only a request-byte crossing still bypasses it.A request-byte crossing keeps its bypass. With an exact provider cap the
rebuilt request must shrink and fit that cap, or the turn fails as
irreducible instead of compacting again. With the transcript estimate, a
rewrite keeps the System rows, the summary and at most two retention
budgets of verbatim rows, which stay under a byte threshold measured in
megabytes unless the System rows alone approach the provider’s request
cap.3
Threshold evaluation
Compaction triggers if ANY of:
last_input_tokens >= auto_compact_threshold(input tokens from the last LLM response), OR- the projected input of this request crosses
auto_compact_threshold: the provider’s count for the last answered request plus an estimate of only the rows appended since (a tool result, for one). The count lags by exactly those rows and the estimates below under-read dense content such as JSON, so the projection keeps the exact count and estimates only the growth. It bypasses the cadence when the growth made the crossing (the last answered request was under the threshold) and the last rewrite, if any, got the history under the threshold. Its anchor is persisted with the session, so an agent rebuilt from its store projects its first request too; a committed rewrite (a compaction or a host’s transcript edit) and a fork at a point drop it until the next answer, and the answer to a request that carried the run’s tool results shortened (see below) sets none, OR estimated_history_tokens >= auto_compact_threshold(a conservative current-history estimate: text and serialized structure use a bytes/4 heuristic, while images and video use media-aware estimates), OR- the measured request budget for this boundary crosses the threshold on its input side. This is the same measurement the pre-dispatch preflight performs - fully hydrated request messages, the exact visible tool definitions, and the effective output reserve - so tool schemas and hydrated media, which the provider counts but a transcript-only estimate cannot see, are visible to the trigger. It is available whenever the active model declares a context window, OR
- request bytes cross four-fifths of the effective byte cap. An exact
provider-lowered witness is preferred; otherwise an explicitly
configured
max_request_bytesis compared with the conservative transcript request estimate, OR - the last request did not fit: the provider refused it for not fitting the model’s context, or it carried the open run’s own tool results shortened. A refused run still fails, and the refused request reported no usage; a shortened result stays whole in the transcript. The agent records either in the session’s compaction cadence (committed with the run’s boundary, so an agent rebuilt from the store sees it too), and the next check whose compaction covers the whole transcript (the next run’s first) compacts it once, bypassing the cadence. The attempt clears the fact whether it commits or not, and so does an answered request that carried every result whole (the transcript then fits).
Compaction a host asks for
A host can ask a live session to compact now, without a rebuild and without changing any threshold:session/compact over JSON-RPC (mob/compact_member
for a mob member), Session.compact() in the SDKs, or
MeerkatMachine::request_compaction in Rust. The session’s runtime
registration records the request, and the next check consumes it: the check
before the next model request, in the running turn once its tool results are
in, or in the next turn. That check compacts without the decision flow above
(the cadence and the thresholds); what happens
during the compaction is unchanged, so a turn in flight keeps its request and
newest round, and a check with nothing to fold compacts nothing. A session’s
first model request has nothing to compact either: a request made before it
is consumed there and compacts nothing. A check that consumes the request
and compacts nothing says so: it publishes requested_compaction_skipped
with the run and the reason (first_model_request or nothing_to_fold)
instead of compaction_started. A nothing_to_fold skip records a
compaction attempt like any other, so the ordinary trigger may then wait up
to min_turns_between_compactions checks. Two requests
before a check give one compaction. The request is never persisted, and a
session with no live runtime is refused rather than loaded for it. A session
whose agent has no compactor never consumes a request: it stays requested.
What happens during compaction
When compaction triggers:1
Emit CompactionStarted event
Emitted with input/estimated token counts and message count.
2
Produce the summary
Without a curator, a provider-safe projection of the current history plus
the compaction prompt is sent to the summarizing model (the session’s
active model after any model fallback, or the admitted controller) with no
tools and
max_summary_tokens as the response limit. Media and reasoning
are projected for summarization: each image or video in a user row or tool
result becomes a placeholder such as
[image (image/png) omitted from this summary request], while the rows
the rewrite keeps carry their media unchanged. The source may take two
bytes per token of the summarizing model’s context window left after
max_summary_tokens, capped by the model’s declared input ceiling, less
the prompt, and never less than 4 KiB. For a model that declares no window
it may take a quarter of auto_compact_threshold. A larger source is
trimmed (large tool results first, then the other context, user requests
last), and every cut text is marked for the summarizer. If even the fully
trimmed source does not fit, only the newest user requests that fit are
sent, under a note that says how many of them it carries. With a curator,
this LLM call is skipped.3
Handle result
On ordinary failure: a CompactionFailed event is emitted and the session is not mutated (safe failure).If the summarization request itself exceeds provider capacity: the
built-in path can use a deterministic mechanical handoff summary with zero
provider usage for recognized capacity errors, but only when current
history has no protected, unsummarized assistant transcript observations
with source
SpokenUnmeasured. If those observations remain, the compaction
attempt emits CompactionFailed and preserves the original history rather
than mechanically discarding them. This fallback is not used for curator
failures.On success: DefaultCompactor::rebuild_history produces new messages:- Every unkeyed System message and the latest version of each keyed System prompt are preserved verbatim and in source order. Superseded keyed versions are discarded.
- A User-channel summary message is injected with the typed
CompactionSummarytranscript role and the rendered prefix[Context compacted]. - Up to
recent_turn_budgetcomplete turns are retained. The compactor shrinks that set when request-body pressure requires more room. - All other messages become
discarded.
4
Project discarded memory
If semantic memory is enabled, the runtime pairs the validated transcript
rewrite with a scoped memory batch. The in-memory store publishes the pair
synchronously; the durable HNSW store stages the batch under the exact
rewrite identity for session-runtime commit.
5
Commit the rewrite
The session messages become authoritative only after the memory projection
accepts the same rewrite. A store rejection, transcript validation error,
or runtime handoff refusal preserves the original history.
6
Record usage and emit completion event
Compaction usage is recorded against the session and budget. A
CompactionCompleted event is emitted with the summary token count, the
before/after message counts, and
summary_source: who wrote the summary
(provider_call, host_curator or mechanical_fallback) and, for a
provider call, how much of the history its source carried (whole,
trimmed with the last kind of text cut, or requests_only with how many
of the user requests it kept), and estimated_history_tokens_after: the
history estimate the rewrite left, in the unit of
estimated_history_tokens. With the durable memory store the runtime
finalizes the projection after the run’s boundary commit, so the event
follows the run’s run_completed on the session’s event stream, which an
RPC client receives through session/stream_open; a turn’s own
session/event stream has ended by then and does not carry it.Host-supplied summary curation
AgentBuildConfig.compaction_curator_override accepts an
Arc<dyn CompactionCurator>. When present, the curator receives the typed
CompactionWindow and produces the summary instead of the summarization LLM
call. Summary usage is recorded as zero. The runtime does not fall back to an
LLM if the curator fails or returns an empty summary; it emits a typed
CompactionFailed event and preserves the original history.
The curator does not control trigger timing, retained/discarded provenance,
memory indexing, or transcript commit. Those remain runtime-owned and are
validated before mutation.
The compaction prompt
The compaction prompt
The compactor sends this prompt to the LLM:
You are performing a CONTEXT COMPACTION. Your job is to create a handoff summary so work can continue seamlessly. Include:Images and videos shown as bracketed placeholders have their content omitted from this summary request. Their absence here says nothing about what the agent could see. Do not infer the agent’s ability to see them from these placeholders, and do not invent their contents; keep what the surrounding recorded text says about them and any references needed to act on them. Be concise and structured. Prioritize information the next context needs to act, not narrate.
- Current progress and key decisions made
- Important context, constraints, or user preferences discovered
- What remains to be done (clear next steps)
- Any critical data, file paths, examples, or references needed to continue
- Tool call patterns that worked or failed
Memory indexing after compaction
When both aCompactor and a MemoryStore are wired into the agent, discarded messages are indexed into semantic memory before compacted history is committed. If the memory store rejects indexing, Meerkat preserves the original history, emits a CompactionFailed event, and skips that compaction attempt instead of dropping the only authoritative copy of the discarded text.
For each discarded message:
- The producer carries
message.indexable_content()into a typedMemoryIndexRequest. The store owns the include/exclude decision; the producer does not use an empty-string convention. Compaction summaries and injected context are excluded from ordinary semantic recall. - The request carries
MemoryMetadatacontaining the session ID, the typed source handle (the offset range of the source message), and a timestamp.
memory_search tool.
With the durable HNSW store, a compaction’s staged batch becomes visible
once the runtime boundary that commits its rewrite is acknowledged: the live
agent finalizes it at that boundary, or at its startup when the session is
revived. A session no agent will host again can still owe one, for a mob
member whose stopped run compacted and that is rotated or deleted before it
wakes. PersistentSessionService::settle_owed_compaction_projections
finalizes such a session’s owed batches cold (no actor is built and no model
is called) and answers the records it published
(OwedCompactionSettlement::Settled), or Live when the session is hosted
in the process, whose agent owns the finalization. A mob’s rotation onto a
fresh session settles the session it leaves this way, and
MobHandle::settle_member_owed_compaction_projections lets a host settle a
seated member’s session, and read its discards, before it ends that session.
The memory_search tool
When memory is enabled, the agent gains a memory_search tool.
Tool definition
Parameters
Response format
Returns a JSON array of result objects:Similarity score from 0.0 (no match) to 1.0 (exact match). Typical useful matches are above 0.7.
The half-open
[start, end) offset range of the source message(s) the entry
was indexed from. Memory is scoped to a single session; results do not carry a
session_id, and there is no cross-session recall.Memory store implementations
- HnswMemoryStore (production)
- SimpleMemoryStore (test-only)
Uses:
- hnsw_rs (v0.3) for approximate nearest-neighbor search with cosine distance.
- SQLite for persistent metadata and text storage.
{store_path}/memory/memory.sqlite3Key characteristics:- Embedding: Bag-of-words TF with hash-based dimensionality reduction (4096-dimensional vectors, L2-normalized). Each word is hashed to a bucket and its presence increments that dimension.
- Persistence: Data survives process restart.
open()runs a one-time in-place schema migration (indexedsession_idprojection column + durable point-ID allocator table, inside oneBEGIN IMMEDIATEtransaction) and an idempotent heal of legacy rows; it scans and embeds nothing. A scope’s HNSW graph is built lazily on first use (search / index / enumerate / drop), so opening the store no longer pays for every session in the realm. - Scoping: One HNSW index per session owner, built on demand from the scope’s durable rows.
- Lifecycle:
drop_scope(owner)deletes a scope’s durable rows all-or-nothing and drops its live index (dropped point IDs are never reused);enumerate_scoped(scope, request)pages raw scope rows in durable-id order with optionalsource_range-overlap andindexed_afterfilters (a zerolimitis rejected with a typed error). - Score conversion: HNSW cosine distance (0 = identical, 2 = opposite) is converted to a 0..1 similarity score:
score = 1.0 - (distance / 2.0). - Thread safety: Point IDs are allocated transactionally from the durable allocator table (never reused, collision-free across concurrent store instances). Insertions, scope drops, and lazy scope loads are serialized via a
Mutex; the scoped-index map sits behind aRwLockfor concurrent searches. - Parameters (
HnswParamsdefaults):max_nb_connection = 16,max_layer = 16,ef_construction = 200,ef_search = 200.
How memory gets wired
When thememory-store-session feature is compiled in and memory is enabled:
- An
HnswMemoryStoreis opened at{store_path}/memory/. - The
memory_searchtool is added to the agent’s tool set. - A
DefaultCompactoris attached only ifsession-compactionis also enabled. - The embedded
memory-retrievalcompanion skill is available in the skill inventory when its capability gate is satisfied. It is not automatically preloaded; callers or the agent can activate it through the normal typed skill paths.
Examples
- CLI
- SDK
Custom CompactionConfig
See also
- Configuration: memory and compaction - config file settings
- Architecture - how compaction fits into the agent loop
