Memory & compaction

For the full guide, see Memory.

The runtime toggles shown below still depend on a memory-enabled build. If the binary was compiled without the relevant memory features, enable_memory and --tools full do not magically add semantic memory on their own.

Enable semantic memory

When memory is enabled, discarded conversation turns are indexed into a semantic store. The agent gains a memory_search tool to recall past context on demand.

rkat run "Analyze the full codebase" \
  --tools full

{
  "jsonrpc": "2.0", "id": 1,
  "method": "session/create",
  "params": {
    "prompt": "Analyze the full codebase",
    "enable_memory": true,
    "enable_builtins": true
  }
}

curl -X POST http://127.0.0.1:8080/sessions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Analyze the full codebase",
    "enable_memory": true,
    "enable_builtins": true
  }'

{
  "name": "meerkat_run",
  "arguments": {
    "prompt": "Analyze the full codebase",
    "enable_builtins": true,
    "enable_memory": true
  }
}

result = await client.create_session(
    "Analyze the full codebase",
    enable_memory=True,
    enable_builtins=True,
)

const result = await client.createSession("Analyze the full codebase", {
  enableMemory: true,
  enableBuiltins: true,
});

let factory = AgentFactory::new(store_path)
    .builtins(true)
    .memory(true);

let agent = factory
    .build_agent(AgentBuildConfig::new("gpt-5.6-sol"), &config)
    .await?;

Memory search

When memory is enabled, the agent automatically gains the memory_search tool. It calls this tool when it needs to recall information from earlier, compacted-away turns. The tool call appears in the event stream like any other tool invocation:

{
  "type": "tool_call_requested",
  "name": "memory_search",
  "args": {
    "query": "database migration steps from earlier",
    "limit": 5
  }
}

Results are returned as a JSON array with similarity scores and the typed source handle (the half-open offset range of the source message(s) the entry was indexed from). Memory is scoped to a single session; results do not carry a session_id, and there is no cross-session recall:

[
  {
    "content": "We agreed to use sequential migrations with...",
    "score": 0.87,
    "source_range": { "start": 3, "end": 4 }
  }
]

Scores range from 0.0 (no match) to 1.0 (exact match). Useful results are typically above 0.7.

Compaction

Context compaction triggers automatically when the conversation history exceeds a token threshold. The compactor summarizes older turns, keeps recent ones, and indexes discarded messages into memory. The compaction cycle emits events into the session stream:

// CompactionStarted -- context has grown past threshold
{
  "type": "compaction_started",
  "input_tokens": 105000,
  "estimated_history_tokens": 112000,
  "message_count": 48
}

// CompactionCompleted -- history rebuilt with summary
{
  "type": "compaction_completed",
  "summary_tokens": 2048,
  "messages_before": 48,
  "messages_after": 12
}

// CompactionFailed -- session is not mutated on failure
{
  "type": "compaction_failed",
  "reason": { "kind": "empty_summary" }
}

If the built-in summarization request fails with a recognized over-context or request-too-large error, Meerkat can use a deterministic mechanical handoff summary to shed old rows. This is allowed only when current history has no protected, unsummarized assistant transcript observations with source SpokenUnmeasured. With those observations, the compaction attempt emits compaction_failed and preserves the original transcript, as it does for ordinary LLM errors. A configured curator never falls back to the mechanical or LLM path. Custom thresholds via the Rust SDK:

Rust

let compactor = DefaultCompactor::new(CompactionConfig {
    auto_compact_threshold: 50_000,
    max_request_bytes: Some(9_000_000),
    recent_turn_budget: 6,
    max_summary_tokens: 8192,
    min_turns_between_compactions: 5,
});

max_request_bytes is a host cap, not an estimate of the current request. Compaction triggers at four-fifths of the effective cap. When the provider also supplies an exact request-body cap, the stricter cap wins.

Supply a compaction curator

Use a curator when a host application has its own deterministic or supervised handoff-summary path. The runtime still decides when compaction is required and validates the resulting rewrite.

use async_trait::async_trait;
use meerkat_core::{
    CompactionCurator, CompactionCuratorError, CompactionWindow,
    CuratedCompactionSummary,
};
use std::sync::Arc;

struct HostCurator;

#[async_trait]
impl CompactionCurator for HostCurator {
    async fn curate_summary(
        &self,
        window: CompactionWindow<'_>,
    ) -> Result<CuratedCompactionSummary, CompactionCuratorError> {
        let summary = host_summary_service(window.messages).await
            .map_err(|error| CompactionCuratorError::Failed(error.to_string()))?;
        CuratedCompactionSummary::new(summary)
    }
}

let mut build = AgentBuildConfig::new("gpt-5.6-sol");
build.compaction_curator_override = Some(Arc::new(HostCurator));

There is no automatic LLM fallback. A curator error emits compaction_failed, records no transcript rewrite, and leaves the original history authoritative.

Budget limits

Cap resource usage per session with token, time, and tool-call limits. When a budget is exhausted, the agent loop terminates gracefully.

Configure the cumulative token budget in the active realm config:

[limits]
budget = 10000

rkat run "Research this topic" \
  --max-tokens 10000 \
  --max-duration 30s \
  --max-tool-calls 5

--max-tokens limits each model call’s output, not cumulative consumption. The limits.budget setting above supplies the aggregate token limit corresponding to budget_limits.max_tokens in the API examples.

{
  "jsonrpc": "2.0", "id": 1,
  "method": "session/create",
  "params": {
    "prompt": "Research this topic",
    "budget_limits": {
      "max_tokens": 10000,
      "max_tool_calls": 5
    }
  }
}

curl -X POST http://127.0.0.1:8080/sessions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Research this topic",
    "budget_limits": {
      "max_tokens": 10000,
      "max_tool_calls": 5
    }
  }'

{
  "name": "meerkat_run",
  "arguments": {
    "prompt": "Research this topic",
    "enable_builtins": true,
    "budget_limits": {
      "max_tokens": 10000,
      "max_tool_calls": 5
    }
  }
}

result = await client.create_session(
    "Research this topic",
    budget_limits={
        "max_tokens": 10000,
        "max_tool_calls": 5,
    },
)

const result = await client.createSession("Research this topic", {
  budgetLimits: {
    max_tokens: 10000,
    max_tool_calls: 5,
  },
});

let build_opts = SessionBuildOptions {
    budget_limits: Some(BudgetLimits {
        max_tokens: Some(10_000),
        max_duration: Some(Duration::from_secs(30)),
        max_turn_duration: None,
        max_tool_calls: Some(5),
    }),
    ..Default::default()
};

Budget events

When consumption nears a limit, a warning event is emitted before the budget is fully exhausted:

{
  "type": "budget_warning",
  "budget_type": "tokens",
  "used": 8500,
  "limit": 10000,
  "percent": 85.0
}

Retry policy

Transient LLM errors (rate limits, network timeouts) trigger automatic retries with exponential backoff. Each retry attempt emits an event carrying the typed failure and the scheduled retry plan:

{
  "type": "retrying",
  "retry": {
    "failure": {
      "provider": "anthropic",
      "kind": "rate_limited",
      "retry_after_ms": 4000,
      "message": "rate limit exceeded"
    },
    "plan": {
      "attempt": 2,
      "max_retries": 5,
      "computed_delay_ms": 2000,
      "selected_delay_ms": 4000,
      "rate_limit_floor_applied": true,
      "budget_capped": false
    }
  }
}

Retries consume time budget but not token budget. If the time budget expires during a backoff wait, the agent terminates with a budget-exhausted error.

Getting started

Concepts

Guides: Models And Hosting

Guides: Runtime Behavior

Guides: Orchestration

Examples: Foundations

Examples: Capabilities

Examples: Orchestration And Deployment

Memory & compaction

Enable semantic memory

Memory search

Compaction

Supply a compaction curator

Budget limits

Budget events

Retry policy

Next step

​Enable semantic memory

​Memory search

​Compaction

​Supply a compaction curator

​Budget limits

​Budget events

​Retry policy

​Next step

Enable semantic memory

Memory search

Compaction

Supply a compaction curator

Budget limits

Budget events

Retry policy

Next step