MeerkatMachine host composition is intentionally not enough.
This page covers expert-level direct agent construction, provider
configuration, budgets, hook helpers, and lower-level delivery internals.
Runtime-backed vs standalone builds
The Rust contract is explicit:- runtime-backed surfaces should call
MeerkatMachine::prepare_bindings(session_id) - those bindings should be threaded through
SessionBuildOptions.runtime_build_mode = RuntimeBuildMode::SessionOwned(bindings) - standalone/testing/embedded paths should use
RuntimeBuildMode::StandaloneEphemeral
AgentBuilder vs AgentFactory
meerkat::AgentBuilder is the public facade builder. It accepts explicit client/tool/store overrides for embedded use, then still routes through AgentFactory::build_agent() so provider defaults, session metadata, hook wiring, and runtime build-mode defaults stay aligned with the factory path.
AgentFactory (in the meerkat facade) is the opinionated composition layer.
It knows about tool categories, runtime resources, comms, memory, mobs, and
skills, and wires them into the dispatcher before passing them to
AgentBuilder. Production surfaces pair its persistent session service with
MeerkatMachine so the machine remains the transition authority.
The lower-level meerkat_core::AgentBuilder is not re-exported by the facade.
Its standalone build path is reserved for core tests; public embedding code
should use the facade builder or AgentFactory.
Use AgentFactory via build_persistent_service_with_runtime_adapter() for
a native persistent host, build_ephemeral_service() for a direct
embedded/testing path, or the public meerkat::AgentBuilder facade when you
intentionally own the lifecycle. Direct core builder usage means you own
composition, persistence, event handling, and any drift from the standard
runtime-backed path.
A non-runtime service, such as build_ephemeral_service() or an
AgentFactory given a store through session_store(), must not share a
realm’s SQLite session store: it saves sessions whole, without the canonical
head a runtime-backed host commits through, so a runtime-backed host that
opens the realm later cannot adopt them. Their first runtime commit is refused
and their data stays intact.
If you are building a runtime-backed surface around SessionService, avoid the
old hand-rolled register_session() + registry extraction pattern. Prefer the
binding seam above so the runtime/session owner stays canonical.
AgentBuilder
For most production hosts, prefer the persistent service plus machine path
(see overview). The public
AgentBuilder delegates to the
factory pipeline. Use the core builder only inside core-owned tests that
bypass the session service and facade composition entirely.All builder methods
All builder methods
provider_tool_defaults, compactor, memory_store, and
with_turn_state_handle remain source-compatible rejection points. They record
an unsupported override and make try_build() fail instead of silently
discarding the requested behavior.
Providers
Built-in clients for major LLM providers:- Anthropic
- OpenAI
- Gemini
Provider parameters
Pass provider parameter overrides viaAgentBuildConfig. The JSON bag is retired:
provider_params is the typed ProviderParamsOverride (temperature, top_p,
max_output_tokens, reasoning mode, thinking_budget_tokens, and an optional
provider-specific provider_tag), parsed fail-closed at each surface ingress.
Provider tags also carry provider cache policy: OpenAI can override Responses
store (default remains false) and set prompt_cache_key /
prompt_cache_retention. Factory-built public OpenAI API GPT-5.6 sessions default to
prompt_cache_options: {"mode":"explicit","ttl":"30m"} plus
prompt_cache_key: "meerkat:profile:openai:<model>"; a caller-supplied key
wins. GPT-5.6 and later also accept "ttl":"12h", a 12-hour minimum lifetime
for a stable prefix reused across longer gaps, such as a long-lived agent that
responds every few hours. It applies to every breakpoint of the request, so
every cache write of a 12h request costs 2.5x the uncached input rate instead
of 1.25x, the conversation tail included; it is never a default. These cache
fields, defaults or explicit, reach only the public OpenAI
API: requests through the private ChatGPT backend, Azure OpenAI, GitHub
Copilot or a self-hosted OpenAI-compatible server carry none of them. In
explicit mode Meerkat authors append-monotone breakpoints at deterministic
message boundaries. Set prompt_cache_enabled: false for a durable opt-out; it emits
explicit-only mode without a breakpoint and incurs no cache-write charges.
The default key is per model, not per session: every session on the same
OpenAI model sends the same key, so a system prompt and tool set that are
identical across sessions can be read from OpenAI’s prefix cache instead of
written again for each new session. The key only routes requests toward a
cache; it never proves a cache hit. OpenAI still requires a byte-identical
prefix, so a timestamp, a changed tool list or any other difference before the
breakpoint misses regardless of the key. Usage.cache_read_tokens and
cache_creation_tokens report what was actually read and written.
Sharing one key per model maximises cross-session reuse of the static prefix.
The trade-off is that all traffic on that model shares the key: OpenAI
recommends keeping traffic near 15 requests per minute per cache key, and
beyond that requests can overflow OpenAI’s routing for the key and land where
the prefix is not cached. Tool-calling agents make several model requests
during one user turn, so the rate that counts is the aggregate model-request
rate across all sessions on the model. A host that needs to spread higher
volume can set prompt_cache_key itself with a stable partition (per tenant or
per identity, for example); each narrower key shares the static prefix with
fewer sessions. Choose and monitor the partition using the observed
model-request rate. Cache reads consider only the latest 50 breakpoints.
Anthropic API, Vertex, and Foundry clients default to
cache_control: automatic, which keeps a moving five-minute breakpoint on the
growing conversation. disabled opts out, system_prefix narrows the
breakpoint to the system prompt, and system_and_conversation marks the
system prefix plus the three most recent conversation boundaries; cache_ttl
(5m, the default, or 1h) selects the breakpoint lifetime and rides the
automatic default when set alone. Amazon Bedrock and the GitHub Copilot
backend default to disabled because they do not support Anthropic automatic
caching and reject an explicit automatic override locally; manual
system_prefix caching remains available. Automatic lookup scans backward at
most 20 cacheable blocks. AnthropicClientBuilder starts from automatic
with automatic support assumed. A direct builder user targeting a backend
without automatic caching must call automatic_cache_control_supported(false);
with that set and default_cache_control left at automatic, the first
request fails locally with an invalid_request naming the fix, so also set a
non-automatic default_cache_control (disabled or system_prefix). A
builder user who sets neither sends the request-wide breakpoint to the
backend, which rejects or ignores it remotely. The provider runtime does both
for every configured backend, the plain API-key path included. Gemini can
pass an explicit cached_content_name. When OpenAI store is explicitly
enabled, stored response IDs may be reused as previous_response_id hints;
Meerkat still keeps the local transcript as the canonical replay source and
falls back to full replay if the provider rejects the hint.
Implementing a custom LLM client
Implementing a custom LLM client
This toy client declares synthetic inclusive token counts through
TurnUsage::host_declared. Real provider adapters should attach their actual
normalized provider accounting and convention instead; do not relabel
provider counters as host evidence just to suppress an accounting warning.
Leaving Usage.provider_accounting absent is legal, but emits
TurnUsageAccountingUnmeasured, leaves TurnCompleted.usage absent, and
advances neither session token totals nor token-budget accounting. The answer
does not fail merely because accounting is unmeasured.Budget configuration
Retry configuration
When to bypass SessionService
Direct agent construction still makes sense when you need one of these expert-only scenarios:- You are embedding a single transient agent loop with fully custom persistence and tool composition.
- You are testing a lower-level trait contract in isolation from session orchestration.
- You are implementing infrastructure that itself sits underneath a
SessionService.
SessionService so runtime admission, history, interruption, and external-event handling stay canonical.
Hook helpers
AgentBuildConfig::hook_engine_override) reads no convention file and asks no HookSourceTrust: the engine runs the hooks it was made with. A host that gates its hook files by their content leaves their resolution to the build.
Comms delivery and observation
Public comms injection is a delivery API.event_injector() gives you an Arc<dyn EventInjector>, and inject() queues a runtime-backed external event for later admission.
How it works
How it works
inject()queues a plain event into the session inbox.- The runtime-backed surface admits that queued event as future turn work.
- Observation is selected independently:
- use the session’s primary
event_txfor full session activity, - use mob member subscriptions for agent-scoped activity,
- use mob event subscriptions for attributed mob-wide activity.
- use the session’s primary
- Public callers no longer receive an interaction-scoped stream from injection.
- Agent must be in keep-alive mode if you want the keep-alive loop to drain inbox events.
- Comms must be enabled.
- If you need streaming output, configure an observation surface separately.
SSE streaming example
SSE streaming example
This application helper requires
axum, async-stream, and tokio, plus
meerkat and a matching exact-pinned meerkat-core dependency. Your HTTP
handler supplies the service and a per-session broadcast feed of
AgentEvent payloads from the configured observation surface. The local
sse_event formatter below is application code, not an SDK export.The host supplies a service implementing SessionServiceCommsExt, whose
event_injector extension can expose a session’s optional injector.Injection only queues input; observation is session-wide, not correlated to
that input. Closing on the next run terminal, as below, is appropriate only
when the host ensures no other in-flight or queued work can produce it.
RunCompleted ends the main run; structured-output extraction can follow.Durable event projection and live observation
Persistent disk-backed runtime hosts can configure a dedicated lossless projector queue for session events. UI and other live subscribers use best-effort broadcast observation and can receive typed lag markers when they fall behind. Do not use a live broadcast receiver as a durability boundary. When projection is enabled, reconcile from the persisted event cursor or snapshot after lag, restart, or reconnect. Those replay APIs are unavailable when the host disables event projection, including explicitly ephemeral memory realms.Trusted one-shot member role migration
MemberLaunchMode::Resume can carry resume_from_role: Option<ProfileName>
when a trusted Rust host spawns or materializes a remote member. This declares
one exact durable role migration for the resumed session. It is intentionally
absent from agent-callable spawn wires and standing profiles, and reuse or a
mismatched source role fails closed. Ordinary resume should leave it as
None.
Source owners and retained continuations
A host that owns governed originals implementsAuthenticatedOriginalSource.
Since 0.9.4 it answers retained_custody, which replaces retained_original
and returns RetainedCustody (non-exhaustive):
Work that names a pending original is unavailable, not refused, and nothing
waits for it. Native admission of input that names such a contributor,
require_source_original_live and a retained resume report
ControllerReadinessUnavailable with
ControllerReadinessFailure::OriginalAdmissionPending (wire kind
original_admission_pending, INPUT_NOT_READY on JSON-RPC). Operation
authorization of work that names it, the further originals of a helper
admission or child issuance, and the controller-grant mutation check report it
unavailable as well. A retained resume therefore stays retryable until the
owner settles the original. A cancelled original is still refused, and the
controller-grant mutation check no longer counts a pending contributor as
unreferenced. 0.9.3 and earlier refused such work as malformed or denied.
Native admission, require_source_original_live and a retained resume also
report an owner change that one of their grant-lineage reads meets (the
original’s own mandate, or a named contributor’s liveness) as
ControllerReadinessFailure::PolicyChanged (wire kind policy_changed), and a
later attempt may pass. require_source_original_live meets one only outside
an exclusion of owner changes. Each of these reads, and controller admission’s
own, is first made once more in place when the change it met has completed
(LocalGrantAuthority::owner_change_in_progress names none). Admission never
waits, because it runs inside the session’s ingress transaction, so a change
still in progress, or back-to-back changes, still answer policy_changed. A
host that submits again waits first, outside its own locks, with a bounded
meerkat_core::authorization::OwnerChangeChurn::wait_after_stale_read, then
resubmits once.
The job delivery drain treats a continuation the runtime is not ready to admit
the same way. retained_admission_outcome maps it to
ContinuationAdmitError::NotReady(cause) (the enum is now
#[non_exhaustive], so a match needs a wildcard arm),
JobOutboxProjectionError::NotReady carries the cause, and the drain blocks
the row with BlockedDeliveryReason::NotReady(cause). That is the enum’s first
variant with data, so its variants no longer have defined discriminants. The
continuation stays pending and is retried, as for Failed; nothing is stored
for the cause, which is read again on every retry.
MeerkatMachine::durable_admitted_original (new in 0.9.4) reads the historical
admission of a keyed governed original from the session’s durable runtime
store alone, so a source owner can resolve it after the session retired. It
needs no registration, attachment or actor, consults no live state, and
revives, submits, replays or releases nothing. It returns
DurableAdmittedOriginal (the actual input id, the retained association, the
durable phase, terminal outcome, last run and attempt count) only for the exact
admission: one own contributor with that association and the exact replay of
the expected original. Otherwise it returns DurableAdmittedOriginalRefusal
(non-exhaustive): Conflict for a mismatch, NotAdmitted for nothing under
the key, UnknownRuntime for an unknown session, and Unavailable for
unreadable evidence. SessionServiceRuntimeExt::durable_admitted_original
forwards it; its default answers Unavailable.
TurnStateHandle gains three provided methods in 0.9.4: turn_phase reads the
turn phase alone, apply_turn_input_observing_phase applies a typed turn input
and returns its effects with the phase the authority holds after it, and
scalars returns TurnStateScalars, the scalar fields of a
TurnStateSnapshot without its collections or terminality classification.
The defaults derive the answers from snapshot and apply_turn_input, so
existing implementors keep working unchanged.
Retiring a runtime delivery owner
Since 0.9.4RuntimeDeliveryOwnerRetirement::join returns
RuntimeDeliveryOwnerRetired instead of (). Its
notification_registration_retained names the directory
(meerkat_runtime::RetainedNotificationRegistration) whose file notification
registration the operating system still held when the owner stopped its store
watch. Nothing can interrupt that platform call (macOS FSEvents can hold it for
tens of seconds), so the retirement cancels the registration without waiting;
its thread stays with the process until the call returns and never touches the
store. The join logs the retained directory once, as an info event. The report
is #[must_use]: code that used ? or matched Ok(()) binds it, matches
Ok(_) or reads it. DeliveryStoreWatch::stop returns the same registration,
and SqliteChangeWatch::pending_registration describes the one a watch would
leave.
Complete example
Python and TypeScript SDKs
Both communicate with a localrkat-rpc subprocess over JSON-RPC 2.0 (with optional explicit realm/instance scoping) — no native bindings required.
- Python: Python SDK overview
- TypeScript: TypeScript SDK overview
See also
- Rust SDK overview - getting started, sessions, events
- Tools and stores - tool system, stores, MCP
- API reference - type index
- Architecture - system design and internals
