meerkat-mob remain authoritative for agent execution, identity, membership,
lifecycle, and wiring; Meta packages gateway startup, provider adapters,
operational modules, persistence projections, and operator surfaces.
Installation
The wheel is client-only and does not contain
rpc_gateway. Download
meta-rpc-gateway-<version>-<target>.tar.gz for Unix or macOS, or
meta-rpc-gateway-<version>-<target>.zip for Windows, and extract its
rpc_gateway binary (rpc_gateway.exe on Windows). Alternatively, follow the
source-build instructions. Pass that binary’s path to
.gateway(...). Linux x86_64 and aarch64 and macOS arm64 archives are
published with the release and listed in index.json and checksums.sha256;
macOS x86_64 and Windows archives may arrive later, listed in
index-trailing.json and checksums-trailing.sha256.Quick start
In a fresh project directory, createconfig/mob.toml:
OPENAI_API_KEY in your environment for the model above, or choose a
supported model and its matching provider credentials. Run the following
script from that project directory, replacing /path/to/rpc_gateway with
the binary you installed. Profiles are templates, not pre-created members,
so the example explicitly creates an ephemeral worker before sending.
Durable detached tools
Declare long-running host work explicitly instead of holdingcallback/call_tool open:
Identity-First API
For apps with durable agent identities, the identity-first API provides durable lifecycle management. Agents are addressed by stable identities (identity:alice, triage:main), not member IDs.
send_and_wait, dispatch_and_wait and dispatch_text_and_wait are
per-admission: the runtime mints a turn ticket for the delivery and keeps the
admitted turn’s own completion handle under it, and the wait reads
meta/turn_result for that ticket, which reports that turn’s own output
from its committed run result. A failed turn raises TurnFailedError, and a
ticket the gateway does not know (for example after a gateway restart) raises
TurnUnknownError. Text cut at the 256 KiB bound emits
TurnOutputTruncatedWarning; a turn that completed without output of its own
(the runtime folded it into a run in progress, or deduplicated it) returns
None with TurnOutputUnavailableWarning. wait_for_turn returns the typed
TurnResult (output_status, output, output_truncated) instead.
send_and_wait_outcome, dispatch_and_wait_outcome and
dispatch_text_and_wait_outcome take the same arguments and return the typed
AwaitedTurn: text, attributed (whether the text provably belongs to this
delivery’s turn), admission, ticket, output_status and untracked_code.
The plain methods return its text.
The delivery happens exactly once and the call never resends. Every failure
after admission carries the send/dispatch result as admission, so the
caller keeps the receipt instead of redispatching the work:
- No turn ticket (any delivered
turn_unavailable.code, below, or a gateway that predates turn tickets): the call waits on the identity-wide completion cursor and emitsTurnTrackingUnavailableWarningnaming the code. That result is NOT attributed: another delivery’s completion can satisfy the wait, and the text is the session’s latest output. The outcome’sattributedisFalseanduntracked_codecarries the code.require_attribution=TrueraisesTurnTrackingUnavailableError(withcodeandreason) in every untracked case instead. - The exact-ticket wait fails for a transport or RPC reason (for example
observation_lane_saturated): only that ticket’s observation is retried, with bounded backoff, untiltimeout; thenPostAdmissionObservationErrorcarriesadmission,ticketand the last failure as__cause__. Read the turn later withwait_for_turn(ticket). - Every other error after admission keeps its type and carries
admission(andticket,Nonewhen untracked):TurnFailedError,TurnUnknownError,WaitEndedError, the identity-wide wait’s missing-baseline, superseded-incarnation and live-aliasRuntimeErrors, the persistent transport’s plainRuntimeErroron a response timeout or closed stdout, a caller’sasyncio.CancelledError(the same instance, re-raised), and an SDK warning a caller escalated withfilterwarnings("error"). A wait still pending attimeoutraisesTurnWaitTimeoutError, aTimeoutErrorthat also carriesadmissionandticket. The one exception is a custom exception type that rejects attribute assignment, which cannot carry the fields and still propagates unchanged.
turn_unavailable.code says why a turn could not be tracked. Every
code but not_delivered means the delivery still happened, exactly once:
autonomous_host: the member runs inautonomous_hostmode (the default), read from the runtime’s live roster entry, so a profile or override changed without a respawn does not count until the member respawns. The runtime reports no per-turn completion for autonomous inbox delivery, so on these members a per-admission wait is unavailable.runtime_refused: the runtime refused per-turn completion for the member’s live mode for another reason.externally_bound,host_human_input,bridge_cannot_report_output.- A gateway that predates turn tickets returns no
turn_unavailableat all.
not_delivered means nothing was delivered (no session bridge or no bound
runtime), so the call raises TurnNotDeliveredError instead of waiting
(nothing ran, so a retry is safe). session_rotated means a re-dispatch
landed on a different session after a repair or respawn and ran as its own
turn. Run a member with
runtime_mode = "turn_driven" for per-admission results.
Do not mix ticket waits and cursor waits on one identity: a ticket can settle
before the identity health monitor counts the same turn on the cursor, so a
later wait_for_output(after=baseline) can be satisfied by an earlier
send_and_wait turn.
wait_for_completion(baseline) and wait_for_output(after=cursor) remain
identity-wide primitives: any completion on the identity after the baseline
satisfies them, and the returned output_preview is the session’s latest
output, not necessarily a specific send’s answer. Use wait_for_turn(ticket)
or wait_for_output(turn=ticket) to wait for one specific turn.
The recommended completion barrier is track_turn=True plus
wait_for_turn(ticket) (or send_and_wait): it reads only the ticket ledger.
Every wait is server-side and event-driven: wait_for_turn is one
meta/wait_for_turn request, and the identity-wide waits
(wait_for_completion, wait_for_output(after=...), rt.wait_until_ready)
are one meta/wait_for_completion request each. The gateway answers when
the ticket settles or a completion is recorded, so nothing polls, and the
member’s output is read once, at completion. timeout is only the deadline;
poll_interval applies only against a gateway that predates the server-side
waits. completion_cursor() is a one-shot cursor read that never touches the
member session. inspect() does read the member session, so do not poll it.
A wait that ends before a completion (a failed or cancelled run, a broken,
retiring or deleted identity, or gateway shutdown) raises the typed
WaitEndedError, whose outcome names why; only the deadline raises
TimeoutError.
DurableAgentSpec fields
The roster provider returns oneDurableAgentSpec per identity. The SDK
serializes the shared wire shape that Rust deserializes: identity, profile,
addressability, labels, and additional_instructions are always included,
even at their defaults. Optional fields whose value is None are omitted,
including unset runtime_mode_override and initial_message.
Roster provider context
The gateway callsroster(context) with a typed RosterContext
(meerkat_meta.identity_first_models.RosterContext), a one-to-one mirror of
the Rust struct:
DispatchInput convenience constructors
Scope-bound dispatch
A host that must never admit a delivery to a session other than the one it recorded captures the identity’s scope fromstatus(), persists it, and
dispatches against it. A moved scope raises StaleScopeError with nothing
submitted; a lost reply is recovered from the scope’s original session.
expected_scope needs both idempotency_key and correlation_id, does not
combine with track_turn, and applies no delivery preparation. A scope with
an unknown version raises ContractMismatchError from
DeliveryScope.from_dict; in status() it leaves delivery_scope None and
the rest of the status usable. recover_delivery returns ScopedRecovery
with state ABSENT, IN_FLIGHT (durable_witness), COMPLETED
(output_status, output), FAILED, TERMINAL_WITHOUT_RUN, BROKEN or
UNRESOLVED (cause); a state this SDK does not know reads as UNRESOLVED,
never ABSENT. See Scope-bound dispatch.
Mob-Level API
For direct mob-member operations (discovery-based), the mob handle API is still available. The app-facing surface has three categories. Each is a distinct concern — no method crosses categories.Comms — deliver work to agents
SendMessageResult with accepted, member_id, and session_id fields.
The message enters the agent’s inbox and is processed by the host loop.
Observation — watch what’s happening
Structural mob events — durable observability
MobHandle.query_mob_events() and subscribe_mob_events() project
every current Meerkat MobEventKind into a typed envelope
preserving mob_id, run_id, step_id, agent_identity, and the
full payload.
MobStructuralEvent.cursor is the meerkat ledger cursor —
durable across meta gateway restarts when the runtime is built
with .persistent_state(path). Checkpoint a cursor and resume by
passing it as after_seq on the next call.
Flow runs
MobRun carrying the full meerkat ledger projection
(step_ledger, failure_ledger, frames keyed by frame id, loops
keyed by loop id, loop_iteration_ledger, flow_state,
activation_params, schema_version, etc.).
Control plane — manage the runtime
ensure_member() and find_members() both return typed MemberSnapshot objects
(single and list[MemberSnapshot] respectively).
Roster — inspect and manage members
MemberSnapshot exposes agent_identity, role, state, wired_to, and
labels. Known state strings are exported from the top-level package as
MEMBER_STATE_ACTIVE, MEMBER_STATE_RETIRING, MEMBER_STATE_BROKEN,
MEMBER_STATE_COMPLETED, and MEMBER_STATE_UNKNOWN. The underlying Rust
status enum is non-exhaustive, so consumers should tolerate future state
strings.
Common pattern: new user first contact
Per-user agents are durable members, so stand them up on the identity plane — declare them through your roster provider and reconcile — rather thanensure_member-ing them into the mob roster. Identity-plane members
resume their session across restarts (Core keeps it); mob-plane members do
not.
ensure_member for ephemeral workers (helpers a parent spawns
and an idle-retire label reaps) — the worker plane. See the
identity-first doctrine.
Builder pattern
Builder methods
A member forked with
fork_off, a delegation fork or a council does not run
the instructions its source activated (meta/member_activate_instruction,
skill bodies activated that way included) unless the gateway’s fork instruction
policy says so. Under the default, such a fork is refused with
fork_instruction_refused before anything is committed. To let forks carry
their source’s activations, opt in at boot:
reactivate_inherited trusts
before you enable it.
Boot-time membership is declared, never derived: a member exists only once a
.roster(provider) names it (identity plane) or ensure_member creates it
(worker plane); [profiles.*] in mob.toml are role templates and spawn
nothing on their own. The former .discovery(callback) and
.pre_spawn(callback) builder methods were removed: they stored the callback
and never transmitted it, so no boot spawn ever came from them.
Identities and their sessions live in Core’s stores under
.persistent_state(path): Core’s mob machine holds each identity’s session
and generation, and a host supplies no continuity storage and holds no
leases. The external identity providers of Meta 0.9.3 are removed; see
Upgrading to Meta 0.9.4 to move a deployment
that used them.
A durable member whose role changed refuses to resume with
MobError::MemberRoleMigrationRequired until the host declares the migration.
.role_migrations([...]) is that declaration: it emits a top-level
role_migrations init param, not a runtime_options field, and the authority
lasts exactly one boot because nothing persists it. Meerkat re-verifies the
declared from_role against durable state and refuses with
MobError::MemberRoleMigrationRejected on mismatch, and ignores the declaration
entirely once the roles already agree, so a declaration left in place after the
migration landed is inert. It arms only where an identity plane exists; on
rpc_gateway that means alongside .roster(...). See
roster and member lifecycle.
Every explicit .identity_bootstrap_mode(...) call declares identity-first
intent and requires .roster(...), including explicit eager mode. Omitting the
setter keeps a gateway without a roster on the classic path; a configured
roster with no explicit mode reports the eager_materialize mode. Under every
mode each roster identity is a member of the mob from ready, and is
materialized when work reaches it; .member_restore(...) decides what is
materialized at start.
Gateway memory config accepts {backend: "local_json"} with an optional
health_check_endpoint (memory.local_json(...)), the only backend. The
predecessor’s backend helper was removed in 0.9 with no alias, and the gateway
refuses its backend value at init.
memory(stores=...) is intentionally rejected because the Rust gateway does not
accept store lists in meta/init. JWT auth is the gateway-supported auth mode;
Google/OIDC helpers are not accepted by gateway init today.
Agent memory is configured with .agent_memory() and requires
.persistent_state(path) plus an identity roster when using the bundled gateway
store. Options include realm, selection, max_entries,
recall_timeout_ms, recall_failure_policy, and instruction_header; the SDK
serializes them to the gateway wire contract. Automatic injection defaults to a
500 ms timeout and skips the memory block if recall fails.
Storage durability declarations. The gateway is fail-closed about
durable storage: a store that cannot open is a startup error, never a
silent in-memory fallback. The config.runtime_store and
config.event_log modules produce the explicit declarations:
StatusResult.storage /
CapabilitiesResult.storage parse the gateway’s storage object into
StorageSummary (blob durability, the H2 incremental probe, and the
per-slot StorageSlotSummary census with domain, durability_class,
resolution, backend, degraded, detail). A fail-closed startup
refusal — file-name twins, a store that failed to open — surfaces as the
typed StorageResolutionError (code -32014,
STORAGE_RESOLUTION_CODE), not a TransportError; its message names the
remediation (meta/storage/doctor, or the explicit ephemeral
declaration). Blob-store durability has no wire declaration today — it is
declared by Rust embedders via ephemeral_blobs(true) /
binary_blob_store(...) and reported read-only in the census.
A composition provenance refusal surfaces as the typed
CompositionProvenanceError (code -32019, COMPOSITION_PROVENANCE_CODE).
The gateway raises it when the persistent mob storage’s recorded composition
cannot be proven to match this launch, for example a mob.toml edited since
the persistent mob storage was created. Its kind names the refusal
("divergent", "created_by_rehearsal", the manifest kinds, and
"candidate_divergent" from gateways launched as a candidate) and fields
lists the diverged definition fields as dotted paths, so a host can show what
changed without parsing the message. Revert the edit, or move the stored
definition with declare_spec_update(expected_revision=...):
tools.mob_tasks in a config written for
an earlier release) is never one of the fields: removing it to act on the
warning, or adding it, does not cause this refusal.
Console access
The bundled console is fail-closed. Without.auth(...) the gateway’s
decision state trusts no signing key, and because require_app_auth defaults
to true every console request is refused with 401; the gateway logs a
startup warning naming this. Opening the console is an explicit builder call:
.console_auth_required(False) only behind a loopback bind or a reverse
proxy that authenticates; for anything shared, configure .auth(...) instead.
Earlier SDKs had no setter and hosts overrode
MetaRuntime._build_init_params to inject console_require_app_auth or
console_config_path; both now have builder methods and the override is no
longer needed.
The gateway binds 127.0.0.1 on an ephemeral port by default. To publish it
from another container, .http_listen("0.0.0.0:8080") together with either
.auth(...) or the explicit .allow_remote() acknowledgement, and
.http_public_base_url(...) to advertise the proxy’s address (read back as
runtime.rust_http_public_base_url; runtime.rust_http_base_url stays the
same-host form the SDK itself dials). A non-loopback bind without either is
refused at init. See Deployment.
Runtime lifecycle
Startup: no built-in total deadline
connect() (and build()) runs meta/init as accepted-then-settled. The
gateway accepts the init at once, reports each startup phase, and then
settles it ready or failed. The SDK has no built-in total deadline for
that wait. Startup legitimately includes provider callbacks (each bounded at
130 s by the gateway) and the native identity restore, which is
deliberately unbounded and can run for minutes on a large roster. Any fixed
SDK timer would sooner or later abort a healthy restore. A 60 s timeout still
bounds the wait for the accepted answer, and for an older gateway’s single
init response. A host whose gateway answers init late can raise it with
Meta.builder().gateway_init_timeout(seconds) (a positive finite number of
seconds). That bounds a slow host startup; it does not fix one. It applies only
to the meta/init request: ordinary RPCs and provider callback deadlines
keep their own timeouts.
The wait ends only when one of these happens:
- the init settles
ready(connected); - the init settles
failed. You get the typed error, for exampleStorageResolutionError, witherr.data["durable_effects"]:"none"when nothing durable was touched, otherwise"possible"; - the gateway exits, or the transport reader fails. You get
InitOutcomeUnknownError; - your own deadline runs out. Set it with
Meta.builder().init_deadline(seconds), and running out also raisesInitOutcomeUnknownError; - your task is cancelled.
InitOutcomeUnknownError carries init_id, last_phase (the last
meta/init_progress phase seen, such as "roster" or "restore") and
reason. That makes a stuck or interrupted init diagnosable. The gateway may
already have changed native state, so the SDK never retries it automatically
and asks the gateway to shut down.
Provider callbacks that run during startup (roster, topology, session
builder, customizer) must not call ordinary runtime RPCs on the same
runtime. Ordinary dispatch starts only after init settles, and the
gateway refuses an earlier request at once with InitInProgressError
(-32018, exported as INIT_IN_PROGRESS_CODE).
Adopting a gateway child the host started
By defaultconnect() spawns rpc_gateway --persistent and owns that
process. A host that supervises the gateway itself (it spawns the process and
owns its lifetime, credentials and cleanup) can hand the SDK that exact child
instead. The SDK then speaks the same stdio protocol over the child’s pipes:
init with its accepted-then-settled readiness, host callbacks, RPC with its
timeouts and the shutdown handshake. It never spawns, signals, reconnects to
or replays a gateway.
Ownership and lifecycle:
AdoptedGatewayChild(...)andAdoptedGatewayTransport(child)only validate: they read, write and close nothing.connect()(or the transport’sstart()) hands the child over, and from then on the transport writesinputand readsoutput. Onlystart()hands it over: a request before it is refused, so the first line the child reads ismeta/init. A refusal before the hand-off leaves both pipes with the host, untouched.- A raw stream is buffered by the transport itself, at the hand-off: a raw
write can take fewer bytes than it was offered and a raw
readline()reads one byte per call, so the transport writes every byte of each line before it flushes, reads lines in blocks, and retries interrupted and partial reads and writes. The wrapper sits over the host’s own stream object, and closing it closes that stream once; the transport opens no second file object over the same descriptor and closes nothing else of the host. Do notos.close()a descriptor you handed over inside such an object: the transport closes it through that object, and by then its number could belong to another file. - From the hand-off, one exit watcher thread is the only caller of
wait_process, in waits of at most 60 seconds, until the child exits or a stop returns; it never calls it from two threads at once. Once the child exited, a request still waiting on a stdout that a process the child started holds open fails about a second later withGatewayChildEndedError, as with a spawned gateway, and so does an init still settling. Ifwait_processraises something other than a timeout, the watcher logs it and ends: the next stop observes the child itself, and until then an exit fails no waiter early. shutdown()stops the transport on a runtime-owned worker thread. The stop makes the stock protocol shutdown (the onemeta/shutdownrequest when the gateway negotiated it, then the end ofinput) on a helper thread that it waits for only until the gateway’s shutdown horizon ends, so a writer stuck on a full pipe cannot hold it. If that phase times out, the helper and the stuck writer stay blocked until the child’s stdin closes, normally through the retirement. Then, whatever that phase did, it callsrequest_retirement(), once per child however often it runs, and waits for the direct child’s exit as the watcher observes it, for the shutdown horizon plus 10 seconds from when the first stop began (10 seconds more on each later stop, which callswait_processitself). It never runs or awaits the host’s own cleanup, and the SDK’s reader never calls the host. The reader closesoutputwhen it ends.- A child whose exit is still pending at the end of that bound raises
GatewayChildPendingError. The runtime keeps the transport, and the nextshutdown()observes the same child again without requesting its retirement twice. - A start that fails after the hand-off (init refused or failed, an init
outcome that is unknown, a cancelled caller) is stopped the same way. Its
failure is recorded before that stop is scheduled, so cancelling the caller
drops neither the retirement request nor the child’s observation; as with
a spawned gateway, the exception of a cancelled caller carries
gateway_cleanupwhile the stop has not succeeded. - The transport’s own
stop()is synchronous: run it withawait asyncio.to_thread(transport.stop). On an event-loop thread it raisesTransportErrorat once and does nothing. Always stop it (throughshutdown()): a transport dropped without a stop ends the child’s input and callsrequest_retirement()from its finalizer, which may run on any thread, the SDK’s reader included, or in the collection at interpreter exit. - One transport serves one connect. To connect again after
shutdown(), passgateway_transport()a factory taking no arguments that returns a transport over a child the host started for that connect..gateway(...)may be set as well; the SDK still never spawns it. - The SDK’s own SSE, multipart and console calls use the
http_base_urlthe gateway reports. A spawned gateway shares this process’s network namespace; for an adopted child, keeping that address reachable from this process is the host’s job.
GatewayTransportConflictError (a ValueError) refuses, before anything is
handed over, a transport that was already started or stopped, and a
transport given both to MetaRuntime(config, transport) and to
gateway_transport(). The TypeScript SDK’s factory takes precedence over a
constructor transport instead; Python refuses the pair. A transport given to
MetaRuntime(config, transport) alone is used in place of a spawn whenever
the configuration names a gateway, and an AdoptedGatewayTransport given
there is used even when it does not.
Gateway logging (stderr)
The persistent gateway reports tracing lines, panic hooks, and storage migration progress on its stderr. By default the SDK inherits the host process’s stderr, so those lines are visible where your application logs. Two environment variables adjust this:
Discarding is the legacy pre-0.8.9 default and is not recommended: it hides
panic output and makes a slow-but-working storage migration indistinguishable
from a hang.
Session agent builder and per-profile tools
build_agent runs once per member build on the worker plane (fresh spawn or
resume). The gateway pre-fills SessionBuildOptions before it calls you:
meerkat sets the fork lineage on the build that seats a durable fork
(
fork_off children, fork_member children, local council participants) and
on every later rebuild of that member, so a builder can build the child as its
source. The child’s own labels and session_id still name the child; they
never carry the source’s agent_identity. The lineage is receive-only: it is
not sent back, and the gateway ignores it in a build response.
Read profile_name, do not assign it. There is no way to select a profile
from build_agent: the gateway applies only additional_instructions (mint
builds only; on a resume they are ignored with a warning, because a resume
inherits the persisted prompt state), labels, resume_session_id, and the
registered tools from what you return, and ignores a host-set
profile_name. Per-profile tools are therefore
one builder that branches on the pre-filled value:
.session_service(builder) is the public setter; build() connects and
registers the builder on the callback dispatcher for you. MetaBuilder._config
and MetaRuntime._dispatcher are internal and change without notice. The
builder must be set before build(): meta/init tells the gateway whether a
session builder exists, so there is no post-connect registration.
Identity-first equivalent: customize_build
On the identity plane the same hook is AgentCustomizer.customize_build,
registered with the public .agent_customizer(customizer). It runs when an
identity is seated (created), and when its member is built (see below). An
identity the mob already holds is reused without customization until its
next build. Do not use reconciliation to refresh an already-active identity’s
tool handlers. The hook receives the durable spec (so spec.profile is the profile
name), and registers tools on the draft:
draft also carries model, system_prompt, additional_instructions,
labels, app_context, and provider_params. A tool registered here rides
the same callback/call_tool dispatch as build_agent tools.
Each identity member has one stable tool dispatcher for its whole life, and
every customize_build publishes its tools into it. The tool list and the
handler scope that serves it always change together. The tools therefore
follow the member through every rebuild: a gateway restart, a member that is
adopted instead of respawned, a respawn. customize_build runs:
- once when an identity is seated and the mob held no member for it (a new
identity, an identity a first-boot manifest row brings in, a hosted
identity its host creates), when a reset or a delete starts its fresh
life, and when a declared role migration moves it to its new role, with
the runtime’s real context (every registered identity as
active_peers, the roster’s managed edges). That call publishes the tools and records the customized spawn spec. It builds nothing, except where the seat builds the member from it: a seat that revives the session of a member that was retired after it was built (a respawn, for example), and a declared role migration’s move of the member onto its session.resolved_tools, which reports the tools a build actually resolved, answersno_build_recordeduntil the member’s first build. Yourafter_create(identity, session_id, context), when you define one, is then called with the session the identity is bound to, unless that incarnation has ended by then. It is attempted at least once: a gateway stopped before the call makes it at its next start, over a durable metadata store. Each call has 30 seconds to answer and is made at most three times; one that failed is reported aslifecycle_effect_failedand not made again; - once per build of a member, right before it: when work reaches a member
that is not materialized (a send, a dispatch, a peer message, a kickoff),
and at start for the members that must be live then. No
after_createfollows a build.
customize_build once and after_create
once, with the session the respawn keeps, and the member is built once, with
that call’s tools: the mob keeps the retired member’s session archived, so
the respawn’s seat revives the member on it, and that revival is the build.
The same holds for a materialization, a warm or a delivery whose seat builds
the member: the member stays live for it. The roster pass asks the mob to
release a member its seat built, so that identity is then like a lazily
seated one. A lazily
seated identity (one seated with nothing built yet) gets two
customize_build calls before its first run: its seat’s, then its first
build’s. A respawn of a member that was never built does the same: its seat
calls customize_build, and its build calls it again.
A member is never built without its tools, so its first run sees them. A
restart over idle members the mob already holds calls customize_build for
none of them. A member the gateway builds before your customizer is composed (a
gateway that stopped uncleanly restores the members that hold work while it
builds the mob) is built, then starts no run until the roster pass has
published its tools. A member whose identity the roster no longer lists
starts no run either, until the roster retires it (a host-adopted identity’s
member, which the roster never retires, is not held). A customize_build
that fails refuses the member’s build: the member stays as it was, the
delivery that needed it answers a typed retryable error (the artifact
refusal itself, typed, when the host’s installed artifacts refused the
tools), and the next delivery tries again. While a member’s tools are not
published, its identity status reports customizer_tools_pending with the
reason. A tool from an earlier build that the latest build no longer
declares is refused, never routed to stale handlers.
meta/identity/resolved_tools lists a member’s tools without building it:
a live member’s live tools, or, for a member that is not materialized, the
tools its last build had (tools_source says which, and reads
no_build_recorded for a member not built since the upgrade or since its
fresh life began). runtime.mob_handle().identity_warm(identity) builds a
member without a turn and records its tools; its record_revision counts
the record’s writes and is not the identity’s generation.
Customizer tools belong to roster identities only. Helpers, forks,
flow-provisioned members, and members spawned under their own ids never get
them; use profile tools or the session builder’s per-spawn tools for those.
A raw spawn that reuses a roster identity’s member id does get that
identity’s current tools.
Provider parameters (provider_params)
AgentBuildDraft.provider_params mirrors meerkat’s ProviderParamsOverride:
the shared sampling knobs at the top level and the provider-specific knobs
nested under provider_tag, which is tagged by provider. Anthropic prompt
caching is one such knob. With no profile or request override, the pinned Meerkat
dependency defaults to automatic on Anthropic API, Vertex, and Foundry, and to
disabled on Bedrock and Copilot. Bedrock and Copilot do not support request-wide
automatic caching and reject an explicit automatic policy. On a supported
backend, this example explicitly selects automatic caching with a one-hour TTL:
provider_tag.cache_control to "disabled" and omit
cache_ttl.
The flat form {"cache_control": "automatic"} is refused: both
ProviderParamsOverride and the provider tags deny unknown fields, so the
gateway rejects the returned draft instead of dropping the knob. Knobs the
draft leaves unset are filled from the profile’s own provider_params, and a
provider_tag from a different provider family than the profile’s own tag is a
typed merge error, not a silent union. A declared provider_params also reaches resumed
sessions (the bridge masks it into resume_overrides); it requires an inline
definition profile to carry it, so an identity bound to a realm-reference
profile fails the build with a message naming the realm profile as the place
to declare it.
Mob announcements
With mob announcements enabled (.mob_announcements() on a persistent,
identity-first launch), MobHandle wraps every meta/mob_announcements/*
method. Results are frozen dataclasses; every state, phase, status, outcome,
action and kind is a plain string, so a newer gateway’s value passes
through. The declarations you write (a source’s identity, observation,
audience, routes, …) are dicts in wire form. Optional parameters are sent
only when set.
A
member is {"mob_id": ..., "identity": ...} or an AnnouncementMember.
The method reference describes every
field. A collector step reads the source, then advances it from the
position it read; a step that lost a race raises with the current position:
MobAnnouncementsUnavailableError (-32044),
AnnouncementConflictError (-32045; SourceStaleRevisionError and
SourceVersionConflictError carry the source’s current_version, cursor
and revision), AnnouncementFrontierError (-32046),
AnnouncementRefusedError (-32047) and AnnouncementNotFoundError
(-32048), each with the refusal’s kind (a kind this SDK does not know
keeps its code’s class). A source method on a gateway without sources raises
RpcError -32601 with “Meta does not support sources” in its message.
CapabilitiesResult.mob_announcements (MobAnnouncementsCapability) reads
meta/capabilities.mob_announcements, None when they are disabled.
A batch’s handling and an observation’s handling
(AnnouncementBatchHandling: state, cause, redelivered, held,
dismissed, read_at_ms) say what became of the turn
(Delivery handling). Each is None until
the batch is admitted. A batch that holds nothing for an operator refuses
announcement_delivery_redeliver and announcement_delivery_dismiss with
AnnouncementRefusedError not_held. An AnnouncementAuditEntry for a
plain announcement has source_id and generation None.
Escalations (Mob announcement health)
reach .on_error(callback) as mob_announcement_escalated and
mob_announcement_escalation_resolved, each with its event cursor in
context. After a reconnect, read the events you missed:
StatusResult.mob_announcements (MobAnnouncementsStatus: sources,
members, handling, open_escalations, last_event_cursor) reads
meta/status.mob_announcements. It is None when mob announcements are
disabled or their database could not be read. A source’s
AnnouncementSourceHealth adds escalations, quiet_monitoring,
empty_coverage_monitoring and runner_scheduled_here. Each
AnnouncementEscalation keeps its cause and its subject (a dict, with
its kind as subject_kind) as sent.
A role template (Role templates) subscribes
every member of a mob with a role:
children, and
announcement_subscription_get reads it as template.
announcement_declarations_apply(declarations) applies declared sources
and subscriptions in wire form, with revision defaulting to
declarations["revision"] (Mob announcement declarations).
Called without declarations, it asks a gateway with a declarations
sidecar to read the file again. Each
AnnouncementDeclarationRecord keeps its kind, status and
collection as strings, with refusal_kind and delivery_kind read
from their objects. The refusals declaration_rollback,
declaration_digest_conflict, role_template_limit and
subscription_id_taken are AnnouncementRefusedErrors with their data.
The builder passes declarations or declarations_path in
mob_announcements({...}) unchanged.
Error hierarchy
All SDK errors inherit fromMetaError:
Advanced API
These methods are available for power users and module-system internals. Most apps should usesend, subscribe_agent/subscribe_mob, and status/reconcile instead.
gating_decide() preserves the owner’s recorded approver_id, outcome,
decided_at_ms, reason, and next_pending_id. Older responses that omit these
fields leave them as None; an absent or unrecognized outcome must not be treated
as permission. Escalation returns pending_approval with a successor
next_pending_id. The SDK makes no automatic follow-up request.
Approval records a decision and does not execute the action. These methods expose
Meta gating; native retained-action consent and model-review
configuration are separate contracts.
Mobpack authoring
MobHandle wraps the full Flow Editor authoring surface (meta/mobpacks/*) with snake_case typed methods — mobpack_templates, mobpack_catalogs, mobpack_validate, mobpack_source, mobpack_export, mobpack_import, mobpack_list, mobpack_get, mobpack_create, mobpack_save, mobpack_delete, mobpack_undo, mobpack_redo, mobpack_apply_operation, mobpack_deploy_command, and mobpack_deploy. The same wrappers exist on the SDK’s typed JSON-RPC clients.
mobpack_deploy(document, execute=True) writes the archive and runs rkat mob run on the host — gated by the surface’s advertised authoring_capabilities and, on ABAC-enforced runtimes, the caller’s mobpack.deploy grant. See the Flow Editor guide.
Public surface exports
Everything below is importable directly frommeerkat_meta:
ModuleSpec, define_module, etc.) live in meerkat_meta.helpers — not top-level.
See also
- JSON-RPC API — full method reference
- SSE API — event streaming protocol
- Modules — module configuration and lifecycle
- Rust SDK — primary Rust interface
- TypeScript SDK — TypeScript equivalent
