Mobs are the multi-agent path in Meerkat. The agent-facing
delegate tool,
explicit mob_* tools, SDK Mob classes, mob/* RPC methods, public MCP mob
tools, and mobpack deployment all route through the mob system.Choose A Path
Mental Model
AMobDefinition describes profiles, limits, wiring rules, topology, and
flows. The running mob records the roster, member lifecycle, events, flow runs,
and work state. Each member is an agent session, but public mob APIs address the
member by stable AgentIdentity, not by an internal session or runtime binding.
Core Concepts
Agent Tools Vs Host APIs
Keep this distinction sharp:
Agent-side tools are late-bound through the session build path. Host APIs are
the stable control plane for applications and SDKs. Do not treat raw
mob_*
agent tools as if they were JSON-RPC methods.
Fast Path: Delegate
delegate creates an implicit session-owned mob on first use, spawns a helper,
and wires the helper to the creating session. Use it for bounded helper work
that should report back.
Define A Mob
A small mob definition has profiles and optional wiring:Profile model and provider fields
Beyondmodel, a profile can pin provider identity and per-member behavior:
Custom model registry entries
[models.<id>] tables declare uncatalogued models once, at the definition
level. One entry feeds provider inference, compaction scaling, capability
gates, and call timeouts for every profile that references the model:
[models.<id>] table shape works in the host’s config.toml under
[models], next to the per-provider default model strings.
At load time, rkat mob validate rejects a profile model that is neither
catalogued, custom-defined under [models.<id>], nor provider-annotated
(unknown_model), instead of failing at the member’s first delivery.
Create And Spawn
- JSON-RPC
- Python
- TypeScript
- Web SDK
- CLI
Autonomous members run as long-lived peers.
turn_driven members are useful
when a host wants explicit dispatch control.
Identity And Respawn
Mobs separate stable member identity from runtime binding details:
Use
AgentIdentity for facts that survive respawn, such as wiring and durable
configuration. Runtime IDs and fence tokens protect the lower-level binding.
Wire Peers
Wiring controls which members can see and message each other.- JSON-RPC
- Python
- TypeScript
Send Work
Use member send for direct content delivery. Use the work lane when the caller needs a tracked, cancellable work reference.- Member send
- Tracked work
member_ref is an opaque handle returned by spawn, member list, member send,
helper spawn, fork, and respawn responses. Application code should pass it back
to work-lane APIs as-is instead of constructing it from mob_id and
agent_identity.
Flows
Flows are declarative mob workflows. They let a host dispatch repeatable work without hard-coding all member turns in application code. The classic flow shape is a flat DAG: steps declare roles, messages, dependencies, fan-out/fan-in behavior, optional conditions, and tool overlays. Frame-based flows add nestedFlowSpec.root frames and repeat_until loops.
Both are owned by the mob runtime; support modules such as flow-run projection
are not separate public machines.
- JSON-RPC
- Python
- TypeScript
- CLI
Observe And Operate
The event log is append-only. For UI and service loops, prefer event cursors or
SDK subscriptions over repeated full snapshots.
rkat mob force-cancel <MOB_ID> <AGENT_IDENTITY> (RPC mob/force_cancel)
cooperatively cancels a member’s in-flight turn without retiring it — the
member stays in the roster and can take new turns. As the operator remedy for
a wedged member it is legal whenever the mob is running and idempotent:
cancelling a member whose runtime is no longer live, or one already retiring,
converges as a no-op success rather than an admission error. Only an identity
the roster has never seen is refused, with a typed MemberNotFound.
Persistence
Persistent mobs use SQLite/WAL-backed storage. In-memory storage is used for tests and WASM/browser-embedded paths. The mob store is realm-scoped in the runtime-backed surfaces, so a process restart can recover mob state, members, events, flow snapshots, and work records.Supervisor Rotation
mob/rotate_supervisor is a synchronous-looking view over a durable
operation. The mob records a stable operation ID and the complete target
authority before sending the one-way handoff to any member. Each member fences
the old epoch, advances the handoff independently, and exposes a durable
pending, completed, or rejected receipt for that operation ID.
A caller timeout means only that the terminal receipt was not observed before
the deadline. It does not cancel or roll back the member operation; a retry
uses the persisted operation ID and resumes observation. See
Delivery, Interaction, and Durable Operations
for the protocol boundary and recovery semantics.
Multi-Host Placement
Use a member-host daemon when the controlling mob should place and supervise members on another machine. The daemon uses its own restart-stable realm, ideally a dedicated realm or context root; realms are not distributed and do not need to match the controller’s realm.--isolated is rejected because it
would select a new throwaway realm on every restart and lose the daemon’s
durable member-host state.
advertise_tcp must be dialable from every member host. Without an explicit
controller endpoint, mixed-host route installation remains pending and fails
closed rather than publishing a process-local address.
The descriptor handoff above does not require network pairing. If you enable
the host’s pairing branch, provide its runtime-only secret through
--pairing-password-env <ENV> or --pairing-password-file <PATH> so it stays
out of process arguments. These options conflict with each other and with the
compatibility-only --pairing-password <PASSWORD> form. The secret is never
written to [mob_host] configuration and must be at least 32 bytes.
The bind report’s host id is the placement value for mob/spawn and
mob/spawn_many. Omitting placement keeps the member local:
mob/hosts, mob/route_installs, and mob/member_history (or the
matching CLI commands) for placement diagnostics. Python and TypeScript expose
the full typed RPC family. REST and public MCP intentionally expose the three
read-only observations, while host binding, grants, hard cancel, and member
live control remain outside REST and public MCP. Host binding, grants, and
member live control also have explicit CLI verbs; hard cancel is RPC/SDK-only
(rkat mob force-cancel is the distinct cooperative boundary cancel).
Browser/WASM mobs remain single-host and reject non-local placement through
their typed capability boundary.
Embedded and explicitly delegated non-owner console principals need control
scopes. Stock v1 RPC, REST, stdio, MCP, and CLI entrypoints mint the owning
console principal; manage narrower delegated principals with rkat mob grant,
rkat mob revoke-grant, and rkat mob grants. Scope denials and
host/cursor/fence failures are typed consistently across console surfaces.
External Members
Most members are normal session-backed agents. External members are advanced: they require an external runtime binding with a concrete address and trusted peer identity so the orchestrator can route supervisor bridge traffic to the right process. This binds one already-running external member; for a managed host that can materialize multiple placed members, use the member-host flow above. For a remoterkat process, start it with the signed comms listener and
write the binding file:
--comms-binding-out writes the current external binding shape: kind: "external", advertised address, Ed25519 public identity, and typed
bootstrap_token. Current mob supervisors require that typed bootstrap token
for external bridge binding; a bare External backend tag, a raw address, or a
query-string-only bootstrap token is rejected.
Use external members only when the member must run outside the local Meerkat
runtime, such as another host, sandbox, or service process.
Live Channels
Live channels are per session. To use live audio/text or model-gated image input with a mob member, give that member a realtime-capable model such asgpt-realtime-2, spawn the member, then open a live channel against the
member’s session through the live/* surface.
See Live channels.
Mobpacks
A mobpack packages a mob definition and optional assets into a portable artifact:mob web build copies required prebuilt
wasm-pack output into that bundle; it does not compile wasm32.
Troubleshooting
See Also
Mob architecture
Runtime ownership, member identity, flows, persistence, and live-channel boundaries.
Mobs concept
The conceptual model behind members, profiles, wiring, and host-vs-agent surfaces.
Mobpack
Package, sign, validate, deploy, and build browser-target mob artifacts.
