comms Cargo feature to be compiled in.What this guide is for
Use this guide when you want:- long-lived keep-alive sessions
- peer-to-peer agent messaging
- host-side external event ingress
- a practical operational understanding of the comms system
Overview
The comms system provides:- Five LLM-facing tools:
send_message,reply_to_peer,send_request,send_response, andpeers - Three transport layers: Unix Domain Sockets (UDS), TCP, and in-process (
inproc) - Ed25519 cryptographic identity: Every agent has a keypair; all messages are signed
- Trust-based peer model: Agents only accept messages from explicitly trusted peers
- External event ingestion: Push plain-text events from stdin, webhooks, RPC, or TCP/UDS listeners
- Auth-optional mode: Signed listeners for agent-to-agent comms; separate plain listeners for external events when
auth = "none" - Keep-alive: A runtime-backed session processes its initial prompt, then remains alive waiting for future admitted work
- Runtime-backed queueing: Incoming messages and external events are queued once and admitted through runtime ingress
Architecture
Setup
Enable comms feature
comms Cargo feature is compiled in (enabled by default in the CLI; opt-in on the meerkat facade crate).Generate identity
CoreCommsConfig resolves identity_dir relative to the base directory supplied by its host.Configure trusted peers
trusted_peers.json by hand: a non-empty persisted file is rejected at runtime startup because it is a projection, not trust authority. Route by the derived PeerId, never by the display name.Run with comms enabled
--comms-name <NAME> on the CLI or set enable_comms on AgentFactory.Identity and cryptography
Each agent has an Ed25519 keypair managed by theKeypair type (crates/meerkat-comms/src/identity.rs).
- Key generation:
Keypair::generate()creates a new random keypair usingOsRng. - Key persistence:
Keypair::save(dir)writesidentity.key(mode0600on Unix) andidentity.pubto disk.Keypair::load(dir)reads them back.Keypair::load_or_generate(dir)is the canonical entry point. - Public key format:
PubKeyis a 32-byte Ed25519 public key. The canonical string format ised25519:<base64>(standard Base64 with padding). - Standalone default identity directory:
<base-dir>/.rkat/identity/. Session-scoped factory builds use a durable per-session identity root instead (see where a session’s keys are kept).
Where a session’s keys are kept
With in-process comms, the default, a session with comms has its own key pair, in a directory named by the session id that holdsidentity.key and
identity.pub. That directory is not in the state root and not in the realm:
by default it lies under the user’s home, and the place depends on the command
that built the session.
--user-config-root <root> every one of these commands keeps them under
<root>/.rkat/session_comms_identity/<session id>/ instead. A Rust host that
builds sessions with an AgentFactory gets the same places:
<user_config_root>/.rkat/session_comms_identity when the factory has a
user_config_root, else the per-user state directory of the three servers.
With [comms] mode = "tcp" or "uds" there is no directory per session:
the host has one key pair, in <realm directory>/.rkat/identity/, which lies
inside the realm directory and moves with it. rkat mob host has, besides its
members’ session keys, a host identity directory of its own:
<state root>/host-identity, beside the realm directories, unless
--identity-dir or [mob_host].identity_dir names another.
A backup or a move of a state root or a realm therefore does not hold the
session keys of in-process comms: copy the session_comms_identity directory
with it, to the place the command on the new host reads. A copy of a mob
host’s realm directory alone does not hold its host identity directory either.
Without its session keys a session that is not a mob member is still resumed:
it gets a new key pair, and so a new PeerId, without a word on stderr, and
peers that trusted the old key do not know the new one. A mob member restored
without its keys is marked broken, with the reason
restored member '<id>' cannot publish its comms endpoint: ...; respawn the member to mint a new endpoint.
Envelope signing
Envelope signing
Envelope:(id, from, to, kind) as CBOR, then recursively sorting all map keys by canonical order (RFC 8949) before encoding. This ensures deterministic signing across implementations.Content-bearing kinds (Message / IncarnationFencedMessage / Request /
Response) carry optional signed-when-present metadata fields (blocks,
content_taint, handling_mode, and objective_id). A field that is absent
(None) is omitted from the encoding entirely, so older envelopes stay
byte-identical and keep verifying, while a present field lives inside the
signed region because kind is part of the signable tuple. content_taint is
the sender-declared content-taint declaration (clean / tainted); an
omitted declaration is a real third state that receivers must never coalesce
into clean.Cross-version note: an envelope carrying content_taint sent to an older receiver fails signature verification there (the older signable projection differs), so under the default require_peer_auth = true the envelope is dropped with a typed InvalidSignature error - loud and fail-closed. With require_peer_auth = false the older receiver skips verification and the declaration silently drops out of its typed view.Peer identity and naming
peer_id is the canonical routing key: a UUID derived from the peer’s Ed25519
public key. Every send addresses a peer_id. The name in the peer directory
is a display label; it may collide, and it is never resolved into a route.
Mob members are named by the typed MemberCommsName, rendered
{mob_id}/{role}/{member}, where role is the profile name and member is
the AgentIdentity. That is the string a peer sees in peers output and in
every projected peer message (Peer message from release-triage/analyst/analyst-1:);
projected peer requests carry the peer_id plus that display name. Each
component must start with an ASCII letter or _ and may contain only ASCII
alphanumerics, -, or _, so Meta identity-first members whose identities
carry other characters (for example domain:payments) appear as mk-- encoded
segments (release-triage/domain/mk--domain_cpayments). Meta decodes those
segments only at its console and RPC projections; the model always sees the
encoded form.
Because the mob id is part of every member’s name, renaming a mob (changing
definition.id) re-mints every member’s comms name and moves the members into
a new mob.<id> realm with fresh sessions. Live trust and peer-directory
entries are re-registered under the new names. The old {old_mob}/... names
survive only in identity-keyed state a host keeps outside the mob realm, such
as Meta’s per-identity agent memory, and that is where a renamed mob’s stale
names come back from.
Trust model
Agents maintain trusted peers in aTrustStore (crates/meerkat-comms/src/trust.rs), keyed by the canonical PeerId — a UUIDv5 derived from the peer’s Ed25519 public key. PeerName is display-only metadata: duplicate names across entries are legal, while a duplicate PeerId is a hard error. Resolving a name to a PeerId (TrustStore::resolve_name) returns a typed TrustResolveError::Ambiguous when more than one entry shares the name — the store never guesses.
TrustEntry structure and file format
TrustEntry structure and file format
<runtime-root>/.rkat/trusted_peers.json uses the
stable row shape { name, pubkey, addr, meta }; the PeerId is re-derived
from the pubkey when the file is inspected. This is not a user-editable trust
configuration or restart seed. CommsRuntime rejects a non-empty persisted
seed at startup because live trust must be replayed through generated machine
or composition authority. The shape is shown here for diagnostics only:Trust enforcement
Incoming connections are validated inhandle_connection() (crates/meerkat-comms/src/io_task.rs):
Read envelope
Verify signature
envelope.verify()) when peer auth is enabled (require_peer_auth).Verify recipient
envelope.to == keypair.public_key()).Admit through the inbox seam
inbox_sender.send_connection_ingress(...)), where ingress classification checks the sender against the TrustStore. Untrusted senders are dropped with a typed admission reason (untrusted_sender).ACK or reject
Message and Request kinds). Any failed check returns a typed IoTaskError (invalid signature, misaddressed, inbox full, untrusted sender) so the listener records the rejection rather than silently dropping.Transport layer
Signed peer TCP and UDS transports use the length-prefixed CBOR framing protocol implemented byTransportCodec. Inproc delivers typed envelopes
directly without transport serialization; signing and admission are separate
from framing. Plain external-event listeners use their own protocol, not this
signed peer codec.
Signed peer TCP/UDS wire format: 4 bytes (big-endian) payload length followed by CBOR-encoded Envelope (up to 1 MB max).
Address formats
Transport details
Transport details
spawn_uds_listener(). The socket file is created at the configured path (existing files are removed first). Parent directories are created automatically.TCP transport: TCP listeners are spawned by spawn_tcp_listener(). Accepts connections and processes each in a dedicated tokio task.Inproc transport: The InprocRegistry (crates/meerkat-comms/src/inproc.rs) is a process-global registry segmented by namespace. Meerkat uses realm-scoped namespaces so inproc peers from different realms are isolated by default. Messages are delivered directly in-memory without serialization.InprocRegistry::global()returns the singletonregister_with_meta_in_namespace(...)adds an agent in a namespaceunregister_in_namespace(namespace, pubkey)removes an agent from a namespace- delivery within a namespace is internal to the registry (crate-private send paths keyed by pubkey)
retire_inproc_route()releases a runtime’s own published route at a point the owner chooses, instead of at the lastArcdrop
- a participant name that already has a live route is refused without mutation (
RegistrationRejection::NameOccupied { holder_pubkey }) and the incumbent keeps routing, whichever key holds it. Key identity does not fork the rule: delivery is key-addressed and the registry cannot see hosts, so one peer rebuilding itself and a second live host of that same identity arrive as the same event, and admitting the newcomer would orphan a route that is more likely to be live precisely when the keys match.holder_pubkeyis evidence of who holds the name - it may be the claimant’s own key - not a key or trust failure. There is no name-takeover opt-in. - succession is admitted on evidence, in one of two forms. Either the incumbent releases the name first (
CommsRuntime::retire_inproc_routeat a chosen point, or dropping the runtime), after which the successor publishes ordinarily - including under the same key; or the successor names the exact predecessor generation withPreparedCommsRuntime::publish_replacing. Release and replacement are both generation-exact, so a superseded generation can never remove a successor’s route. - re-registering an existing key under a free name is that registrant’s own rename (
RegistrationOutcome::ReplacedPubkey). The claimed name was unbound, so the one rule is satisfied. - an inbox that already left the registry through its own runtime (the runtime released its route, was dropped, or a successor replaced it) is registered nowhere again (
RegistrationRejection::InboxGone). A route never outlives its inbox: registrations of that inbox in other namespaces leave with it, and a host that read the inbox before it left cannot install a route to it afterwards. The participant is not there now; a seated member is reached through its wake route.
MobHandle::shutdown(), which reaches MobSupervisorBridge::shutdown and releases {mob_id}/__mob_supervisor__). Holding the same persisted supervisor authority key is not a claim on the route.When CommsRuntime is created, it automatically registers itself in the active namespace. When dropped, it unregisters from that namespace (generation-exactly, so a superseded generation’s drop is a no-op).Message types
MessageKind and Status
MessageKind and Status
content_taint and reply_endpoint are omitted when absent, preserving the
signed bytes of older envelopes. reply_endpoint is short-lived response
transport metadata for one exact request correlation, never a durable trust
route. Correlated callback admission requires a signature-verified TCP Request
and combines its signed, nonzero declared port with the connection’s
kernel-observed source IP. The sender-selected host is discarded; payload peer
addresses, UDS endpoints, unsigned/open-auth ingress, and arbitrary sender
addresses are never callback authority. The legacy uncorrelated staging seam
accepts only machine-authorized endpoints already held in runtime state.
Controlled bridge responders use the correlated path for alternate-authority
probes whose temporary listener port differs from the peer’s durable route.objective_id is optional signed causality metadata for content-bearing kinds.
On message, in_reply_to is the delivery a reply_to_peer reply answers: it
is signed when present, omitted when absent, and only a label (routing and
authorization never read it).
incarnation_fenced_message is internal placed-member routing: the exact
recipient generation and fence are signed and checked before work admission.ACK behavior
PeerOffline, but runtime and host surfaces preserve the
uncertainty as SendError::AmbiguousDelivery with the envelope id. A receiver
may have committed the envelope before the acknowledgement was lost, so this
is not automatic retry authority.
MessageIntent variants
MessageIntent variants
MessageIntent enum (crates/meerkat-comms/src/agent/types.rs) is the
receiver-side classification of incoming requests and lifecycle notices. It is
not an input vocabulary: none of the strings in this table can be passed as the
intent of the send_request tool.Custom.The LLM-facing send_request tool accepts a separate, closed intent vocabulary
(CommsPeerRequestIntent in meerkat-core): "checksum_token" and
"supervisor.bridge". Unknown intents fail at the serde boundary. The closure
is deliberate: every intent carries typed params and a typed result, so the
request/response protocol stays fail-closed and schema-generated instead of
free-form. An ordinary ask is send_message or reply_to_peer.LLM-facing tools
Five tools are exposed to the LLM when comms is enabled.send_message
send_message
{"type":"text","text":...} or {"type":"image_ref","source":"current_turn","index":0} / {"type":"image_ref","source":"blob","blob_id":"sha256:...","media_type":"image/png"}."queue" for ordinary delivery or "steer" for immediate steer processing on runtime-backed sessions.{"status": "sent", "kind": "peer_message", "receipt": {...}}send_request
send_request
intent + params and a correlated response via send_response. Requires a runtime-bound command authority; without it the tool reports a typed RuntimeCommandAuthorityUnavailable reason."checksum_token" or "supervisor.bridge". Unknown intents are rejected at the serde boundary. The MessageIntent table above is not this field’s vocabulary; "review", "query", and the other classification strings are invalid arguments here.{"subject": "..."} for checksum_token)."queue" for ordinary delivery or "steer" for immediate steer processing on runtime-backed sessions.{"status": "sent", "kind": "peer_request", "receipt": {...}}Only the peer the request was sent to can answer it. Before the request
leaves, the runtime’s machine authority records its addressee: the canonical
peer ID of the trusted peer it is routed to. A response, progress or terminal,
is accepted only when the key that signed its envelope is that peer’s. A
response signed by any other peer, even one that knows the request ID, is
dropped at ingress before it is queued: nothing reaches the session, a
pending stream or a waiting tool, and the request stays open for its
addressee’s answer. A response to a request ID this session never sent, has
already completed or timed out, or sent before a restart is refused as
before. Under open auth (require_peer_auth = false), ingress does not verify
signatures, so the check compares the key the response envelope declares.
A sender whose response is dropped learns little: over TCP no response frame
is acknowledged, so its send succeeds as for an accepted response; in process
its send fails with AdmissionDropped, which tells a sender holding the
request ID that this session holds the request for another peer. The ID is
only ever sent to the addressee, so a sender that holds it already sees the
exchange.send_message by default for ordinary collaboration. send_request is for structured ask/reply semantics only. It is not task tracking, a stronger delivery mode, or a reserved response channel.send_response
send_response
send_request."queue" appends the response as a durable notice and runs the requester’s reaction turn at its next run start; "steer" additionally preempts the requester’s current run at its next boundary. Either way the response starts a new run and never lands inside the requester’s running turn (see Request and response across turns). Forbidden on "accepted" progress responses.{"status": "sent", "kind": "peer_response", "receipt": {...}}A response, progress or terminal, goes only to the request’s requester. When a
request arrives, the runtime’s machine authority records its requester: the
canonical peer ID of the key that signed the request’s envelope, as ingress
admitted it, never a field of the request’s payload. A response addressed to
any other peer ID is refused before anything is sent. The tool reports it as
invalid_arguments, and the host comms/send command as send_failed. The
request stays open, so the response can still be sent to the requester. Only
the canonical peer_id addresses the response; display_name is diagnostic
and does not change who receives it. A response to a request ID the session
did not receive, has already answered, or received before a restart is
refused as before (execution_failed from the tool).With peer authentication required, ingress verifies the envelope’s signature,
so the recorded requester is the peer that holds that key. Under open auth
(require_peer_auth = false), ingress does not verify signatures, and the
recorded requester is the key the envelope declares: the same identity that
trust and routing use in that mode.A request ID belongs to one request while that request is open. A request,
message or event from another peer (a host event, which has no peer sender,
too) that reuses the ID of a request the session has not yet answered is
refused before the session sees it, and the open request keeps its
requester. A durable peer delivery’s sender gets a rejection
(DurableAdmissionRejected); a volatile delivery is dropped without a
signal, and so is a host event, whose injecting call returned when the
event was queued. The request that first used an ID keeps it; the request
IDs Meerkat generates are random UUIDs, so another peer can only reuse one
it has learned. The requester record lasts for the responder process: after a
restart, an ID whose request is still queued can be claimed by another peer
until it is answered. A host that mints its own request IDs
(send_peer_request_with_id, Router::send_with_id) must keep them
unguessable; a predicted ID can be claimed first, and the host’s own
request is then refused.comms/send returns richer typed receipts such as peer_message_sent, peer_request_sent, and peer_response_sent, with identifiers and ACK-related data for application code.peers
peers
TrustStore (plus runtime-resolved sources when a
runtime command authority is attached). The current agent and private
control-plane routes are excluded. Names are display labels and may not be
unique — always use peer_id for sends.Input: Empty object {}Response:peer_id, never name or a reconstructed address string.
address.transport is one of inproc, uds, or tcp; source is one of
trusted, inproc, trusted_and_inproc, or unknown. Check
sendable_kinds before choosing a comms operation. capabilities is a
versioned extension envelope, while meta is supplementary display and
discovery metadata only.Tool availability
Comms tool availability is reported per tool via the catalog (comms_tool_unavailable_reason in meerkat-comms). send_message,
reply_to_peer, and peers do not require runtime command authority.
reply_to_peer still needs a valid runtime-minted reply capability when it is
called. send_request and send_response require runtime command authority;
without one they are marked unavailable with the typed reason
RuntimeCommandAuthorityUnavailable.
Request and response across turns
Peer traffic is admitted by the runtime machine at boundaries it owns; it is not spliced into whatever the model is doing. These are the rules that matter when one member (the hub) asks others (the spokes) for something and wants to act on the answers:- A
queuedelivery (the default) that reaches a running target is staged for that target’s next run start. It becomes the next turn, never part of the current one. An idle keep-alive session or mob member wakes on a queued peer input by itself (the mob runtime owns the comms drain, including forturn_drivenmembers), so a host nudge is only ever needed for sequencing, not for delivery. - A terminal
send_response(completedorfailed) is appended as a durable notice and run as a new reaction turn of the requester. Withhandling_mode: "steer"it also preempts the requester’s current run at its next boundary, but it still starts a new run; a reply never lands inside the requester’s running turn. Anacceptedprogress response is coalesced into the requester’s next run and does not wake an idle requester. - Only
send_message,reply_to_peer, andsend_requestwithhandling_mode: "steer"inject into a running turn, at the target’s next run checkpoint, and only while the target is mid-turn. Steer aimed at an idle target is normalised toqueueand becomes an ordinary next turn. - There is no agent-facing await-reply primitive. The
waittool was removed in 0.5.2 andsend_requesthas no timeout. A hub that fans out asks and then wants to wait ends its turn; the replies arrive as later turns, and the hub reasons about them then.
delegate: it spawns a helper and returns the helper’s exact bounded result to
the calling turn. For host-driven sequencing, send and then wait for the
completion of the turn that send started (Meta’s send_and_wait and
dispatch_and_wait do this with a completion baseline captured before
delivery), or write the fan-out and fan-in as a mob flow (dispatch_mode: "fan_out" followed by a depends_on step with dispatch_mode: "fan_in"; see
the Mobs guide). send_request remains the correlated
ask/reply protocol for its closed intent set, not a general “ask a peer and
block” call.
Inbox
TheInbox (crates/meerkat-comms/src/inbox.rs) is a bounded queue (default capacity: 1024) with a Notify mechanism for waking waiting tasks.
Inbox::new_transport_only()creates transport plumbing without semantic peer-admission authority; its sender rejects semantic inbox items- runtime peer/event ingress uses the classified queue with an installed machine handle and returns a typed admission outcome (
AdmittedorDropped { reason }); inbox-full, closed-session, and untrusted-sender drops are explicit, never silent - each consumer obtains an exact FIFO-head claim; durable-runtime input remains queued until the runtime commits a typed terminal receipt, while volatile-control input is removed only by an exact handoff
- queue snapshots, claim age, handoff counters, and delivery correlations are read-only diagnostics derived from the canonical queue; they cannot commit, release, or reconstruct admission authority
- queueing is transport/inbox work
- durable consumption is runtime-backed admission with an exact terminal receipt
- volatile consumption is an exact FIFO handoff to a control consumer and does not claim durable admission
ack_timeout_secs. If the bound expires before queue
admission, nothing was delivered. If it expires after admission, the result is
AmbiguousDelivery: the envelope remains on the receiver’s queue and may
still commit. Do not blindly retry an ambiguous send. Reconcile by envelope
or durable work identity first; a retry with a fresh interaction id is not
deduplicated against the original peer delivery. A higher-level durable
idempotency key helps only when it is coarser than the retry. A per-attempt or
timestamped key is not protection.
REST, JSON-RPC comms/send and the MCP meerkat_comms_send tool report this
case as send_ambiguous, not send_failed, with the same typed payload: the
envelope_id, retry_safe: false and required_action: "reconcile".
Configuration
Config file (realm config.toml)
Config file (realm config.toml)
CoreCommsConfig (internal)
CoreCommsConfig (internal)
CoreCommsConfig is the internal config used by CommsRuntime:{name} interpolation (replaced with the agent’s comms name). Relative paths are resolved against the base directory via resolve_paths(base_dir).CLI usage
comms
feature at compile time. Finite --stdin auto / --stdin blob input does not
require comms or keep-alive.Signed TCP remote peer
Userkat run --comms-listen-tcp when the Meerkat itself should expose the
signed agent-to-agent comms channel. This is the path for remote mob members
and other peer-addressable agents; it is separate from rkat-rpc --tcp, which
only exposes the JSON-RPC control plane.
kind: "external" runtime binding containing the
advertised address, the target’s Ed25519 public identity, and the typed
bootstrap_token used by mob supervisor bridge binding. Prefer generating this
file with --comms-binding-out; a binding that only carries an address or a
query-string bootstrap token is not enough for current external mob members.
Peer metadata
Agents can advertise a description and arbitrary labels so that peers can discover what each agent does — not just its name. This metadata flows through to thepeers() tool output.
peers(), the output includes the metadata:
Keep-alive mode
Keep-alive keeps the agent alive after processing the initial prompt, waiting for runtime-backed comms messages and external events to be admitted as future turns:Process initial prompt
Enter idle runtime-backed session
Handle incoming messages and events
Exit conditions
mob.dismiss lifecycle signal from its supervisor, its budget is exhausted, or it encounters a graceful error. A literal “DISMISS” string in a peer message body is ordinary content, not a control signal.SDK / programmatic usage
Building a comms runtime
Building a comms runtime
config.comms.mode and creates the appropriate runtime:Inproc— creates an inproc-only runtime (usebuild_comms_runtime_from_config_scopedto pass a realm namespace)Tcp— creates a full runtime withCommsRuntime::new()and starts TCP listenersUds— creates a full runtime withCommsRuntime::new()and starts UDS listeners
Composing tools with comms
Composing tools with comms
CommsToolSurface (built from
CommsToolMaterial) via DynamicToolComposite, registering send_message,
reply_to_peer, send_request, send_response, and peers.Using AgentFactory
Using AgentFactory
comms_name, creates the runtime, composes tools, attaches the runtime to the agent, and records keep_alive and peer_meta in SessionMetadata.CommsBootstrap (nested runtime integration)
CommsBootstrap (nested runtime integration)
PreparedComms contains:runtime: CommsRuntime— ready to useadvertise: Option<CommsAdvertise>— for child agents, contains the name/pubkey/addr to register with the parent
Agent loop integration
Inbox consumption
The important architectural split is:CommsRuntimeand the inbox own delivery- the active keep-alive/drain lifecycle path owns durable admission, while explicit control responders own volatile handoff
- formatted message injection into the session is a projection of the durable lifecycle truth
Transcript and model projection
Incoming comms are persisted as typedsystem_notice blocks with comms payloads. role=user remains reserved for human/operator-authored text. The runtime projects those typed blocks into provider-facing text only while assembling model input:
- Message:
comms.kind = "message"plus peer identity and optional content blocks. - Request:
comms.kind = "request"plus peer identity, request id, intent, params payload, and response guidance in the model projection. - Response:
comms.kind = "response_terminal"orresponse_progressplus peer identity, request id, status, and payload.
peer_content_ingested event containing the typed comms kind, canonical peer
fact, request correlation, the delivery id of a peer message, and the sender’s
signed content-taint declaration.
This event is best-effort observation. The typed transcript block remains the
durable owner, and an absent taint declaration remains distinct from clean.
Typed peer lifecycle notices
Some peer lifecycle notices are informational and should not trigger an LLM turn. Mob lifecycle routing is typed at peer ingress (PeerLifecycleKind) instead of relying on silent_comms_intents folklore:
mob.peer_addedmob.peer_retiredmob.peer_unwiredmob.dismiss(supervisor-directed dismissal of a live executor)
mob.kickoff_pendingmob.kickoff_startingmob.kickoff_startedmob.kickoff_callback_pendingmob.kickoff_failedmob.kickoff_cancelled
lifecycle comms notice naming the kind, no inbound request is recorded, and no send_response reply is expected.
Kickoff notices are progress hints, not a guaranteed record. The member’s kickoff phase in its status is the fact; a peer can miss a notice:
- Order: the notices of one member reach one peer one at a time, in the order the kickoff went through its phases. A notice to a peer that is not live wakes it and is delivered once the peer is live. Notices to different peers, and of different members, are independent.
- Bound: at most eight notices of one member are pending for one peer, the one being sent included. A notice owed beyond that takes the place of the oldest one still waiting (never of the one being sent), so the latest phase, a failure or a cancellation included, survives. The bound is per pair of members; there is none across peers or members.
- No retry: a notice whose send fails, or whose send budget ends (the sending member’s comms ack timeout, 30 seconds by default), is not sent again. The next notice of that member to that peer is attempted when that budget has ended.
- Cancel: a stop, a complete, a reset, a shutdown or a destroy of the mob, an unwire of the edge, a retirement of either member and the rollback of a failed spawn cancel the notices that are pending when they start. A notice such a command emits itself (the cancellation of a kickoff that a stop closes) is still handed off and ends within its send budget; the stop does not wait for it, and nothing is delivered after a destroy.
from), the peer (to), the kind (intent) and a class: preparation_failed, send_failed, pending_bound or cancelled. Not confirmed is not the same as not delivered: a notice whose send budget ended, or that was cancelled, after its recipient had admitted it can still arrive, once. Three cases have no such record: a sending member with no comms runtime (nothing is prepared), a placed sending member (logged without the peer and the class), and notices still pending when the mob’s actor exits.
Batched peer lifecycle updates are presentation-only: the underlying comms stream remains one event per peer change.
You can control whether these peer lifecycle updates are inlined by peer-count threshold using max_inline_peer_notifications:
None(default): use runtime default (50)0: never inline peer lifecycle updates-1: always inline>0: inline only when current peer count is less than or equal to the threshold< -1: invalid (rejected by mob definition validation; factory builds also reject)
peers() on demand to inspect the current roster.
External event ingestion
External systems can push events into a running agent without Ed25519 authentication. Surface-level convenience routes now admit those events through runtime-ownedExternalEvent inputs instead of a separate injector-owned execution path.
Event sources
CLI stdin events
Use--stdin lines for newline-delimited events; it implies keep-alive on the
normal CLI path. --stdin auto (the default) and --stdin blob instead read
finite piped input as prompt context, while --stdin off ignores stdin.
Line events default to --line-format text, which preserves each line
literally, including JSON-looking text. Opt into --line-format json to retain
the whole structured JSON value (falling back to text for non-JSON input).
It never extracts a guessed body property: the object in this example stays
an object rather than becoming just "deployment failed on prod".
REST webhook
Push events to a running session via HTTP. Auth is optional viaRKAT_WEBHOOK_SECRET env var with constant-time comparison.
202 Accepted with {"queued": true}. Runtime admission failures are returned as ordinary REST errors.
RPC session/external_event
Queue a runtime-backed external event for a running session via JSON-RPC.kind (generic_json), event_type, payload, and optional blocks. The method returns a runtime acceptance envelope on success.
TCP/UDS plain event listeners
Whenauth = "none" in config, a separate plain-text listener starts on event_address for unauthenticated external events. The signed agent-to-agent listener is never replaced.
Event transcript format
External events are persisted as typedexternal_event system-notice blocks with source, event_type, optional body, payload, and content blocks. The provider-facing source/body text is an internal projection and is not stored as user-authored transcript text.
- Stdin:
source = "stdin" - Webhook:
source = "webhook" - RPC:
source = "rpc"with optional structured source metadata. - TCP/UDS:
source = "tcp"orsource = "uds"
Delivery vs observation
When injecting events from application code, delivery and observation are separate concerns. Delivery — the service’sevent_injector() returns an Arc<dyn EventInjector>. Use inject() to push an external event into the session inbox:
- session-scoped: use the session’s primary
event_txorsession/stream_open(RPC) /Session.subscribe()(Web SDK) /session.subscribe_events()(Python SDK) /session.subscribeEvents()(TypeScript SDK) - agent-scoped: use
MobHandle::subscribe_agent_events(...)ormob/stream_open(RPC) /Member.subscribe()orMob.subscribeMemberEvents(agentIdentity)(Web SDK) /mob.subscribe_member_events(agent_identity)(Python SDK) /mob.subscribeMemberEvents(agentIdentity)(TypeScript SDK) - mob-scoped: use
MobHandle::subscribe_mob_events(...)ormob/stream_open(RPC) /Mob.subscribeEvents()(Web SDK) /mob.subscribe_events()(Python SDK) /mob.subscribeEvents()(TypeScript SDK)
- The agent must be running in keep-alive mode if you expect inbox events to be drained promptly.
- Comms must be enabled. Without comms,
event_injector()returnsNone. - Observation subscriptions are independent of delivery — configure them separately.
Security
- All messages are signed with Ed25519 using canonical CBOR encoding
- Trust is explicit: only messages from peers in the live, machine-authorized
TrustStoreare accepted; the persisted JSON projection is not startup authority - Misaddressed messages are dropped: the receiver verifies
envelope.tomatches its own public key - Private keys are stored with restrictive permissions:
identity.keyis written with mode0600on Unix - Secret bytes are zeroized:
Keypair::from_secret()zeroizes the input after copying - ACK validation: ACK signatures, sender, recipient, and
in_reply_toID are all verified - Exact replay handling: durable runtime ingress deduplicates the same interaction identity instead of starting a second turn
See also
- Built-in tools reference - comms tool parameter details
- Configuration: comms - config file settings
- Examples: comms - keep-alive mode and messaging across surfaces
