comms Cargo feature to be compiled in.What this guide is for
Use this guide when you want:- long-lived keep-alive sessions
- peer-to-peer agent messaging
- host-side external event ingress
- a practical operational understanding of the comms system
Overview
The comms system provides:- Five LLM-facing tools:
send_message,reply_to_peer,send_request,send_response, andpeers - Three transport layers: Unix Domain Sockets (UDS), TCP, and in-process (
inproc) - Ed25519 cryptographic identity: Every agent has a keypair; all messages are signed
- Trust-based peer model: Agents only accept messages from explicitly trusted peers
- External event ingestion: Push plain-text events from stdin, webhooks, RPC, or TCP/UDS listeners
- Auth-optional mode: Signed listeners for agent-to-agent comms; separate plain listeners for external events when
auth = "none" - Keep-alive: A runtime-backed session processes its initial prompt, then remains alive waiting for future admitted work
- Runtime-backed queueing: Incoming messages and external events are queued once and admitted through runtime ingress
Architecture
Setup
Enable comms feature
comms Cargo feature is compiled in (enabled by default in the CLI; opt-in on the meerkat facade crate).Generate identity
CoreCommsConfig resolves identity_dir relative to the base directory supplied by its host.Configure trusted peers
trusted_peers.json by hand: a non-empty persisted file is rejected at runtime startup because it is a projection, not trust authority. Route by the derived PeerId, never by the display name.Run with comms enabled
--comms-name <NAME> on the CLI or set enable_comms on AgentFactory.Identity and cryptography
Each agent has an Ed25519 keypair managed by theKeypair type (meerkat-comms/src/identity.rs).
- Key generation:
Keypair::generate()creates a new random keypair usingOsRng. - Key persistence:
Keypair::save(dir)writesidentity.key(mode0600on Unix) andidentity.pubto disk.Keypair::load(dir)reads them back.Keypair::load_or_generate(dir)is the canonical entry point. - Public key format:
PubKeyis a 32-byte Ed25519 public key. The canonical string format ised25519:<base64>(standard Base64 with padding). - Standalone default identity directory:
<base-dir>/.rkat/identity/. Session-scoped factory builds use a durable per-session identity root instead.
Envelope signing
Envelope signing
Envelope:(id, from, to, kind) as CBOR, then recursively sorting all map keys by canonical order (RFC 8949) before encoding. This ensures deterministic signing across implementations.Content-bearing kinds (Message / IncarnationFencedMessage / Request /
Response) carry optional signed-when-present metadata fields (blocks,
content_taint, handling_mode, and objective_id). A field that is absent
(None) is omitted from the encoding entirely, so older envelopes stay
byte-identical and keep verifying, while a present field lives inside the
signed region because kind is part of the signable tuple. content_taint is
the sender-declared content-taint declaration (clean / tainted); an
omitted declaration is a real third state that receivers must never coalesce
into clean.Cross-version note: an envelope carrying content_taint sent to an older receiver fails signature verification there (the older signable projection differs), so under the default require_peer_auth = true the envelope is dropped with a typed InvalidSignature error - loud and fail-closed. With require_peer_auth = false the older receiver skips verification and the declaration silently drops out of its typed view.Peer identity and naming
peer_id is the canonical routing key: a UUID derived from the peer’s Ed25519
public key. Every send addresses a peer_id. The name in the peer directory
is a display label; it may collide, and it is never resolved into a route.
Mob members are named by the typed MemberCommsName, rendered
{mob_id}/{role}/{member}, where role is the profile name and member is
the AgentIdentity. That is the string a peer sees in peers output and in
every projected peer message (Peer message from release-triage/analyst/analyst-1:);
projected peer requests carry the peer_id plus that display name. Each
component must start with an ASCII letter or _ and may contain only ASCII
alphanumerics, -, or _, so MobKit identity-first members whose identities
carry other characters (for example domain:payments) appear as mk-- encoded
segments (release-triage/domain/mk--domain_cpayments). MobKit decodes those
segments only at its console and RPC projections; the model always sees the
encoded form.
Because the mob id is part of every member’s name, renaming a mob (changing
definition.id) re-mints every member’s comms name and moves the members into
a new mob.<id> realm with fresh sessions. Live trust and peer-directory
entries are re-registered under the new names. The old {old_mob}/... names
survive only in identity-keyed state a host keeps outside the mob realm, such
as MobKit’s per-identity agent memory, and that is where a renamed mob’s stale
names come back from.
Trust model
Agents maintain trusted peers in aTrustStore (meerkat-comms/src/trust.rs), keyed by the canonical PeerId — a UUIDv5 derived from the peer’s Ed25519 public key. PeerName is display-only metadata: duplicate names across entries are legal, while a duplicate PeerId is a hard error. Resolving a name to a PeerId (TrustStore::resolve_name) returns a typed TrustResolveError::Ambiguous when more than one entry shares the name — the store never guesses.
TrustEntry structure and file format
TrustEntry structure and file format
<runtime-root>/.rkat/trusted_peers.json uses the
stable row shape { name, pubkey, addr, meta }; the PeerId is re-derived
from the pubkey when the file is inspected. This is not a user-editable trust
configuration or restart seed. CommsRuntime rejects a non-empty persisted
seed at startup because live trust must be replayed through generated machine
or composition authority. The shape is shown here for diagnostics only:Trust enforcement
Incoming connections are validated inhandle_connection() (meerkat-comms/src/io_task.rs):
Read envelope
Verify signature
envelope.verify()) when peer auth is enabled (require_peer_auth).Verify recipient
envelope.to == keypair.public_key()).Admit through the inbox seam
inbox_sender.send_connection_ingress(...)), where ingress classification checks the sender against the TrustStore. Untrusted senders are dropped with a typed admission reason (untrusted_sender).ACK or reject
Message and Request kinds). Any failed check returns a typed IoTaskError (invalid signature, misaddressed, inbox full, untrusted sender) so the listener records the rejection rather than silently dropping.Transport layer
Signed peer TCP and UDS transports use the length-prefixed CBOR framing protocol implemented byTransportCodec. Inproc delivers typed envelopes
directly without transport serialization; signing and admission are separate
from framing. Plain external-event listeners use their own protocol, not this
signed peer codec.
Signed peer TCP/UDS wire format: 4 bytes (big-endian) payload length followed by CBOR-encoded Envelope (up to 1 MB max).
Address formats
Transport details
Transport details
spawn_uds_listener(). The socket file is created at the configured path (existing files are removed first). Parent directories are created automatically.TCP transport: TCP listeners are spawned by spawn_tcp_listener(). Accepts connections and processes each in a dedicated tokio task.Inproc transport: The InprocRegistry (meerkat-comms/src/inproc.rs) is a process-global registry segmented by namespace. Meerkat uses realm-scoped namespaces so inproc peers from different realms are isolated by default. Messages are delivered directly in-memory without serialization.InprocRegistry::global()returns the singletonregister_with_meta_in_namespace(...)adds an agent in a namespaceunregister_in_namespace(namespace, pubkey)removes an agent from a namespace- delivery within a namespace is internal to the registry (crate-private send paths keyed by pubkey)
retire_inproc_route()releases a runtime’s own published route at a point the owner chooses, instead of at the lastArcdrop
- a participant name that already has a live route is refused without mutation (
RegistrationRejection::NameOccupied { holder_pubkey }) and the incumbent keeps routing, whichever key holds it. Key identity does not fork the rule: delivery is key-addressed and the registry cannot see hosts, so one peer rebuilding itself and a second live host of that same identity arrive as the same event, and admitting the newcomer would orphan a route that is more likely to be live precisely when the keys match.holder_pubkeyis evidence of who holds the name - it may be the claimant’s own key - not a key or trust failure. There is no name-takeover opt-in. - succession is admitted on evidence, in one of two forms. Either the incumbent releases the name first (
CommsRuntime::retire_inproc_routeat a chosen point, or dropping the runtime), after which the successor publishes ordinarily - including under the same key; or the successor names the exact predecessor generation withPreparedCommsRuntime::publish_replacing. Release and replacement are both generation-exact, so a superseded generation can never remove a successor’s route. - re-registering an existing key under a free name is that registrant’s own rename (
RegistrationOutcome::ReplacedPubkey). The claimed name was unbound, so the one rule is satisfied.
MobHandle::shutdown(), which reaches MobSupervisorBridge::shutdown and releases {mob_id}/__mob_supervisor__). Holding the same persisted supervisor authority key is not a claim on the route.When CommsRuntime is created, it automatically registers itself in the active namespace. When dropped, it unregisters from that namespace (generation-exactly, so a superseded generation’s drop is a no-op).Message types
MessageKind and Status
MessageKind and Status
content_taint and reply_endpoint are omitted when absent, preserving the
signed bytes of older envelopes. reply_endpoint is short-lived response
transport metadata for one exact request correlation, never a durable trust
route. Correlated callback admission requires a signature-verified TCP Request
and combines its signed, nonzero declared port with the connection’s
kernel-observed source IP. The sender-selected host is discarded; payload peer
addresses, UDS endpoints, unsigned/open-auth ingress, and arbitrary sender
addresses are never callback authority. The legacy uncorrelated staging seam
accepts only machine-authorized endpoints already held in runtime state.
Controlled bridge responders use the correlated path for alternate-authority
probes whose temporary listener port differs from the peer’s durable route.objective_id is optional signed causality metadata for content-bearing kinds.
incarnation_fenced_message is internal placed-member routing: the exact
recipient generation and fence are signed and checked before work admission.ACK behavior
PeerOffline, but runtime and host surfaces preserve the
uncertainty as SendError::AmbiguousDelivery with the envelope id. A receiver
may have committed the envelope before the acknowledgement was lost, so this
is not automatic retry authority.
MessageIntent variants
MessageIntent variants
MessageIntent enum (meerkat-comms/src/agent/types.rs) is the
receiver-side classification of incoming requests and lifecycle notices. It is
not an input vocabulary: none of the strings in this table can be passed as the
intent of the send_request tool.Custom.The LLM-facing send_request tool accepts a separate, closed intent vocabulary
(CommsPeerRequestIntent in meerkat-core): "checksum_token" and
"supervisor.bridge". Unknown intents fail at the serde boundary. The closure
is deliberate: every intent carries typed params and a typed result, so the
request/response protocol stays fail-closed and schema-generated instead of
free-form. An ordinary ask is send_message or reply_to_peer.LLM-facing tools
Five tools are exposed to the LLM when comms is enabled.send_message
send_message
peers tool) to send the message to.{"type":"text","text":...} or {"type":"image_ref","source":"current_turn","index":0} / {"type":"image_ref","source":"blob","blob_id":"sha256:...","media_type":"image/png"}."queue" for ordinary delivery or "steer" for immediate steer processing on runtime-backed sessions.{"status": "sent", "kind": "peer_message", "receipt": {...}}send_request
send_request
intent + params and a correlated response via send_response. Requires a runtime-bound command authority; without it the tool reports a typed RuntimeCommandAuthorityUnavailable reason.peers tool) to send the request to."checksum_token" or "supervisor.bridge". Unknown intents are rejected at the serde boundary. The MessageIntent table above is not this field’s vocabulary; "review", "query", and the other classification strings are invalid arguments here.{"subject": "..."} for checksum_token).image_ref entries)."queue" for ordinary delivery or "steer" for immediate steer processing on runtime-backed sessions.{"status": "sent", "kind": "peer_request", "receipt": {...}}send_message by default for ordinary collaboration. send_request is for structured ask/reply semantics only. It is not task tracking, a stronger delivery mode, or a reserved response channel.send_response
send_response
send_request.peers tool) to send the response to."accepted", "completed", "failed".null.image_ref entries)."queue" appends the response as a durable notice and runs the requester’s reaction turn at its next run start; "steer" additionally preempts the requester’s current run at its next boundary. Either way the response starts a new run and never lands inside the requester’s running turn (see Request and response across turns). Forbidden on "accepted" progress responses.{"status": "sent", "kind": "peer_response", "receipt": {...}}comms/send returns richer typed receipts such as peer_message_sent, peer_request_sent, and peer_response_sent, with identifiers and ACK-related data for application code.peers
peers
TrustStore (plus runtime-resolved sources when a
runtime command authority is attached). The current agent and private
control-plane routes are excluded. Names are display labels and may not be
unique — always use peer_id for sends.Input: Empty object {}Response:peer_id, never name or a reconstructed address string.
address.transport is one of inproc, uds, or tcp; source is one of
trusted, inproc, trusted_and_inproc, or unknown. Check
sendable_kinds before choosing a comms operation. capabilities is a
versioned extension envelope, while meta is supplementary display and
discovery metadata only.Tool availability
Comms tool availability is reported per tool via the catalog (comms_tool_unavailable_reason in meerkat-comms). send_message,
reply_to_peer, and peers do not require runtime command authority.
reply_to_peer still needs a valid runtime-minted reply capability when it is
called. send_request and send_response require runtime command authority;
without one they are marked unavailable with the typed reason
RuntimeCommandAuthorityUnavailable.
Request and response across turns
Peer traffic is admitted by the runtime machine at boundaries it owns; it is not spliced into whatever the model is doing. These are the rules that matter when one member (the hub) asks others (the spokes) for something and wants to act on the answers:- A
queuedelivery (the default) that reaches a running target is staged for that target’s next run start. It becomes the next turn, never part of the current one. An idle keep-alive session or mob member wakes on a queued peer input by itself (the mob runtime owns the comms drain, including forturn_drivenmembers), so a host nudge is only ever needed for sequencing, not for delivery. - A terminal
send_response(completedorfailed) is appended as a durable notice and run as a new reaction turn of the requester. Withhandling_mode: "steer"it also preempts the requester’s current run at its next boundary, but it still starts a new run; a reply never lands inside the requester’s running turn. Anacceptedprogress response is coalesced into the requester’s next run and does not wake an idle requester. - Only
send_message,reply_to_peer, andsend_requestwithhandling_mode: "steer"inject into a running turn, at the target’s next run checkpoint, and only while the target is mid-turn. Steer aimed at an idle target is normalised toqueueand becomes an ordinary next turn. - There is no agent-facing await-reply primitive. The
waittool was removed in 0.5.2 andsend_requesthas no timeout. A hub that fans out asks and then wants to wait ends its turn; the replies arrive as later turns, and the hub reasons about them then.
delegate: it spawns a helper and returns the helper’s exact bounded result to
the calling turn. For host-driven sequencing, send and then wait for the
completion of the turn that send started (MobKit’s send_and_wait and
dispatch_and_wait do this with a completion baseline captured before
delivery), or write the fan-out and fan-in as a mob flow (dispatch_mode: "fan_out" followed by a depends_on step with dispatch_mode: "fan_in"; see
the Mobs guide). send_request remains the correlated
ask/reply protocol for its closed intent set, not a general “ask a peer and
block” call.
Inbox
TheInbox (meerkat-comms/src/inbox.rs) is a bounded queue (default capacity: 1024) with a Notify mechanism for waking waiting tasks.
Inbox::new_transport_only()creates transport plumbing without semantic peer-admission authority; its sender rejects semantic inbox items- runtime peer/event ingress uses the classified queue with an installed machine handle and returns a typed admission outcome (
AdmittedorDropped { reason }); inbox-full, closed-session, and untrusted-sender drops are explicit, never silent - each consumer obtains an exact FIFO-head claim; durable-runtime input remains queued until the runtime commits a typed terminal receipt, while volatile-control input is removed only by an exact handoff
- queue snapshots, claim age, handoff counters, and delivery correlations are read-only diagnostics derived from the canonical queue; they cannot commit, release, or reconstruct admission authority
- queueing is transport/inbox work
- durable consumption is runtime-backed admission with an exact terminal receipt
- volatile consumption is an exact FIFO handoff to a control consumer and does not claim durable admission
ack_timeout_secs. If the bound expires before queue
admission, nothing was delivered. If it expires after admission, the result is
AmbiguousDelivery: the envelope remains on the receiver’s queue and may
still commit. Do not blindly retry an ambiguous send. Reconcile by envelope
or durable work identity first; a retry with a fresh interaction id is not
deduplicated against the original peer delivery. A higher-level durable
idempotency key helps only when it is coarser than the retry. A per-attempt or
timestamped key is not protection.
REST reports this case as send_ambiguous, not send_failed, and includes
envelope_id, retry_safe: false, and required_action: "reconcile" in the
details payload.
Configuration
Config file (realm config.toml)
Config file (realm config.toml)
CoreCommsConfig (internal)
CoreCommsConfig (internal)
CoreCommsConfig is the internal config used by CommsRuntime:{name} interpolation (replaced with the agent’s comms name). Relative paths are resolved against the base directory via resolve_paths(base_dir).CLI usage
comms
feature at compile time. Finite --stdin auto / --stdin blob input does not
require comms or keep-alive.Signed TCP remote peer
Userkat run --comms-listen-tcp when the Meerkat itself should expose the
signed agent-to-agent comms channel. This is the path for remote mob members
and other peer-addressable agents; it is separate from rkat-rpc --tcp, which
only exposes the JSON-RPC control plane.
kind: "external" runtime binding containing the
advertised address, the target’s Ed25519 public identity, and the typed
bootstrap_token used by mob supervisor bridge binding. Prefer generating this
file with --comms-binding-out; a binding that only carries an address or a
query-string bootstrap token is not enough for current external mob members.
Peer metadata
Agents can advertise a description and arbitrary labels so that peers can discover what each agent does — not just its name. This metadata flows through to thepeers() tool output.
peers(), the output includes the metadata:
Keep-alive mode
Keep-alive keeps the agent alive after processing the initial prompt, waiting for runtime-backed comms messages and external events to be admitted as future turns:Process initial prompt
Enter idle runtime-backed session
Handle incoming messages and events
Exit conditions
mob.dismiss lifecycle signal from its supervisor, its budget is exhausted, or it encounters a graceful error. A literal “DISMISS” string in a peer message body is ordinary content, not a control signal.SDK / programmatic usage
Building a comms runtime
Building a comms runtime
config.comms.mode and creates the appropriate runtime:Inproc— creates an inproc-only runtime (usebuild_comms_runtime_from_config_scopedto pass a realm namespace)Tcp— creates a full runtime withCommsRuntime::new()and starts TCP listenersUds— creates a full runtime withCommsRuntime::new()and starts UDS listeners
Composing tools with comms
Composing tools with comms
CommsToolSurface (built from
CommsToolMaterial) via DynamicToolComposite, registering send_message,
reply_to_peer, send_request, send_response, and peers.Using AgentFactory
Using AgentFactory
comms_name, creates the runtime, composes tools, attaches the runtime to the agent, and records keep_alive and peer_meta in SessionMetadata.CommsBootstrap (nested runtime integration)
CommsBootstrap (nested runtime integration)
PreparedComms contains:runtime: CommsRuntime— ready to useadvertise: Option<CommsAdvertise>— for child agents, contains the name/pubkey/addr to register with the parent
Agent loop integration
Inbox consumption
The important architectural split is:CommsRuntimeand the inbox own delivery- the active keep-alive/drain lifecycle path owns durable admission, while explicit control responders own volatile handoff
- formatted message injection into the session is a projection of the durable lifecycle truth
Transcript and model projection
Incoming comms are persisted as typedsystem_notice blocks with comms payloads. role=user remains reserved for human/operator-authored text. The runtime projects those typed blocks into provider-facing text only while assembling model input:
- Message:
comms.kind = "message"plus peer identity and optional content blocks. - Request:
comms.kind = "request"plus peer identity, request id, intent, params payload, and response guidance in the model projection. - Response:
comms.kind = "response_terminal"orresponse_progressplus peer identity, request id, status, and payload.
peer_content_ingested event containing the typed comms kind, canonical peer
fact, request correlation, and the sender’s signed content-taint declaration.
This event is best-effort observation. The typed transcript block remains the
durable owner, and an absent taint declaration remains distinct from clean.
Typed peer lifecycle notices
Some peer lifecycle notices are informational and should not trigger an LLM turn. Mob lifecycle routing is typed at peer ingress (PeerLifecycleKind) instead of relying on silent_comms_intents folklore:
mob.peer_addedmob.peer_retiredmob.peer_unwiredmob.dismiss(supervisor-directed dismissal of a live executor)
mob.kickoff_failedmob.kickoff_cancelled
max_inline_peer_notifications:
None(default): use runtime default (50)0: never inline peer lifecycle updates-1: always inline>0: inline only when current peer count is less than or equal to the threshold< -1: invalid (rejected by mob definition validation; factory builds also reject)
peers() on demand to inspect the current roster.
External event ingestion
External systems can push events into a running agent without Ed25519 authentication. Surface-level convenience routes now admit those events through runtime-ownedExternalEvent inputs instead of a separate injector-owned execution path.
Event sources
CLI stdin events
Use--stdin lines for newline-delimited events; it implies keep-alive on the
normal CLI path. --stdin auto (the default) and --stdin blob instead read
finite piped input as prompt context, while --stdin off ignores stdin.
Line events default to --line-format text, which preserves each line
literally, including JSON-looking text. Opt into --line-format json to retain
the whole structured JSON value (falling back to text for non-JSON input).
It never extracts a guessed body property: the object in this example stays
an object rather than becoming just "deployment failed on prod".
REST webhook
Push events to a running session via HTTP. Auth is optional viaRKAT_WEBHOOK_SECRET env var with constant-time comparison.
202 Accepted with {"queued": true}. Runtime admission failures are returned as ordinary REST errors.
RPC session/external_event
Queue a runtime-backed external event for a running session via JSON-RPC.kind (generic_json), event_type, payload, and optional blocks. The method returns a runtime acceptance envelope on success.
TCP/UDS plain event listeners
Whenauth = "none" in config, a separate plain-text listener starts on event_address for unauthenticated external events. The signed agent-to-agent listener is never replaced.
Event transcript format
External events are persisted as typedexternal_event system-notice blocks with source, event_type, optional body, payload, and content blocks. The provider-facing source/body text is an internal projection and is not stored as user-authored transcript text.
- Stdin:
source = "stdin" - Webhook:
source = "webhook" - RPC:
source = "rpc"with optional structured source metadata. - TCP/UDS:
source = "tcp"orsource = "uds"
Delivery vs observation
When injecting events from application code, delivery and observation are separate concerns. Delivery — the service’sevent_injector() returns an Arc<dyn EventInjector>. Use inject() to push an external event into the session inbox:
- session-scoped: use the session’s primary
event_txorsession/stream_open(RPC) /Session.subscribe()(Web SDK) /session.subscribe_events()(Python SDK) /session.subscribeEvents()(TypeScript SDK) - agent-scoped: use
MobHandle::subscribe_agent_events(...)ormob/stream_open(RPC) /Member.subscribe()orMob.subscribeMemberEvents(agentIdentity)(Web SDK) /mob.subscribe_member_events(agent_identity)(Python SDK) /mob.subscribeMemberEvents(agentIdentity)(TypeScript SDK) - mob-scoped: use
MobHandle::subscribe_mob_events(...)ormob/stream_open(RPC) /Mob.subscribeEvents()(Web SDK) /mob.subscribe_events()(Python SDK) /mob.subscribeEvents()(TypeScript SDK)
- The agent must be running in keep-alive mode if you expect inbox events to be drained promptly.
- Comms must be enabled. Without comms,
event_injector()returnsNone. - Observation subscriptions are independent of delivery — configure them separately.
Security
- All messages are signed with Ed25519 using canonical CBOR encoding
- Trust is explicit: only messages from peers in the live, machine-authorized
TrustStoreare accepted; the persisted JSON projection is not startup authority - Misaddressed messages are dropped: the receiver verifies
envelope.tomatches its own public key - Private keys are stored with restrictive permissions:
identity.keyis written with mode0600on Unix - Secret bytes are zeroized:
Keypair::from_secret()zeroizes the input after copying - ACK validation: ACK signatures, sender, recipient, and
in_reply_toID are all verified - Exact replay handling: durable runtime ingress deduplicates the same interaction identity instead of starting a second turn
See also
- Built-in tools reference - comms tool parameter details
- Configuration: comms - config file settings
- Examples: comms - keep-alive mode and messaging across surfaces
