Multi-Host Mobs — Architecture & Implementation Plan (v4.1)
Implementation status: implemented. The complete phase 1-8 plus 6b implementation merged in PR #867 atfd1e275030c76af059eb7a2ab0a2f6a555617563on 2026-07-16. Restart and rejected-run recovery follow-up #888 landed at32510368852738e703e31b51b2a9192a96d7d205on 2026-07-17. This file remains the historical adjudication and implementation record; current code plus the Mobs guide and Mob Architecture reference are authoritative for the shipping public surface. Post-plan amendment: PR #976 atcbaca57cc38cc71656f1163bff1e1b7492cf3373added optionalresume_from_roleto privateMaterializeLaunchMode::Resume. It authorizes one exact cold durable role migration, fails closed when absent or wrong, and is not a public console, agent-tool, profile, or SDK spawn field. The v4.1 text below intentionally preserves the launch contract as originally designed; this amendment is the current delta. Historical design status: adjudicated v4.1 (2026-07-07, at HEAD 0.7.22 / da467fca8; v4.1 ecosystem expansion same day). Every substrate claim was re-verified against that HEAD (file:line anchors are from it). Structure: §§1-13 are the core plan; §§14-19 are the ecosystem chapters (realms/config, resource resolution, live channels, surfaces, orchestration-adjacent systems, session lifecycle), each independently substrate-mapped and adversarially red-teamed; §20 is the threat model; §21 is operations/packaging/governance. §4.1 is the cross-chapter adjudication ledger - every conflict between chapters was decided there, and the chapter texts already reflect those decisions. Historical v4.1 statement: this document was the merged, authoritative plan. It superseded the v1 sketch (2026-06-03), the v2.1 adversarial adjudication (2026-06-15), and the pasted “RMAT-aware extension plan” draft; the v3 “leaderless peer fabric” concept remains a separate track (see Adjudication history). Companion doctrine:docs/architecture/meerkat-dogma.md. Machine registry:canonical_machine_schemas()inmeerkat-machine-schema/src/catalog/mod.rs.
1. Goal
Distributed mobs whose members run on multiple hosts with the same UX as local:- Agents communicate by identity (
AgentIdentity), never knowing whether a peer is local or remote. - Users observe and control every member — including remote and remotely-spawned members — through the same console/API surfaces: list, status, history, live events, send, cancel, retire, wire.
- Mob membership/topology stays
MobMachineauthority. Comms delivery staysmeerkat-commsauthority. Session execution/history stays runtime/session authority. MobKit stays a projection layer.
send_message(B1) works identically from A (cross-host) and from B2 (same host). The console, attached to Host A, sees and controls all eight identities.
2. Adjudication history (do not re-litigate silently)
3. Constraints (v1, corrected wording)
- Realms are not distributed. A member’s sessions/events/credentials persist in the member host’s local realm. Cross-host reads are bridge-served projections, never remote store handles.
- One process hosts many members; one TCP ingress port per host process (not per agent, not per member).
- No new authority owners. No parallel roster, no second topology counter, no host-local fence rotation, no comms-owned route authority. Every new semantic fact folds into an existing machine or a catalog-generated scoped authority.
- Protocols are separated; transport is shared. Peer data (
MessageKind) and supervisor control (BridgeCommandover thesupervisor.bridgeintent) already share the comms envelope, listener, and inbox — that stays. Separation is at the protocol/authority level: typed payloads, distinct admission authorities, one ingress. (The draft’s “do not merge the planes” wording is retired; read literally it contradicts shipped reality —meerkat-runtime/src/comms_drain.rs:1687dispatches bridge commands out of the ordinary drain.) - The peer envelope does not change in v1 (see D5-adjacent rationale in §7.1).
- MobKit consumes Meerkat surfaces; it grows no roster, directory, or authority.
4. Decisions
D1 — Data plane: one host acceptor, demux onenvelope.to.
A host process runs one TcpTransportListener; inbound envelopes are routed to the addressed member by the existing to: PubKey field. Evidence that this is small: there is no connection-level identity handshake — the only identity coupling in the receive path is the per-envelope gate envelope.to != keypair.public_key() (meerkat-comms/src/io_task.rs:70) and the single inbox_sender parameter of handle_connection (io_task.rs:34-56). The acceptor replaces that pair with a registry lookup PubKey → (member keypair for acks, member inbox). No new frame, no codec change, no envelope version. The v2.1 deferral priced the HostTransportFrame design (two-layer envelope, second routing owner); the demux design has neither. The registry is a host-local projection installed and removed only by machine effects (materialize/retire), never self-populated. require_peer_auth is unconditionally on for the acceptor — cross-host ingress without signature+trust verification is unrepresentable. The existing per-listener first-byte { sniff that branches into the JSON pairing handshake (comms_runtime.rs:3195-3238) is kept, but pairing binds the host identity (§7.2), not a member. Standalone external members (rkat run --comms-listen-tcp) remain supported as the degenerate one-identity acceptor. Outbound is unchanged (router dials per send; connection reuse is a v2 optimization, noted in §13).
D2 — Observation: proxy-and-merge through the controlling host.
The console has one endpoint (the controlling host’s RPC/REST). The controlling host reads remote members’ history/events over the supervisor bridge and merges them into the existing mob stream. Controlling→member-host connectivity exists by construction (the bridge already commands remote members). Direct observer→member-host streams are a v2 optimization. The v3 fabric answers this differently for its own product; that fork stays separate.
D3 — Two auth planes, split; neither lands in AuthMachine.
AuthMachine’s charter is per-binding provider-credential lease lifecycle (catalog/dsl/auth_machine.rs:1-26; meerkat-runtime/src/handles/auth_lease.rs:1-16 records the prior deliberate rejection of cross-domain absorption). Mob control authorization is a foreign fact there. The repo itself names the missing layer: meerkat-rpc/src/secure_rpc.rs:3-4 — transport policy “does not decide who may perform RPC actions once connected; that belongs to auth/grants.” So:
- Plane (a), host↔mob: a host bind ceremony: an out-of-band one-time bootstrap token derives a domain-separated
BridgeHostBootstrapProofbound to the exact supervisor/host/mob/epoch/generation tuple → durable host authority record fenced by the supervisor authority epoch. The raw bearer never crosses the signed-but-unencrypted bridge. Possession-proof + Ed25519 identity; all-or-nothing per host. Comms bootstrap tokens are never principal auth. - Plane (b), principal→mob: typed
ControlScopegrants owned by MobMachine (grant lifecycle = machine facts) and enforced through a sealed resolved policy at the controlling host’s command dispatch — theToolExecutionPolicy/ExecutionPolicyGatedDispatcherpattern (meerkat-core/src/tool_execution_policy.rs), including its fail-closedUnresolvedInheritdiscipline. Bridge commands are only ever sent by the already-authorized supervisor, so plane (b) enforcement lives entirely on the controlling host (MobCommand admission + operator surfaces), not on member hosts.
topology_epoch keeps advancing at membership/wiring commit, exactly as shipped (mob_machine.rs:3354, and in WireMembersRunning at :5783-5794). Cross-host install convergence is tracked as machine obligations generalizing pending_recipient_trust: Set<PeerId> (mob_machine.rs:159, with the existing RecordPendingRecipientTrust/ResolvePendingRecipientTrust/RollbackPendingRecipientTrust input trio at :789-791). Fail-closed is preserved by the receiver: an envelope from a peer whose trust entry is not (or no longer) installed is classified untrusted and rejected — delivery gating already works this way, per-member, with zero envelope support. “RouteInstallIncomplete” is a typed projection of outstanding obligations, not a new owner. Retry probes durably recorded state before trusting it — the pending-rotation precedent (SupervisorPendingRotationRecord.accepted_peer_ids, meerkat-mob/src/store/mod.rs:165-178).
D5 — Event/history protocol: durable-cursor pages served from the owning host’s EventStore; long-poll is the single primitive.
Live session streams are tail-only broadcasts with no cursor resume (three unrelated sequence domains exist today: session-task counter, per-RPC-stream counter, durable store seq), and session/history is a snapshot read. The durable substrate already exists on the owning host: EventStore preserves full envelope identity with a store-assigned, fsync-gated seq and read_from(session_id, from_seq) (meerkat-session/src/event_store.rs:120-186). Therefore: remote reads are pages — ReadMemberHistory (transcript pages) and PollMemberEvents (event pages with optional bounded long-poll wait, the CompletionFeed { watermark, list_since, wait_for_advance } contract shape). There are no stream-open/close commands in v1: “streaming” is the controlling host running a poll pump per remote member; one canonical path, no push/poll divergence. Full replay parity requires persistent (event-store-backed) sessions on member hosts; ephemeral member hosts degrade to a bounded live buffer with typed StaleCursor on overrun — declared as a typed capability at bind, never a silent difference.
D6 — The member host is a composition role, not a new runtime type.
MeerkatMachine already is the per-process multi-session runtime (one instance hosts all local mob member sessions; prepare_bindings / prepare_local_session_bindings, meerkat-runtime/src/meerkat_machine/runtime_control.rs:1653-1683). The member host = MeerkatMachine + host comms acceptor (D1) + host bridge responder (§7.2-7.3) + registration client, packaged as a daemon subcommand rkat mob host. The background-driver template is spawn_schedule_host (meerkat/src/surface/schedule_host.rs:943-983). RuntimeHostInfo remains the wire projection of the process — and MUST stay anti-authority: runtime_host_info_does_not_claim_topology_authority (meerkat-contracts/src/wire/host.rs:181-210) pins that no authority vocabulary lands on it. Do not mint MobHostRuntime/AgentHost types.
Chapter-level decisions extend D1-D6: R1-R8 (realms/config/credentials, §14), resources R1-R8 (build portability, §15 — numbered independently), DL1-DL10 (live channels, §16), SD-1..SD-8 (surfaces, §17), O1-O8 (orchestration-adjacent, §18), L1-L7 (session lifecycle, §19).
4.1 Cross-chapter adjudication ledger (v4.1)
Every conflict the chapters’ independent drafts produced, decided once. The chapter texts already incorporate these; this ledger is the audit trail.- A1 — Spec compile locus: the materialize spec ships STRUCTURED RESOLVED INPUTS (resolved profile + definition extract + overlay) and the member host re-runs the ONE
build_agent_configcompiler — a member-side “realizer” would be a second parallel build path. All divergence-prone inputs are pre-resolved controlling-side:Pathskills →Inline, prompt tri-state neverInherit, model AND provider pinned typed, tool policy neverInherit; the member-host factory appends NO host-config-derived prompt sections for materialized members. §14.2/§15.3. - A2 —
shell_envnever travels: typedSecretBearingFieldadmission reject; host env is declared by NAME (required_env_keys) and verified at preflight. The payload’s non-confidentiality posture depends on this. - A3-final —
tools.scheduleIS permitted on remote members (reverses the draft reject): session-target schedules are realm-local and fire correctly on the owning host; member-side mob-target schedules fail TYPED at fire. Only workgraph stays admission-denied (its failure shape is silent split-brain with no typed backstop). §18.5. - A4 — One non-portability vocabulary, one enforcement point:
NonPortableResource { kind }minted on the machine spawn-exec ladder at theBeginSpawnExecchoke-point (the tool-seam-only admission is bypassed by RPC/embedder paths today). Inherited-OPEN workgraph resolves to disabled-with-typed-record; EXPLICIT assertion rejects. - A5 — Secrets-on-bridge invariant, refined: durable credentials/leases/provider secrets NEVER appear on new multi-host bridge payloads (type-level).
BindHostcarries a tuple-bound HMAC, never its descriptor bearer. The sanctioned multi-host exception is single-use, 60s-TTL, channel-pinned live WS tokens (§16), acknowledged as plaintext-exposed in §20. The olderBindMember.bootstrap_tokenprotocol still sends its bearer directly and is an explicit pre-existing residual, not precedent for new commands; v2 seed = encrypted comms transport (or a separately versioned member-bind proof). - A6 — Launch mode has one wire owner:
MaterializeMember.launch: {Fresh | Resume{session_id}};PortableMemberSpecstructurally excludes the domainlaunch_mode;Forknever crosses as a mode (it collapses to rendered text controlling-side, §19.L2). - A7 — One durability flag:
HostCapabilityFlags.durable_sessions(the host’s realm backend is persistent/event-store-backed). Event replay, archive capability, and console lifecycle flags all derive from it. - A8 — One member-originated command family:
MemberOperatorRequest(member host → controlling host oversupervisor.bridge, request-id idempotent under the signed requester generation+fence tuple), machine-admitted viaResolveMemberOperatorAdmission. It carries member-side operator tool calls AND member-originated spawn requests (§15 R6, §18 O3) — no second upcall channel exists. The controlling host durably recordsPendingbefore execution and an immutable terminal reply afterward (§15.5). - A9 —
ControlScopev1 = ten variants:List, ReadHistory, SubscribeEvents, SendCommand, Cancel, Retire, WireTopology, Live, AdminHost, AdminGrants.AdminHoststays exactly bind/revoke hosts; grant administration isAdminGrants.RewriteTranscript/Approvearrive with their features (both v2). - A10 — Portable MCP declarations: secret-bearing values (stdio
env, httpheaders) are structurally absent from the wire type — required env-key/header NAMES only, satisfied host-locally; inexpressible profile MCP config is a typedSecretBearingFieldreject. - A11 — Preflight adjudicator: shell observes,
MobHostBindingAuthorityadjudicates (ResolveMaterializePreflightfrom typed observations). One check inventory: model+provider pin, binding resolvable, required env keys present, stdio commands discoverable, memory backend,durable_sessionswhen required, engine/protocol range. NoPreflightMemberverb — materialize-integral. - A12 — Digest: SHA-256 over RFC 8949 canonical CBOR of
PortableMemberSpec(the envelope-signature canonicalization mechanics — stable across struct evolution; serde_json is not canonical and is not used). Machine-authorized atAuthorizeSpawnProfile; a SIBLING of the spec on the commands (never a field of the bytes it digests); host recomputes (SpecDigestMismatch), ack echoes,CommitSpawnMembershipguards echo == authorized. - A13 — Naming: the console projection family is
mob/hosts; the bridge verb staysHostStatus. - A14 — Multi-tenancy:
MobHostBindingAuthorityis MOB-KEYED (Map<MobId, SupervisorBinding>,Map<(MobId, AgentIdentity), …>); one daemon serves many mobs/controlling hosts; same-mob second supervisor = typedAlreadyBounduntilRevokeHost. - A15 — ErrorCode registry: exactly four new
ErrorCodevariants —ScopeDenied,HostUnavailable,StaleCursor,StaleFence(allocations §17.4) — with all four projections + roundtrip tests + schema/SDK regen in phase 1. Every other new cause stays bridge/domain-level and maps onto existing codes — includingNonPortableResource, which is spawn-admission machine vocabulary (MobSpawnMemberAdmissionKind) surfacing through an existing validation code, whileStaleFenceis a live bridge cause (materialize/release/command paths) that needs a stable console rendering. (v4.1.1 errata: this line originally readNonPortableResourcein place ofStaleFence, contradicting §17.4’s worked allocation table; the table was implemented and this line corrected to match — phase-1 implementation review.) CLI mob-verb typed-exit path and MCPMcpToolErrortyped mapping are NEW seams (phase 7), not existing ones. - A16 — Principal identity (plane b): v1 local RPC/REST/stdio surfaces authenticate as the owner (implicit full scope, unchanged — MobKit keeps working); the second-principal class in v1 is remote comms peers. A member upcall is signed by its member peer key for transport admission, but executes in the separate generated agent-authority lane and never acquires principal grants (§15.2). Bearer-token →
PrincipalIdauth forrkat-rpc --tcpis the named v2 seed (§21.7). - A17 — Flow outcomes decoupled from observation: dispatching a tracked remote turn records a MobMachine obligation (
pending_remote_turn_outcomes); the poll pump runs while subscriptions OR obligations exist.terminal_seqis pinned to the durableStoredEvent.seqdomain;durable_sessions=falsehosts areRejectedHostIncapableat dispatch classification. - A18 — No shadow inputs to machine classification:
ClassifyFlowStepDispatchreadsself.member_runtime_modes(machine state), never a shell-supplied mode. - A19 — Disposal order:
ReleaseMember= outer quiesce (cancel → runtime retire → drain) THEN the commit-first archive protocol; the retire AND destroyObserve*RetirementArchivedsignals both gain the typeddisposalfield in one change-set. §19.L3/L4. - A20 — Revival source: host-autonomous from the durable spec row at the STORED
(generation, fence)(partition-resilient);HostReboundreconciles stale fences viaReleaseMember; new specs only ever arrive machine-issued at a higher fence. §14.6/§15.7. - A21 — Live ordering + scope carve-out: phase 6b lands AFTER phase 6 (pump + single-flight relaxation are prerequisites); §12’s direct-observer non-goal explicitly carves out the live MEDIA plane as the one declared D2 exception (§16 DL1).
- A22 — Fence containment honesty: nothing validates fence/epoch member-side on delivery in v1, and host-local realm writes sit outside the fence path entirely — stated plainly here and in §20, never laundered.
5. Authority model
One row per semantic fact. “Projection” means rebuildable, never consulted for semantic decisions.
Phantom-authority note:
PeerDirectoryReachabilityAuthority (named as owner in the prior draft) does not exist — zero hits at HEAD. Reachability is new work, and it is a projection.
6. Machine deltas (catalog first — Rule: no production-only semantics)
All DSL changes land inmeerkat-machine-schema/src/catalog/dsl/ first, then make machine-codegen / machine-check-drift / machine-verify, schema+alphabet parity, and effect dispositions for seam-inventory. TLC bounds for the new state must include ≥2 hosts × 3 members.
6.1 MobMachine — hosts and placement
New state (field-driven, per the DSL design principle — no phase enums):HostId is a sealed newtype over the host’s comms PeerId (identity-first; no second id space; sealed ctor validates pubkey derivation). It is distinct from RuntimeHostInfo.host_id: String, which stays a wire projection.
New inputs: BeginHostBind, CommitHostBind { host_id, pubkey, endpoint, capabilities, epoch }, RevokeHost, HostRebound { host_id, epoch }, ResolveMemberMaterialization { agent_identity, outcome }. Spawn-path extension: ResolveSpawnMemberAdmission (mob_machine.rs:661-673) gains placement validation (host bound + capability present, typed reject otherwise); SpawnExecPhase (:10433, currently Opened | MembershipCommitted | Activated, absence = settled) gains MaterializePending entered only for remote placements: Opened → MaterializePending → MembershipCommitted → Activated. CommitSpawnMembership for a remote member carries the materialization-ack facts (real member pubkey, endpoint, session binding), so roster truth is never published before the owning host has confirmed.
New effects (each with an explicit disposition for seam-inventory, following mob_machine.rs:1459-1477): RequestHostBind (routed), RequestMemberMaterialization { agent_identity, generation, fence_token, spec_digest, host } (routed), RequestMemberRelease (routed), HostRegistered/HostRevoked (external seam), RouteInstallRequested { edge, host, a_endpoint, b_endpoint, epoch } (routed).
New invariants: member_placement ≠ ∅ ⇒ owner_bridge_session_id ≠ None (§19.L5 — the legacy ops-owner None-skip never applies to placed members); placement targets a bound host (member_placement values ⊆ mob_hosts with host_bind_phase = Bound); a remote member in MembershipCommitted+ has a member_peer_endpoints entry whose signing key matches the materialization ack; host_authority_epochs monotonic per host.
6.2 MobMachine — wiring install obligations (D4)
RecordRouteInstall, ResolveRouteInstall { obligation }, RollbackRouteInstall. The pending ledger is Install-only. The existing plural wiring inputs (WireMembers, WireMembersWithTrust, UnwireMembers, WireExternalPeer; the singular WireMember/UnwireMember names are bridge commands, not DSL inputs) stay the only topology mutators; topology_epoch semantics are unchanged. Edge usability is enforced by the receiver (uninstalled trust ⇒ untrusted ⇒ rejected), so a pending Install is fail-closed and the ledger makes convergence observable, retryable, and reportable. Unwire uses synchronous AuthorizeRouteRemovalBeforeUnwire: every surviving remote Remove lane must ACK while the exact edge and peer material still exist, then the durable unwire commits. Remove never enters the volatile pending ledger. Transport selection per edge is planner work in the wiring planning helper: both endpoints on one host use that host’s in-process route; cross-host uses the host acceptor address.
6.3 MobHostBindingAuthority — new catalog-generated scoped authority (host side)
The member host needs machine-adjudicated admission for host-addressed commands (bind, materialize, release, trust install, status), and durable memory for restart/dedup. It has no MobMachine (that lives on the controlling host) and MeerkatMachine authorities are per-session, so this is a process-scoped fact set. Precedent:session_persistence_version_authority — a catalog DSL source generated into a production crate, registered for schema parity, not a canonical machine. MobHostBindingAuthority owns:
NotBound | StaleSupervisor | SenderMismatch | InvalidBootstrapToken | StaleFence | Unsupported. Persistence: one SQLite table in the host’s realm (shape mirrors mob_runtime_supervisors), CAS-capable — this is what makes host restart rebind and materialize-retry dedup durable.
6.4 MeerkatMachine — member-side admission extension
Member-addressed bridge command admission is already machine-owned (ResolveSupervisorBridgeCommandAdmission → SupervisorBridgeCommandAdmissionResolved, realized at meerkat-runtime/src/comms_drain.rs:645-739). The new member-addressed commands (ReadMemberHistory, PollMemberEvents, HardCancelMember execution) extend that classification coverage — the typed command manifests (runtime_alphabet_parity) force this to be explicit. Today nothing validates fence/topology-epoch member-side on delivery (only supervisor identity+epoch); that stays true for delivery in v1 (supervisor-side StaleFenceToken on submit covers it), but the new host-addressed materialize/release path validates (generation, fence_token) in MobHostBindingAuthority — that check is new work, not reuse.
6.5 MobMachine — subscriptions and grants
- Event subscription: the adjudication seam exists with two outcomes —
AuthorizeAgentEventSubscription { agent_identity, session_id }/RejectAgentEventSubscription { reason: MemberNotFound | NoSessionBinding }(mob_machine.rs:1323-1324). Add the third outcomeAuthorizeExternalAgentEventSubscription { agent_identity, host }guarded onmember_placement/external_peer_edges. ExtendAuthorizeAllAgentEventSubscriptionto carry the external member set next tosession_bound_runtimes— today the mob-wide stream silently omits external members (AuthorizedMobEventRouterfilters to session-bound,meerkat-mob/src/runtime/event_router.rs:48, handle.rs:2582-2653); this plan kills that silent cap. - Grants:
operator_grants: Map<PrincipalId, GrantRecord { scopes, expires_at_ms }>withGrantOperatorScopes/RevokeOperatorScopesinputs. Expiry is data checked at the enforcement seam (shell reads the clock; the sealed policy compares — mirrors howToolExecutionPolicykeeps policy resolution out of the machine while lifecycle stays machine-owned).
7. Wire & protocol deltas (meerkat-contracts; one versioned protocol)
BridgeProtocolVersion gains V4; SUPPORTED = [V2, V3, V4]; every new command requires V4 (per-command version + fail-closed decode already exist: decode_bridge_command, supervisor_bridge.rs:318-330). BridgeCapabilities (already exchanged on bind, :769-816) advertises: durable_sessions, autonomous_members, hard_cancel_member (flips to true), tracked_input_cancel (separate from version support: the host must durably cancel an exact tracked input), and protocol range. The comms Envelope does not change: it signs (id, from, to, kind) (meerkat-comms/src/types.rs:157); topology epoch already rides where it belongs — inside control payloads (BridgeMobPeerOverlayHandoff.topology_epoch, contracts :939) — and delivery staleness is receiver-side trust gating. If the envelope ever must evolve, the mechanism is the content_taint precedent (absence-preserving optional field inside the signed region, fail-closed on old receivers; types.rs:54-61), not a version integer.
New commands (same BridgeCommand enum, deny_unknown_fields, #[non_exhaustive]):
Member-addressed (admitted by MeerkatMachine, as today):
- Placed plain
DeliverMemberInputseparates its canonical non-nil transport/idempotencyinput_idfrom optionaltranscript_interaction_id.IngressAcceptedalways mints a fresh retry-stable transport key and carries only a caller-supplied transcript id (Noneremains absent for ordinary runtime minting); it requests no terminal custody.TurnCompletedrecords controller custody before SubmitWork and uses the same canonical UUID for transport, transcript, waiter, and retained host sidecar, with explicitoutcome_tracking: Interaction. The receiver rejects non-canonical/nil ids, under-V4 use, a transcript carrier without exact placed residency, and any tracked split between the two ids. ReadMemberHistory { supervisor, epoch, protocol_version, from_index, limit }→BridgeReply::MemberHistoryPage(mirrorsSessionHistoryPage; transcript snapshot pages). The encoded reply is byte-bounded below the transport ceiling: a multi-row page returns the maximal fitting prefix with exactnext_index, while one untransportable row rejects as typedHistoryRowTooLarge { index, encoded_bytes, max_bytes }rather than returning an empty loop or silently corrupting fork context.PollMemberEvents { supervisor, epoch, protocol_version, cursor: MemberEventCursor | Tail, max, outcome_acks: Vec<(generation, input_id)>, max_outcomes, wait_ms }→BridgeReply::MemberEventsPage { generation, events: Vec<EventEnvelope>, next_seq, watermark, turn_outcomes, outcomes_complete }. Event and outcome windows are independently count-bounded and the encoded reply is byte-bounded below the transport ceiling.outcome_acksnames only rows consumed from the prior page; the member host prunes those exact durable rows. An absent ack is a no-op with no tombstone, so a journal commit delayed behind the terminal event cannot be lost. Long-poll:wait_msbounded well under the bridge transport budget. Events are served fromEventStore.read_fromwhendurable_sessions, else from a bounded live buffer with typedStaleCursoron overrun.HardCancelMember { supervisor, epoch, protocol_version, operation_id, expected_run_id, reason }— variant exists on the wire today but every receiver rejects it and nothing sends it (provisioner.rs:3297-3312; comms_drain catch-all). v1 implements the receiver arm through the machine-admitted hard-cancel path and a production sender, gated by theCancelscope. Both identity fields are REQUIRED: the sender observes and pins the current runtimeRunId, mints oneOperationId, and every transient transport retry reuses the byte-identical tuple. Receiver semantics are level-triggered and exact-run-scoped: compareexpected_run_idwith the machine-owned current run under the same per-session mutation gate asInterruptCurrentRun; while it still matches, reassert the hard cancel; once it is unbound or lifecycle-terminal, ACK. A newer current run therefore proves only that the expected run is terminal and is never interrupted by a stale retry. ACK means exact-run terminal truth, not merely that one interrupt signal was delivered.
MobHostBindingAuthority):
BindHost { supervisor, epoch, protocol_version, expected_host_peer_id, expected_address, bootstrap_proof }→ capabilities reply.bootstrap_proofis a dedicated HMAC newtype; the descriptor’s rawbootstrap_tokenis not valid wire vocabulary here.MaterializeMember { spec: PortableMemberSpec (§15.3), spec_digest, generation, fence_token, launch: MaterializeLaunchMode { Fresh | Resume { session_id } } }→MemberMaterialized { member_pubkey, member_peer_id, advertised_address, session_id, spec_digest (echo), engine_version, launch_outcome, resolved_auth_binding }. Idempotent on(spec.agent_identity, generation, fence_token): replay returns the recorded result; a lower tuple is a typedStaleFencereject; same tuple + different digest isSpecDigestMismatch(A12). Member identity has exactly one carrier (spec.agent_identity), launch mode has exactly one carrier (A6), and budget has exactly one digest-covered carrier (spec.overlay.budget_limits, §18 O4).ReleaseMember { agent_identity, generation, fence_token }— FULL durable disposal on the owning host (outer quiesce, then commit-first archive; typedMemberReleased { disposal }reply — §19.L3); also the reconciliation verb for orphans.InstallPeerTrust { agent_identity, peer: TrustedPeerDescriptor }/RemovePeerTrust { ... }— realizeRouteInstallRequestedon the owning host through the member’s machine-gatedapply_trust_mutationseam (never direct TrustStore writes).HostStatus {}→ materialized-member inventory + health + refreshedBridgeCapabilities(including its singleengine_versioncarrier); feeds reconciliation, reachability projection, and the periodic poll that doubles as orphan-reconciliation driver (§21.2).
DeliverMemberInput gains the optional turn: BridgeTurnDirective (§18 O1). Exactly ONE member-ORIGINATED family exists: MemberOperatorRequest (§15 R6 / A8), machine-admitted controlling-side.
New rejection causes extend BridgeRejectionCause (typed only — the bridge-classifier gate forbids ResponseStatus reinterpretation; the gate is a hardcoded file allowlist, every new reply-consumer file must be added): StaleFence, StaleCursor, Unavailable, ScopeDenied { required, presented }, plus the chapter cause families (§14.4 build rejects, §15.6 preflight, §16.4 live, §18/§19 flow-and-launch causes). Console-facing ErrorCode gains exactly FOUR variants (A15): ScopeDenied, HostUnavailable, StaleCursor, StaleFence — allocations and rendering seams in §17.4.
Cursor type: MemberEventCursor { generation: Generation, seq: u64 } — seq is the owning host’s durable StoredEvent.seq for the session bound at that generation. A fresh-session respawn starts a new seq domain. A higher-generation same-session Resume is never an in-place live rebind: the host unregisters/quiesces the old runtime, closes the live service session, awaits the old projection task’s terminal drain witness, then records generation_start_seq = drained watermark + 1 and rebuilds from the persisted snapshot. The host clamps stale/current cursors to that floor, and every page carries the current generation, so even an old terminal emitted during cutover cannot replay as a new-generation fact. A drain fault/timeout occurs before authority commit: no replacement success is emitted and the old durable row remains replay-revivable.
Projections (contracts): mob/member_status gains placement, reachability fields (§7.5), lifecycle capability flags (§19.L7), and the typed non_portable_disabled record (A4); new mob/hosts (A13), mob/member_history (§17 SD-1), route-install status, and grant DTOs. Remote-served history/event projections carry typed provenance (HostClaimed vs ControllingHostVerified) so consoles and MobKit can label what only the owning host attests (§20). Blast radius gates: make regen-schemas, SDK codegen, verify-version-parity, verify-rpc-surface-alignment / verify-rest-surface-alignment, verify-sdk-wrapper-freshness; verify-sdk-event-inventory deliberately does NOT fire (no new AgentEvent types, §17 SD-5).
7.1 Why the envelope stays untouched (recorded so it is not re-proposed)
The draft proposed signingversion + target host + mob/topology epoch into the envelope. Rejected: (a) epoch-in-envelope couples the data plane to authority propagation — mid-rotation, honest senders with stale epochs get bounced; the property it buys (“stale topology rejects delivery”) already holds receiver-side via trust classification (unwire ⇒ trust removed ⇒ untrusted ⇒ rejected); (b) target host has no consumer — there is no relay in v1 and the TCP connection terminates at the host, to selects the mailbox; (c) a CommsProtocolVersion integer is not the house evolution style and nothing changes that needs it.
7.2 Host bind ceremony (plane a)
- Operator starts
rkat mob hoston Host B. The host loads/mints its host keypair (Keypair::load_or_generatein the host identity dir), binds the acceptor, and writes a host binding descriptor (the--comms-binding-outshape, host flavor:{kind: "host", address, ed25519 public_key, bootstrap_token}). - Operator (or automation) hands the descriptor to the controlling mob:
BeginHostBind→ routedRequestHostBind→ derive the exact request-bound HMAC from the descriptor token → bridgeBindHostwith the proof, expected identity, and expected address. The raw token remains out-of-band. - Host validates via
MobHostBindingAuthority(token, sender, address — thevalidate_bind_requestshape, comms_drain.rs:879-965), records the supervisor binding at the offered epoch, replies capabilities. CommitHostBindrecords{HostId, pubkey, endpoint, capabilities, epoch}in MobMachine. Supervisor rotation fan-out (already fail-closed with pending-rotation memory) extends its recipient set to bound hosts.
{-sniff JSON handshake) proves possession of the host secret and returns the descriptor containing the host bootstrap token; members are never paired individually — member trust flows through machine-authorized InstallPeerTrust. Because the comms transport is signed but plaintext, this token-bearing pairing branch is loopback-only; remote operators transfer the 0600 descriptor out-of-band or tunnel the loopback listener. The subsequent remote BindHost carries only the tuple-bound HMAC proof, so observing it does not disclose the bearer.
7.3 Placement and materialization
Placement input:SpawnMemberSpec.placement: Option<HostId>. Defaults: a member-initiated spawn (B2 spawns B21) defaults to the requesting member’s host; operator/owner spawns default to the controlling host. Admission rejects unbound hosts or missing capabilities (e.g. autonomous_members for an AutonomousHost-mode profile) with typed causes.
B2-spawns-B21 walkthrough: delegation tool → MobCommand::Spawn → ResolveSpawnMemberAdmission (placement=Host B validated) → BeginSpawnExec (Opened) → ladder enters MaterializePending, machine emits RequestMemberMaterialization → bridge MaterializeMember to Host B → Host B: prepare_bindings, session created via its session service, member comms runtime minted (its own keypair, its own inbox), identity registered on the host acceptor, MobHostBindingAuthority records (identity, generation, fence, session) → MemberMaterialized ack carries pubkey/address/session → ResolveMemberMaterialization → CommitSpawnMembership publishes roster + RegisterMemberPeer publishes the endpoint → CommitSpawnActivation → wiring rules fire (§6.2) → B21 visible and controllable in the console.
Key custody: the member’s private key is minted and stays on Host B; only the pubkey travels (the BackendPeer.peer_id-derives-from-real-key discipline, meerkat-mob/src/backend.rs:34-53, generalized). Budget: the spawn’s budget_limits ride overlay.budget_limits inside the digest-covered spec (§15.3); the member host applies them at session build and enforcement stays local to the member’s session (aggregate mob budget is a non-goal, §12). Budget-split vocabulary does not exist on the wire (§18 O4 deletion). (v4.1.2 errata, ADJ-1: MaterializeMember.budget_seed originally named here as a payload-level pass-through was DELETED in phase 3 — two wire carriers for one budget fact is the Rule-6 shape, and only the digest-covered overlay copy survives member-host revival; the single carrier is PortableSpawnOverlay.budget_limits — phase-3 implementation review.)
Runtime modes: host-materialized members support both AutonomousHost and TurnDriven (MobRuntimeMode, meerkat-mob/src/runtime_mode.rs:7-13) — the member host runs the loop. The forced-TurnDriven normalization (actor.rs:783-791) continues to apply only to legacy peer-only RuntimeBinding::External members. The empty ExternalBackend provisioner stub (provisioner.rs:2580) is deleted; MultiBackendProvisioner dispatches RuntimeBinding — Session (local), External (legacy peer-only, unchanged), new HostMaterialized { host }.
Failure/orphans: materialize timeout/failure → member_materialization_failures + the existing machine-authorized revival flow (observe → classify → realize; Broken terminal). Orphan reconciliation: on (re)bind, HostStatus reports the host’s materialized set; the controlling machine issues ReleaseMember for entries it does not recognize at a current fence — the whole-mob-resume external rebind path (commit “external_tcp: own external members by MobMachine owner bridge session for resume restore” + AuthorizeMemberPeerRebind) is the recovery precedent.
7.4 Events, history, completion, cancel — closing the observation gap
The verified blocker family (nothing carries a remote member’s session events or history across hosts today;BridgeCommand has no read/stream variant; the mob-wide merger is local-only; per-member subscribe hard-rejects peer-only members with machine-emitted NoSessionBinding; the delivery path discards the completion handle at comms_drain.rs:2701 and the supervisor degrades TurnCompleted to a dispatch-ack; HardCancelMember is reject-only):
- History: console
mob/member_history(new RPC/REST/MCP surface, byAgentIdentity) → controlling host: local member ⇒ existing session read; remote ⇒ReadMemberHistoryproxy. Same page shape either way. - Events: the controlling host runs a poll pump per remote member (schedule-driver-style loop) driving
PollMemberEventsfrom the last cursor, wrapping results asAttributedEvent { source, source_fence_token, role, envelope }(the existing merge item,meerkat-mob/src/event.rs:509-518) and feedingAuthorizedMobEventRouternext to the localSelectAll. Dual cursor: the source(generation, seq)is preserved on each item; the merged stream’s ingest order is the router’s own. A ring overrun or immutable event row larger than the bridge reply budget is projected to subscribers first asAgentEvent::StreamTruncatedwith a typedRemoteCursorOverrun/OversizedRemoteEventreason, then the durable cursor advances; loss is never log-only. Console UX is unchanged (mob/stream_eventpush from the controlling host). Pump lifecycle follows the third subscription outcome (§6.5) OR outstandingpending_remote_turn_outcomesobligations (A17) — flow completion never depends on a console watching. - Completion: falls out of events — terminal AgentEvents arrive through the pump;
wait_one/wait_all/flow-step completion for remote members consume the merged stream (thespawn_turn_completed_replydegradation path is upgraded to it).DeliverMemberInputkeeps replying at admission (unchanged; the reply deliberately carries no turn payload). - Hard cancel: receiver arm lands (machine-admitted), capability flips,
Cancelscope gates it at the controlling host. The controlling sender first observes the exact current run, then sends required{ operation_id, expected_run_id }identity. The receiver repeatedly applies the immediate interrupt authority only while that exact run remains current and withholds ACK until it is unbound/terminal. This is bounded convergence over an edge-triggered executor signal, not an edge-ack API. Timeout while the expected run remains bound is typedUnavailable; a transient resend is byte-identical, and if its prior ACK was lost after run A ended, replay ACKs A’s terminal truth without touching a newer run B.
MobSupervisorBridge currently serializes requests through a single-flight request_lock (supervisor_bridge.rs:41-48). Long-polls must not block lifecycle commands: correlation is already by envelope id, so v1 relaxes single-flight to per-request correlation (or a dedicated poller bridge instance per host) — named here because it looks optional and is not.
bridge-classifier gate mechanics: BRIDGE_CLASSIFIER_FILES (xtask/src/bridge_classifier.rs:27-31) is a hardcoded three-file allowlist. Every new file that consumes BridgeReply (the poll pump, the history proxy) MUST be added to it, or it silently escapes the no-ResponseStatus gate.
7.5 Reachability (projection)
Computed on the controlling host from typed outcomes it already sees: bridge request results (verified-ack vsPeerOffline/timeout — liveness today is exactly the per-send verified-ACK, DEFAULT_ACK_TIMEOUT_SECS = 30, router.rs:52) and pump progress. Surfaced per host and per member as control_reachability / comms_reachability: Reachable | Stale | Unreachable | Unknown (the wire enum exists at supervisor_bridge.rs:600) + last_seen + freshness_reason. Freshness is observer-local monotonic (receive-time + elapsed); never compare remote wall-clocks across hosts. Reachability never mutates membership. The existing ExternalMemberReachability projection (handle.rs:842) and SupervisorReachability transport classifier (mob supervisor_bridge.rs:58) are absorbed/fed, not duplicated.
8. Scoped control (plane b)
- Grants are MobMachine facts (§6.5), keyed by
PrincipalId(comms-principal = PeerId; local operator contexts injected — Gotcha #19: operator authority is injected, not ambient;MobToolAuthorityContext+ seal atmeerkat-core/src/service/mod.rs:491,511is the vocabulary to extend). - Enforcement: a sealed
ResolvedControlPolicychecked at the controlling host’s two chokepoints — MobCommand admission and the operator surfaces (RPC/REST/MCP mob handlers + agent-facing mob tools). Deny = typedScopeDenied { required, presented }. Fail-closed: an unresolvable principal has no scopes. - Defaults: the owning session (
owner_bridge_session_id) holds implicit full scope — single-user CLI behavior is unchanged. Every other principal is default-deny until granted. Scope semantics honored end-to-end:SendCommandwithoutReadHistorycan drive but not read;SubscribeEventswithoutCancelcan watch but not stop;WireTopologyis distinct fromSendCommand;AdminHost(bind/revoke hosts) is distinct from everything. - Principal identity (A16): v1 local RPC/REST/stdio connections authenticate as the OWNER (implicit full scope — today’s behavior, so MobKit and single-user CLI are unaffected by default-deny). Remote comms peers are the second principal class.
MemberOperatorRequestuses the member PeerId only as a signed transport/admission fact; its operator capabilities come exclusively from the controlling-host-minted generated agent authority, never a principal grant. Bearer-token →PrincipalIdauthentication forrkat-rpc --tcpis the named v2 seed (§21.7). - Grant durability:
operator_grantspersist through the mob store like every MobMachine fact; supervisor rotation does NOT clear grants (principal→mob facts, not epoch-scoped); expiry is data compared at the enforcement seam (a restored mob’s expired grants stay expired). - Ordering rule: no remote read/mutation surface ships before this lands (phase 5 precedes phase 6; phase 6b after 6).
9. Failure semantics (typed, enumerated)
The chapter failure tables extend this one: §14.4 (build rejects), §16.6 (live), §18.8 (flows/policy/schedules/approvals/taint), §19.F (launch/disposal/fork). Together they are the v1 failure surface; a new cross-host interaction without a row in one of these tables is a review reject.
10. Phases (exit gates, no calendar)
Each phase exits only with its gates green. Machine phases additionally run:make machine-codegen, make machine-check-drift, make machine-verify, runtime_schema_parity, runtime_alphabet_parity, xtask seam-inventory, xtask effect-authority, xtask ownership-ledger --check-drift, xtask rmat-audit --strict, machine-authority-docs-gate and verify-machine-poster-coverage (any canonical-machine alphabet delta trips the poster regen). Contracts phases additionally run: make regen-schemas, verify-schema-freshness, verify-version-parity, verify-sdk-codegen-freshness, verify-sdk-event-inventory, verify-rpc-surface-alignment, verify-rest-surface-alignment, verify-sdk-wrapper-freshness. Every phase runs docs-check; §21.5 is the docs/gates inventory. Chapter work items fold into these phases — each chapter carries its own phase list (§14.5-§14.6, §15.9, §16.7, §17.7, §18.11, §19.P); the lines below name only the core items.
- Adjudication + catalog deltas. This document is the phase-1 artifact. Land §6 in one catalog change-set (fields, inputs, effects+dispositions, invariants,
MobHostBindingAuthorityDSL, MeerkatMachine admission coverage) + §7 contracts (V4 commands, replies, causes, cursor, projections). TLC bounds: 2 hosts × 3 members. - Host bind + acceptor + host role. D1 demux in meerkat-comms (registry install/remove from machine effects only; mandatory peer auth; acks signed per-member; pairing branch host-scoped); §7.2 ceremony end-to-end;
rkat mob hostdaemon (D6); host binding descriptor. - Placement + materialization. §7.3 end-to-end including B2→B21/B22 on Host B;
HostMaterializedbinding variant;ExternalBackendstub deleted; idempotent retry; orphan reconciliation; revival integration. - Cross-host wiring install. §6.2 obligations; placement-aware transport selection;
wire_members_batchlocal-only rejection removed; unwire symmetry; fail-closed partial install with durable retry. - Control scopes. §8 grants + sealed policy at both chokepoints;
ScopeDenied; owner-implicit-full default. Must land before phase 6 ships any remote read. - Events / history / completion / cancel. §7.4: DSL third outcome + all-subscription external fan-out;
ReadMemberHistory+PollMemberEvents+ pumps + router merge with dual cursors; completion consumption (incl. the §18 O2 turn-outcome sidecar + A17 obligations); run-scoped level-triggeredHardCancelMemberreceiver + sender with required operation/run identity and ACK-after-terminal semantics; bridge single-flight relaxed; remote-source fork context (§19.L2);BRIDGE_CLASSIFIER_FILESextended. 6b. Remote live channels (§16; ordered AFTER phase 6 — the pump and the single-flight relaxation are prerequisites; theLiveOrchestrator+ projection-sink four-role extraction may run in parallel as pre-work). - Surfaces. RPC/REST/MCP/SDKs: by-identity
member_history, member event streams for remote members (existingmob/stream_eventnow covers them), host register/status, route-install status, grant management (rkat mob grant/revoke, host verbs). RPC stays the console↔controlling-host transport;rkat-rpc --tcpis never a host↔host control path (one control plane: the bridge). - MobKit projection. Remote/local members project into the existing console identity records; remote history/events consumed via the new Meerkat surfaces; console contracts stable; no MobKit-owned roster/directory (verified by review, not assumed).
(v4.1.2 errata, phase-8 consequence pass, verified against meerkat-mobkit HEAD pinning =0.7.25: (1) all three negatives HOLD — member listing always projects per-call from
MobHandle::{list_members_*, roster()}with no MobKit-owned copy; zero host-directory concepts (contact_directory.rsis an unrelated cross-mob gateway federation map, mob_id→transport); live channels delegate to the handle with no MobKit-side registry. (2) Console contracts are stable by construction:IdentityRuntime/ConsoleFramerecords key onAgentIdentity, not session id, so remote members project without shape change. (3) Meerkat is NOT missing a projection surface — the phase-7 library API carries everything MobKit needs (MobHandle::{member_history, hosts, route_installs, bind_host, revoke_host, hard_cancel_member, member_live_*, grant_scopes/grants},MobMemberStatusResult.placement,WireMobErrorDetail). (4) Recorded DOWNSTREAM asks (MobKit-repo work, out of this plan’s scope): bump the meerkat pin past 0.7.25; read remote transcripts via by-identitymember_historyinstead of realm-local session-id reads (the console backfill viaresolve_bridge_session_id_observation, the memory-hygienistSessionServiceRevisionSeam::read_messages, and theIdentityRuntimesession-continuity layer are all in this class); route remote member events/live deltas through the pump-backed surfaces instead of per-member localsubscribe_agent_events; usehard_cancel_memberfor remote force-cancel (both the RPC path and the HTTP-consolemobkit/force_cancel_member); surfaceplacement; adopt the typed mob-family codes (WireMobErrorDetail, -32025..-32028) on the mob verbs instead of-32000 + format!laundering — the separate session-lifecycle string-sniff (is_stopped_session_archive_retire_rejection) is a pre-existing meerkat-session workaround NOT covered by those codes and needs its own typed seam.)
11. Test matrix (lane-mapped; deterministic lanes are the CI ratchet)
Lane authority:tests/integration/src/e2e_lanes.rs. Known trap: e2e-smoke/e2e-system do not run in GitHub CI, and external-TCP real-TCP coverage has historically been an ignored smoke lane — so the deterministic lanes below are the regression gate, not the live ones.
unit/int/e2e-fast (GitHub CI) — two-hosts-in-one-process harness (two MeerkatMachines + two acceptors over loopback/in-memory transport):
- mixed A/B/C topology: identity-routed send local↔remote both directions; B1↔B2 same-host inproc selection
- many identities, one acceptor: demux by
to, per-member ack signing, misaddressed reject, unregistered-identity reject - mandatory peer auth on acceptor cannot be disabled; envelope byte-compat pin (no signed-region drift)
- host bind ceremony: token single-use, sender mismatch, address mismatch, capability capture, epoch record
- remote spawn ladder: B2→B21 full walkthrough; materialize retry deduplicates; stale-fence materialize rejected; orphan reconciliation releases at stale fence; revival classification on materialize failure;
Brokenrefuses retry - wiring: partial install leaves obligation + unusable edge fails closed; retry drains; unwire removes trust and subsequent delivery is rejected; batch wiring with mixed placements
- events: durable cursor resume across pump restart AND member respawn (fresh sessions reset; same-session Resume fences visibility at persisted
generation_start_seq); mob-wide stream includes remote members; completion for remote members via merged stream; long-poll does not block lifecycle commands;StaleCursoron ephemeral overrun - history: remote page read == local page shape; scope matrix — each
ControlScopegrants exactly its verbs,ScopeDeniedcarries required/presented; work-only cannot read history; read-only cannot cancel - hard cancel: wire decode requires
operation_id+expected_run_id; an edge signal missed before the executor arms is reasserted until the exact run unbinds; ACK is observed only after unbind/terminal; replay the same operation for run A while run B is current and prove B receives no interrupt; transient sender resend preserves both ids byte-for-byte - rotation: bound hosts in fan-out; pending-rotation retry with a host that rebound to current authority
- TLC (
make machine-verify) at 2×3 bounds green for all new state
e2e-system (local make ci / nightly) — real multi-process TCP: rkat mob host daemon lifecycle; host restart rebind with session recovery and fence advance; partition (kill link) → typed Unavailable, membership unchanged, obligations retained; controlling restart resume with host re-probe.
e2e-smoke (live) — kitchen-sink A/B/C with real providers.
The chapter test matrices extend these lanes (§14 pins, §15.10, §16.8, §17.11, §18.11, §19 tests), all lane-mapped under the same ratchet rule. CI note: the added integration volume will likely require re-sharding the mob int lanes in cargo.yml — budgeted in phase 7.
12. Non-goals (v1) and v2 seeds
Out of scope, stated so they cannot creep silently: relay/multi-hop envelope routing; dynamic host discovery (registration is explicit); cross-host aggregate budget enforcement (per-session enforcement + later projection only); distributed realms; direct observer→member-host streams for the OBSERVATION plane (history/events — the live-channel MEDIA plane is the one declared D2 exception, §16 DL1 / A21); leaderless-fabric semantics (separate track); outbound connection pooling (the router dials per send —router.rs:575-643; a known perf characteristic to measure in e2e-system, correctness-neutral). Chapter non-goals extend this list (§14.7, §15.8, §16.9, §17.10, §18.12, §19.N). v2 seeds, in likely order: encrypted comms transport (A5/§20), bearer-principal RPC auth (§21.7), connection reuse, direct observer streams, bridge-proxied workgraph, approval forwarding (Approve scope), remote transcript edits (RewriteTranscript scope), WebRTC signaling proxy, richer placement policy (capability/label matching — note RuntimeHostInfo.placement_labels has zero writers today and an anti-authority pin; any real capability matching feeds MobMachine facts, not that projection).
13. Implementer gotchas (verified traps)
BRIDGE_CLASSIFIER_FILESis a hardcoded allowlist — extend it for every new bridge-reply consumer (§7.4).RuntimeHostInfomust stay anti-authority (pinning test at wire/host.rs:181-210) — do not enrich it with host/placement authority facts.- DSL wiring inputs are plural; the singular names are bridge commands. Do not invent
WireMemberDSL inputs. - No member-side fence/epoch validation exists on delivery today — the host-side
StaleFencechecks in §6.3 are new work. ExternalBackend(provisioner.rs:2580) is an empty stub — delete, don’t extend.MobSupervisorBridgesingle-flightrequest_lockwill serialize long-polls against lifecycle commands unless relaxed (§7.4).- Trust rows are never seeded from config and persisted seeds are rejected at startup (comms_runtime.rs:1620-1632) — all trust flows through
apply_trust_mutationunder machine authority; the acceptor registry follows the same rule. - Three sequence domains exist (session-task seq, per-RPC-stream seq, durable store seq) — only the durable store seq is a cursor; never leak the others into the wire cursor.
- Re-fencing on re-acquire is intentional (fencing mechanism, not churn) — do not “fix” monotonic fence bumps on host rebind.
- Effects need typed dispositions (
seam-inventory) and commands need typed classification (runtime_alphabet_parity) — budget for these in every machine delta, they are CI failures, not review notes.
14. Realms, config, state locality, credentials (host-realm doctrine)
This section is the authoritative answer to “where does a remote member’s state live, which config governs its build, and how do its credentials resolve.” It builds on D1–D6 and changes none of them.
14.0 Decisions
R1 — A materialized member’s state lives in the member-host daemon’s realm;mob.{mob_id} is never a store path.
This is HEAD behavior generalized, not new mechanism: a member session is created through the hosting surface’s SessionService (meerkat-mob/src/runtime/provisioner.rs:1986-1989) and persists in that surface realm’s stores. The mob.{mob_id} realm id (single owner: mob_realm_id, meerkat-core/src/connection.rs:261-263, stamped at meerkat-mob/src/build.rs:198,207) rides only as: durable session-metadata stamp (meerkat/src/factory.rs:5470,5507), comms inproc namespace string (factory.rs:4560-4565 — “transport string, not a typed realm carrier”), workgraph store scope key (factory.rs:4907-4935, path under realm_scope_root = host runtime root, factory.rs:2742-2747), and credential preferred_realm (factory.rs:3983-3988). No realm directory is ever materialized for it — and sanitize_realm_id would path-mangle mob.x → mob_x anyway (meerkat-store/src/realm.rs:267-278), proof the dot form was never a directory contract. WHY not a dedicated per-mob realm dir on Host B (mapper F6): it would mint new realm manifests/leases/config heads per mob, silently change the effective-config head for member builds, and contradict D6’s no-new-realm-semantics posture. Cleanup on ReleaseMember uses archive semantics in the daemon realm, not directory deletion — typed, observable, not a scope cut.
Concretely, a member materialized on Host B occupies Host B’s daemon realm at realm_paths_in(state_root, realm) (meerkat-store/src/realm.rs:280-291): sessions.sqlite3 (session rows), durable EventStore at <projection_root>/.rkat/events (meerkat/src/persistence.rs:304-306), blobs/, artifacts/, workgraph.sqlite3, ops state (under the default Sqlite realm backend the SqliteRuntimeStore runtime_* tables live INSIDE sessions.sqlite3 — meerkat/src/persistence.rs:355-367; a standalone runtime.sqlite3 exists only under the JSONL backend, persistence.rs:315-317), memory at <store_path>/memory (factory.rs:5644), plus its per-process AuthMachine lease (LeaseKey, meerkat-core/src/handles.rs:1301-1327) published at build on Host B (factory.rs:4065-4078). The controlling host sees all of this only as bridge-served projections (D2/D5); there is no wire verb that returns a store handle, so cross-host store access is unrepresentable.
R2 — rkat mob host realm-head contract: workspace-derived default, explicit --realm override, --isolated rejected.
The daemon selects its realm exactly like rkat CLI: default RealmSelection::WorkspaceDerived { root: context_root } → ws-<fnv1a64(canonical root)> (meerkat-cli/src/main.rs:4016-4024; derivation meerkat-store/src/realm.rs:809-813), state root default <context-root>/.rkat/realms (main.rs:4064-4066), global doc at --user-config-root-else-~/.rkat/config.toml (main.rs:4087-4100). WHY not rkat-rpc’s Isolated default (meerkat-rpc/src/main.rs:195-199): Isolated generates realm-<uuidv7> per process (realm.rs:815-817) — a fresh throwaway realm every launch, which destroys the member-host restart contract (§10 row “Member host restart” requires sessions, events, and MobHostBindingAuthority to survive in the host’s realm-local stores). A durable daemon needs a restart-stable realm head; workspace derivation gives one with zero new vocabulary. rkat mob host --isolated is a typed CLI rejection (RealmSelection::Isolated unrepresentable for the daemon), not a warning. Lease posture: RealmLeaseGuard heartbeats block destructive prune only — including unparseable lease files (realm.rs:102-137) — so co-locating the daemon with an interactive rkat workspace realm is safe-by-construction; the documented operational recommendation is a dedicated context root per daemon so mob-host uptime doesn’t extend the workspace’s prune-blocked window.
R3 — Config authority for a remote build (mapper F1): the spec pins member-visible facts; everything else is the member host’s effective config. No config snapshot ever ships.
Every agent build resolves config as head-store-then-inheritance-fold: FactoryAgentBuilder::resolve_config (meerkat/src/service_factory.rs:737-782) composes effective_config_over_head(daemon_head_realm, head_config) (meerkat-core/src/config_store.rs:167-241) over the chain resolved by RealmChain::resolve (meerkat-core/src/connection.rs:743-803). The inheritance HEAD is the daemon’s realm, never the mob’s. WHY not ship a config snapshot in MaterializeMember: it duplicates config authority, names bindings/servers that don’t exist on Host B, risks smuggling credential-adjacent facts, and contradicts the host-realm-local doctrine (§3). WHY not host-config-only: the profile’s whole point is deterministic member identity; leaving model/tools/policy to per-host config is silent divergence. So the split is explicit and enumerated (§14.3/§14.4): profile-compiled facts travel in the wire spec and are identical on every host by construction; host-mechanics facts (stores, credentials, limits, hooks, retry) are DECLARED host-local. spec_digest = SHA-256 over the RFC 8949 canonical CBOR of PortableMemberSpec — the SAME canonicalization mechanics the envelope signature already uses (meerkat-comms/src/types.rs:157-200), giving a byte-domain that is stable across struct evolution and binary versions (A12; serde_json is NOT canonical and is not used). It is machine-authorized at AuthorizeSpawnProfile, carried as a sibling on RequestMemberMaterialization/MaterializeMember, recomputed by the member host over the received spec (mismatch ⇒ typed SpecDigestMismatch, never a best-effort build), echoed in MemberMaterialized, and CommitSpawnMembership guards echo == authorized (a lying or corrupted ack cannot commit membership).
R4 — The profile model is authoritative for mob member builds; the resolving host’s binding.default_model no longer rewrites it (mapper F2).
Today the resolved binding’s default_model overrides the requested model whenever resume_override_mask.model is false (meerkat/src/factory.rs:4024-4033), and mob profiles default that mask to false (meerkat-mob/src/profile.rs:271-281, empty resume_overrides; mask set at build.rs:312). Cross-host, that means the SAME member spec yields different models purely from each host’s local [realm.*] config — a silent divergence this plan’s discipline forbids. Mechanism: a new typed field AgentBuildConfig.model_selection: ModelSelection { BindingMayOverride /* default */ | PinnedByProfile }; build_agent_config sets PinnedByProfile; the override site (factory.rs:4024-4033) consults it. WHY a new enum and not resume_override_mask.model = true: the mask is resume semantics (“profile field wins over durable session metadata on resume”, profile.rs:193-200) — reusing it would conflate two facts in one owner and change resume behavior as a side effect. This applies uniformly (controlling host and member hosts, i.e. also to today’s local mobs) — a deliberate, documented behavior change; standalone (non-mob) sessions keep BindingMayOverride semantics unchanged.
R5 — System prompts assemble on the controlling host; the rendered prompt travels (mapper F5).
assemble_system_prompt reads SkillSource::Path files from the local filesystem of whichever host runs it (meerkat-mob/src/build.rs:517-550, fs read at :533-538). Assembling on Host B would create an undeclared filesystem contract (identical skill files at identical paths on every member host) — a silent divergence vector. Instead the controlling host resolves inline + Path skills into the final prompt string and ships it in the spec; the SpawnSystemPromptOverride::Replace seam carries that initial event for fresh materialization (build.rs:209-217), while resume restores the persisted transcript exactly and never re-reads or re-injects prompt bytes. For materialized members the member-host factory appends NO host-config-derived prompt sections — the skill-engine inventory section (factory.rs:5116-5127, appended via extra_sections at :5258-5297) is disabled for mob-materialized sessions, so prompt bytes are exactly the spec’s bytes (A1; without this, host-level skill repositories silently diverge the prompt and the digest-⇒-identical-prompt test is false). Builtin preload skills (mob-communication / task-workflow / workgraph-workflow) are embedded per binary and derived deterministically from the traveling ToolConfig (build.rs:224-230) — the skill KEYS are implied, the bytes render from Host B’s binary. Acceptable skew is bounded by the bind-time protocol handshake (V4 required; BridgeCapabilities protocol range, meerkat-contracts/src/wire/supervisor_bridge.rs:772-816); byte-identical builtin skills across binary versions is a declared non-goal, made observable by the host binary version string in the HostStatus reply (projection).
R6 — Credentials never travel; binding references resolve on the member host (mapper F3).
End-to-end on Host B:
- Omitted
spec.auth_binding(the default;SpawnMemberSpec.auth_binding: Option<AuthBindingRef>is explicit-only, no ambient promotion —meerkat-mob/src/runtime/handle.rs:1895-1900): candidate walk withpreferred_realm = mob.{mob_id}(factory.rs:3983-3988→resolve_auth_binding_candidates_for_provider,connection.rs:1139-1176). Sincemob.{mob_id}is virtually never a configured[realm.*]section (zero producers at HEAD; consumers onlymeerkat-core/src/lib.rs,connection.rs,meerkat-mob/src/build.rs), the absent head contributes nothing (connection.rs:736-742), the implicit global tail applies (connection.rs:772-778), and resolution lands on Host B’s~/.rkatglobal-doc bindings or the synthetic env_default (allow_env_default = trueat both factory call sites,factory.rs:1744,1765; synthesisconnection.rs:1168-1170). Deliberate affordance, zero new machinery: an operator MAY define[realm."mob.<id>"]in a member host’s config to scope that host’s credentials per mob — the chain walk already honors it. - Explicit
spec.auth_binding: travels as the structuralAuthBindingRef { realm, binding, profile, origin }(no string form,connection.rs:216-235); an env_default-origin ref is rejected before any chain walk (connection.rs:1030-1034); resolution is the strict owner-stamped walk of the named realm’s chain withhead_required = true(connection.rs:1037-1044) on Host B’s config, thenProviderRuntimeRegistry.resolveenforces owning-realm equality (meerkat-llm-core/src/provider_runtime/registry.rs:208-213). The name travels; the material cannot: the token store is machine-local (dirs::config_dir()/meerkat/credentials,meerkat-auth-core/src/auth_store/mod.rs:83-89, keyed byTokenKey{realm,binding,profile},meerkat-core/src/auth/token_store.rs:24-29), env is Host B’s process env (ResolverEnvironment::with_process_env,factory.rs:3961), and the AuthMachine lease is minted on Host B’s per-process handle (factory.rs:4065-4078). Credential writes stay strict-owner and host-local (resolve_write_owner,connection.rs:1213-1240); there is no remoteauth login. - env_default provenance is allowed on member hosts (same posture as any rkat process) and recorded: the
MemberMaterializedack gainsresolved_auth_binding: Option<AuthBindingRef>whereNone⇔ synthetic env_default — exactly the discipline the factory already applies to the durable metadata stamp (build_config.auth_binding = Some(resolved)iff a lease-bearing configured binding, elseNone;factory.rs:4089-4093, stamped at :5474/:5511). Projection-only (feedsmob/member_status); never consumed for a later build. shell_envnever crosses hosts (A2): a remote spec carrying inline shell env is a typedSecretBearingFieldadmission reject — the payload’s non-confidentiality posture (threat model, §20) rests on the type-level absence of secret-bearing fields. Remote members’ shell subprocess environment comes from Host B (process env / host config), with required keys DECLARED by name in the spec (required_env_keys) and verified at the Tier-2 preflight. Local (controlling-host) members keep today’sshell_envsemantics unchanged (handle.rs:1881-1882).
provision_member’s create_session fails (provisioner.rs:1986-2001), and BridgeCapabilities has no availability vocabulary (supervisor_bridge.rs:772-816). v1 adds:
- Tier 1 (advisory, bind-time):
HostCapabilityFlagsgainsresolvable_providers: Set<Provider>— computed by a presence-level probe on the member host (per provider: default-chain candidate resolution over Host B’s effective config + credential-material presence read from env/token store; no network, no OAuth refresh). Captured atCommitHostBind, refreshed onHostReboundand on everyHostStatusreply via a newRefreshHostCapabilitiesinput.ResolveSpawnMemberAdmissionconsults it ONLY for specs that resolve credentials via the default chain (auth_binding = None) — the advertised set describes default-chain resolvability and says nothing about explicit bindings, which may resolve from realms the probe never walks. Miss ⇒ typed rejectProviderUnavailableOnHost { host, provider }; remedy is provisioning the host and re-probing (bind/status refresh). Staleness is declared: the flags are an admission hint, never the gate. - Tier 2 (authoritative, materialize-time): on Host B, after
MobHostBindingAuthorityadmission (fence + dedup) and BEFORE any session/state side effect, the host runs a typed build-preflight: model/provider identity against Host B’s registry (resolve_provider_from_registryincl. travelingcustom_models,factory.rs:3055-3128,3870-3879; self-hosted binding presence :3068-3093), connection-target resolution (explicit or candidates), and provider credential resolve. Any failure is a typedMaterializeBuildRejected— nothing half-builds: nocreate_session, no acceptor registration, no materialized-member record; dedup memory records only successes, so retry re-runs preflight. The preflight is shell-executed effect work whose typed outcome feeds the controlling machine (ResolveMemberMaterialization), so no host-side semantic state depends on it — authority discipline preserved.
MobHostBindingAuthority persists in the host realm’s SqliteRuntimeStore and is the ONLY mob-semantic durable state on a member host.
Concretely: a new runtime_mob_host_bindings table in SqliteRuntimeStore (joins the runtime_* family at meerkat-runtime/src/store/sqlite.rs:20-49 — inside sessions.sqlite3 under the default Sqlite realm backend, persistence.rs:355-367), keyed by mob: supervisor bindings Map<MobId, SupervisorBinding{peer, epoch, phase}>, materialized members Map<(MobId, AgentIdentity), {generation, fence_token, session_id}>, dedup memory per (mob, identity) — one daemon serves many mobs/controlling hosts (A14); a second BindHost for the SAME mob from a different supervisor is a typed AlreadyBound reject until RevokeHost. Record shape mirrors mob_runtime_supervisors (meerkat-mob/src/store/sqlite.rs:60; record_json + CAS). WHY runtime.sqlite3 and not a mobs/ db: the mobs/ directory layout (<realm-root>/mobs/<escaped_mob_id>.db + realm_profiles.db) is the CONTROLLING host’s mob-storage contract (meerkat-mob-mcp/src/lib.rs:307-313,405-431; RPC wiring meerkat-rpc/src/router.rs:944-947) — materializing it on member hosts would look exactly like the second roster the constraints forbid (§3). No mob definition, roster, profile, or wiring copy is ever persisted host-side; the record holds only supervisor binding (peer, epoch, phase), materialized members (identity → {generation, fence_token, session_id}), and dedup memory. Dedup hardening: the recorded tuple includes spec_digest, so a replayed MaterializeMember at a matching (identity, generation, fence) but different digest is a typed SpecDigestMismatch reject — never silently deduplicated to the old member. Controlling-host mob storage is unchanged; SpawnTooling::Profile / override_profile realm-scoped lookups resolve on the controlling host BEFORE the spec ships (handle.rs:1888-1894).
14.1 State locality map (one row per fact; all Host B paths under the daemon realm)
14.2 The portable spec — what travels, what cannot
MaterializeMember.spec is a new contracts type PortableMemberSpec (serde-clean, deny_unknown_fields), NOT AgentBuildConfig (which holds Arc dispatchers and live handles and structurally cannot ship). Compilation locus: the controlling host compiles profile+definition → PortableMemberSpec (including prompt render, R5); the member host compiles PortableMemberSpec → AgentBuildConfig via the build_agent_config path (build.rs:203-341) against ITS factory/effective config.
Travels (pinned facts): identity triplet (mob_id/profile-name/agent_identity — Host B re-derives comms_name and the mob.{mob_id} stamp through the same single owners, build.rs:176-198); model (+ ModelSelection::PinnedByProfile, R4), provider (RESOLVED and pinned controlling-side — never an Option the member host infers from ITS registry; closes the provider-inference divergence at factory.rs:3100-3113), self_hosted_server_id, provider_params, custom_models (definition.models, build.rs:298), output_schema (typed MeerkatSchema, ingress-validated, profile.rs:242-243), resolved tool_access_policy (Inherit is unrepresentable on the wire — encode fails closed, matching the factory’s discipline at build.rs:338-341), explicit ToolConfig category toggles (compiled to overrides on Host B, build.rs:238-248; an inherited tool-visibility authority is non-serializable and remote placement carrying one is rejected typed, A4), auto_compact_threshold, resume_overrides, max_inline_peer_notifications, image_generation_provider (profile∪definition pre-merged, build.rs:303-305), rendered system prompt (R5), declarative mcp_servers as PortableMcpDecl (secret-bearing values structurally excluded — required env-key/header NAMES only, satisfied from Host B’s own config/env; inline secret values are a typed SecretBearingField admission reject, A10; processes spawn ON Host B), labels + peer_description (peer_meta re-assembled deterministically on Host B, build.rs:186-196), additional_instructions, app_context, auth_binding: Option<AuthBindingRef>, budget seed (split computed controlling-side, §7.3), runtime_mode, continuity_intent, and declared host requirements (required_env_keys — names only). NOT fields of the spec: generation, fence_token, and spec_digest are SIBLINGS on MaterializeMember/RequestMemberMaterialization (a digest cannot be a field of the byte-range it digests); launch_mode is owned solely by MaterializeMember.launch: {Fresh | Resume{session_id}} (A6 — Fork is structurally unrepresentable on the wire; the domain SpawnMemberSpec.launch_mode never serializes into the spec); shell_env never travels (A2 — typed SecretBearingField reject; shell subprocess env is host-local, declared via required_env_keys).
Cannot travel (typed, with recomposer):
14.3 Config fact table — pinned vs declared-host-local
Resolved from Host B’s effective config (daemon head ⊕ parent chain ⊕~/.rkat global; service_factory.rs:737-782): budget_limits fallback when the spec seeds none (factory.rs:5517-5519), retry policy + call-timeout fallback (factory.rs:5521-5530), hooks, skills-engine roots, memory store, compaction runtime config (except the traveling profile threshold), model-registry base [models.*] ⊕ catalog (factory.rs:3870-3879), credential/token/env material (R6), MCP server process environment, all stores (§14.1), comms acceptor + keys. Divergence policy: pinned facts are identical everywhere by construction; host-local facts are per-host by DECLARATION (this table is the declaration); no member-visible identity fact (model, provider, prompt, tool policy, output schema) is host-local.
14.4 Typed failure taxonomy — “Host B cannot build the member”
Wire mapping:BridgeRejectionCause (supervisor_bridge.rs:440-465) gains MaterializeBuildRejected { cause: MemberBuildRejection } and SpecDigestMismatch, alongside the §7 additions. MemberBuildRejection is a new closed wire enum; AuthErrorKind is already wire-stable (meerkat-core/src/auth/error.rs:94-113); ConnectionTargetErrorKind is a new wire projection of ConnectionTargetError (connection.rs:647-688).
Controlling side: every cause lands in
member_materialization_failures (stable kind recorded, mirroring member_restore_failures) and flows through the existing machine-authorized revival classification (ClassifyBridgeRejectionRecovery owns recoverable-vs-fatal, supervisor_bridge.rs:467-472); Broken stays terminal. A failed materialize never mutates member_placement, and there is no silent fallback to controlling-host placement — re-placement is a new operator/machine decision, never an error-path default.
14.5 Bind ceremony additions (folds into §7.2)
Step 3 gains the Tier-1 probe: the host computesresolvable_providers and returns it inside the capabilities reply; step 4’s CommitHostBind records it. HostRebound and HostStatus refresh it (RefreshHostCapabilities). The host binary version has one carrier, BridgeCapabilities.engine_version; HostStatus has no sibling version field.
14.6 Member-host restart / resume (completes §10 row)
On restart, the daemon reopens the SAME workspace-derived realm (R2), rebinds via the durableruntime_mob_host_bindings record + HostRebound epoch bump, and rebuilds the acceptor registry from its recovered materialized set. The build input for member revival is the durable spec row (A20): the runtime_mob_host_bindings record stores the spec bytes + digest next to (generation, fence_token, session_id, generation_start_seq) (§15.7), so revival is HOST-AUTONOMOUS — the daemon re-runs the one build compiler on the stored spec at the STORED (generation, fence) with zero controlling-host contact (members keep working through a controlling-host outage; partition-resilient by construction). Authority is preserved by reconciliation, not by re-issue: fences advance only through the controlling machine, so on HostRebound the machine compares the host’s reported materialized set against its own facts — matching fences are confirmed, stale fences (the machine respawned/retired the identity while the host was down) get ReleaseMember. A NEW spec reaches a member only as a machine-issued MaterializeMember at a higher (generation, fence). Host-side, the revival realizes from ITS stores: build_resumed_agent_config runs on Host B (build.rs:346-387) — durable identity facts (realm_id, auth_binding, comms_name, mob_member_binding, provider/model) restore from session metadata (factory.rs:5470-5475; single-owner metadata rule), runtime mechanics (dispatchers, MCP, external tool composition) re-compose from the host role, the prompt restores via SystemPromptOverride::Inherit (build.rs:380-381), and credentials re-resolve on Host B exactly as at materialize (a host that lost its key since then fails typed at resume with the §14.4 taxonomy — surfaced through the revival path, never a half-alive member). Fences arrive re-issued from the machine (re-fencing is intentional, gotcha #9); a same-session Resume continues its durable seq domain but the persisted generation_start_seq fences prior rows from the new generation (§7).
14.7 Non-goals (v1) — each with typed degradation, never silent
- No distributed realms / no store replication: cross-host reads are bridge projections only; a remote store handle is unrepresentable (no wire verb exists).
- No credential distribution: no key, token, lease, or secret in any bridge payload (type-level:
PortableMemberSpechas no credential-material field); an unprovisioned host fails typedMemberBuildRejection::ProviderAuth { missing_secret }at materialize preflight. - No per-host binding alias map (mapper F3-C): one binding vocabulary; a name that doesn’t resolve on the placed host is
BindingUnresolvable, and the remedy is out-of-band provisioning of Host B’s config + token store. - No cross-host inherited tool-visibility handoff:
SpecNotPortable { inherited_tool_filter }typed admission reject;tool_access_policytravels. - No per-spawn in-process tool overlays for remote members:
SpecNotPortable { external_tools }typed admission reject. - No config snapshot shipping (mapper F1-B): host-local facts diverge by declaration (§14.3 table), pinned facts cannot diverge.
- No per-mob realm directories on member hosts (mapper F6-B): release = archive semantics in the daemon realm.
- No remote
auth login/ credential writes over the bridge: strict-owner writes stay host-local (connection.rs:1213-1240). - No byte-identical builtin skills across binary versions: skew bounded by V4-at-bind; host binary version observable in
HostStatus.
14.8 Implementer gotchas (verified, additive to §13)
resume_override_mask.modelis resume semantics — do NOT reuse it to pin the mob model; that’s whatModelSelectionis for (R4;factory.rs:4024-4033gains the second consult, nothing else changes there).- The materialize ack’s
resolved_auth_binding: None⇔ env_default mirrors the metadata stamp discipline atfactory.rs:4089-4093— do not invent a separate provenance enum. - The Tier-1 probe must never trigger OAuth refresh or provider network I/O — presence-level reads only, or bind latency and token churn become host-bind side effects.
sanitize_realm_idmangles dots (realm.rs:267-278) — never derive a path frommob.{mob_id}; it is not a location.- Dedup memory keys on
(identity, generation, fence, spec_digest)— omitting the digest silently replays a stale spec as “already materialized”.
15. Resource resolution: profile material, skills, MCP, hooks, memory, models
This section defines whatMaterializeMember.spec (§7.3) actually carries, what the member host must already hold, and what remote placement typed-rejects. It extends §6.1/§6.3 (machine deltas), §7 (wire deltas), §9 (failure rows), and §11 (tests). All anchors re-verified at HEAD (0.7.22 / da467fca8).
15.1 Verified substrate (what a member build consumes today)
- The controlling host builds every member in-process:
MobActor→build_agent_config(BuildAgentConfigParams)→to_create_session_request→ localSessionService(meerkat-mob/src/runtime/actor.rs:9204-9250;meerkat-mob/src/build.rs:143-344, 487-511). Two in-process overrides apply after compilation:default_llm_client(actor.rs:9229-9231) and per-memberauth_binding(actor.rs:9232-9235). - Everything the compiler reads is serializable except five in-process handle families:
profile.tools.rust_bundles(names ofArc<dyn AgentToolDispatcher>registrations —meerkat-mob/src/profile.rs:54-60, resolved atmeerkat-mob/src/runtime/tools.rs:252-259),SpawnMemberSpec.external_tools(“In-process only and not persisted”,meerkat-mob/src/runtime/handle.rs:1901-1902), the mob-widedefault_external_tools_provider(ExternalToolsProviderArc alias,meerkat-mob/src/lib.rs:186;meerkat-mob/src/runtime/builder.rs:54, 1984-1988), the actordefault_llm_clientoverride, and the surface-injectedMobToolsFactory/schedule/workgraph dispatchers (meerkat/src/service_factory.rs:854-874; Arc fields onAgentBuildConfig,meerkat/src/factory.rs:346-424, 468-500). - The compiler consumes definition-level material beyond the Profile:
definition.skillscontent for referenced names (build.rs:517-550—SkillSource::Pathis read from the building host’s filesystem, fail-closed at 530-538, wasm32-rejected at 540-543),definition.models(build.rs:298;meerkat-mob/src/definition.rs:598-605),definition.image_generation_provider(build.rs:303-305), anddefinition.profiles.keys()for default spawn-profile grants (build.rs:249-271). - The existing machine digest under-covers cross-host build identity:
skills_digesthashes skill NAMES only (meerkat-mob/src/runtime/spawn_profile_authority.rs:49) and nothing digests definition.skills content, definition.models, the spawn overlay, or the assembled prompt (meerkat-machine-schema/src/catalog/dsl/mob_machine.rs:187-198). “Same digest” does not imply “same agent” across hosts today. - Prompt gap (historical implementation note): a skill-less profile with no spawn override left
SystemPromptOverride::Inherit, which resolved through the building host’s prompt sources. The session contract does not persist or reconcile a singleton current prompt: eachSystemis an ordinary ordered transcript message. Rematerialization authors nothing. New System content uses the typed session or mob System-context admission API after resume. - Profile MCP servers (
McpServerConfig: stdio{command,args,env}/ http{url,headers},meerkat-core/src/mcp_config.rs:41-64) are materialized into a session-owned router at build (factory.rs:4667-4701; feature-absent builds fail closed at 4702-4710). A missing stdio binary does not fail the build: the failure arrives asynchronously as typedPendingFailed→McpLifecyclePhase::Failed(meerkat-mcp/src/router.rs:1488-1508, 1668-1745).wait_for_mcpdefaults false and the mob compiler never sets it (zerohook/wait_for_mcphits in build.rs — NOT FOUND). - Hooks are host config, not profile vocabulary:
hook_engine_overrideelse layered{active > context > user}.rkat/config.toml(factory.rs:5090-5114;meerkat/src/sdk.rs:80-129).Profilehas no hooks field (profile.rs:162-251— NOT FOUND). - Memory is host-realm-local and session-scoped:
HnswMemoryStore::open(store_path/"memory"),MemorySearchScope::for_session(session_id), fail-closedCapabilityUnavailablewhen enabled-but-unavailable including feature-absent builds (factory.rs:5636-5697). - Models: the catalog is compiled into the binary (
meerkat_models::canonical(),meerkat-models/src/catalog.rs:142-156), merged per build withcustom_models(factory.rs:3870-3879). Provider credentials resolve entirely host-locally (ResolverEnvironment::with_process_env+ TokenStore + realm binding candidates +ProviderRuntimeRegistry.resolve,factory.rs:3961-4011);AuthBindingRefis a structural name-ref carrying zero credential material (meerkat-core/src/connection.rs:216-235). TheRKAT_TEST_CLIENT=1env shim exists on whichever host runs the build (factory.rs:3922-3927). - Comms envelopes are Ed25519-signed, not encrypted: no tls/noise/encrypt/cipher/aes/x25519 primitive exists anywhere in
meerkat-comms/src(grep NOT FOUND). Anything placed in a materialize payload transits signed-plaintext TCP. - Both agent-facing mob tool surfaces are in-process on the controlling host:
MobOperatorToolDispatcherholdsMobHandle(tools.rs:341-346; twelve operator tools at :358-519; per-call spawn-scope re-check at :865-872) and mob-mcp’sAgentMobToolSurfaceholdsArc<MobMcpState>(meerkat-mob-mcp/src/agent_tools.rs:103, 417). No member→controlling-host command path exists over comms at HEAD — the prior draft’s claim that delegation “already travels via comms” is false; this subsection adds that lane (R6). - Per-turn
injected_contextalready crosses hosts end-to-end:BridgeDeliveryPayload.injected_context(meerkat-mob/src/runtime/provisioner.rs:3378) lowers as InjectedContext-role appends before the peer append (meerkat-runtime/src/comms_drain.rs:818-821). No new work; MobKit’smob/turn_start/mob/submit_worklane is placement-agnostic as shipped.
15.2 Decisions
R1 — The spec is fully-resolved, self-contained build material plus a declared member-host dependency set. No definition distribution.MaterializeMember.spec carries everything build_agent_config reads, resolved by the controlling host: the effective Profile (realm-refs, override_profile, and inherited-filter category opening already resolved), a definition extract (models, image_generation_provider, referenced skills with Path pre-resolved to Inline), and the spawn overlay. WHY: the traveling inputs are serde-clean once the two non-serde authority objects are excluded — InheritedToolVisibilityAuthority derives no serde and has a pub(crate) ctor (meerkat-core/src/session.rs:1586-1598), so a remote spawn carrying inherited_tool_filter: Some is a typed SpecNotPortable admission reject (§14.2), and MobToolAuthorityContext travels only as a sealed wire PROJECTION for member-side visibility composition (dispatch authority re-resolved controlling-side, R6) — Path skills are already read at build time on the building host (build.rs:530-538), and a digest is only meaningful cross-host if it covers the shipped resolved bytes. A reference-shaped spec would require a definition-distribution mechanism this plan does not have and would inherit the proven digest under-coverage. Host-local-by-nature resources (credentials, stdio binaries, realm backend) are NOT shipped — they are declared dependencies checked by R4 preflight. The member host runs the same build_agent_config compiler on the shipped inputs: one compiler, one owner, no serialized shadow of AgentBuildConfig (which cannot serialize — factory.rs:295-547).
R2 — A new machine-owned resolved-spec digest pins build identity; the ack must echo it.
SHA-256 over the RFC 8949 canonical CBOR of PortableMemberSpec (the envelope-signature canonicalization mechanics, meerkat-comms/src/types.rs:157-200 — stable across struct evolution; serde_json is not canonical and is not used; A12), computed after overlay resolution, authorized as an additional fact on the existing AuthorizeSpawnProfile ladder and echoed exactly through SpawnProfileAuthorized (the require_authorized_effect exact-match discipline, spawn_profile_authority.rs:86-121). RequestMemberMaterialization.spec_digest (§6.1) is pinned to this digest; MemberMaterialized must echo it or CommitSpawnMembership refuses. WHY: the shipped-material digest is what makes the plan’s idempotency key (identity, generation, fence) sound — the existing spawn_profile_authority_material_digests treat materially different builds as identical (names-only skills digest, no definition/overlay coverage), so reusing it would make “replay returns the recorded result” unsafe.
R3 — SystemPromptOverride::Inherit is unrepresentable in a remote spec.
The spec’s system_prompt is always Set(...) or Disable. When the local path would leave Inherit (no spawn override, empty profile skills — build.rs:216), the controlling host assembles the base prompt from its own config/AGENTS.md chain and ships Set(base). When profile skills are non-empty, the member host recomputes Set(joined skill sections) byte-identically from the shipped Inline skills (same compiler, build.rs:209-215 + 517-550). For materialized members the member-host factory appends NO host-config-derived prompt sections: the skill-engine inventory section that enumerates the host’s config-level skill repositories (factory.rs:5116-5127, appended via extra_sections at :5258-5297) is disabled for mob-materialized sessions (A1) — without this, host skill-repo config silently diverges prompt bytes under an equal digest. What remains member-composed (builtin-skill sections, tool instructions) derives only from the traveling spec + binary, i.e. deterministic given (spec_digest, engine_version) (R8). WHY: without this, the same spec digest yields different prompts per host via the member host’s config/AGENTS.md — an invisible divergence that defeats the digest’s purpose. Member-host prompt influence becomes an explicit non-feature, not an accident of the Inherit fallback.
R4 — Member-host dependencies are validated by a materialize-time typed preflight, layered over coarse bind-time capability flags.
HostCapabilityFlags (§6.1) gains memory_store, mcp, and engine_version — feature-compiled facts checkable at placement admission (profile.tools.memory ⇒ memory_store; non-empty mcp_servers ⇒ mcp). Per-spec facts that cannot be known at bind are preflighted on the member host before any session is created, adjudicated by MobHostBindingAuthority from shell-supplied typed observations: model resolvable in the member binary’s effective registry (canonical() + shipped custom_models), auth_binding name-ref resolvable via the same ProviderRuntimeRegistry path the build uses, every stdio MCP command discoverable, realm mob.{id} attachable with a persistent backend when durable_sessions was declared. Preflight is TOCTOU-approximate by nature and stated as such; it bounds the failure window, it does not eliminate it. WHY: the async-MCP path is silent at spawn granularity (router.rs:1488-1508 acks nothing; failure lands minutes later at :1723-1743), and a provider-auth failure would otherwise first surface as a turn error on a host the console may not yet observe — both contradict §9’s fail-quiet prohibition. Rejects feed member_materialization_failures and the revival ladder exactly like any materialize failure (§9 row 1).
R5 — Secret-bearing fields are barred from remote specs; the payload is declared non-confidential.
Remote placement admission typed-rejects a spec whose shell_env, any McpStdioConfig.env, or any McpHttpConfig.headers is non-empty (SecretBearingField { field }). Operators provision environment on the member host (the rkat mob host process env, which spawned stdio servers and shell subprocesses inherit) — mirroring how provider keys already never leave the host (D3; factory.rs:3961-4011). WHY: these three fields are exactly where operators put tokens, and the transport is signed plaintext (7.6.1). Silently stripping them would be a capability lie; shipping them would leak secrets to the wire and durably into two hosts’ stores. A member-host env-indirection vocabulary and transport confidentiality are explicitly separate tracks (§12 seed), not smuggled into this payload.
R6 — Process-local tool families get per-family dispositions; tools.mob gets a bridge operator-upcall lane because the settled B2→B21 goal requires it.
Composition with plane (b): the upcall lane is the agent authority lane (generated
MobToolAuthorityContext), distinct from principal ControlScope grants (§8). One fact, one owner: agent mob-operator capability = machine-generated authority context; console principal capability = grants. Phase-5 grants do not gate member upcalls and member upcalls never satisfy principal scope checks.
R7 — Hooks and host skill repositories are host policy, declared as such.
A member materialized on Host B runs Host B’s layered hook config (sdk.rs:80-129) and sees Host B’s config-level skill repositories in its inventory; profiles carry no hook vocabulary (profile.rs:162-251 — NOT FOUND) and this plan does not add one. WHY: hooks and skill repos are operational host policy in the same class as the host’s filesystem and process env — the digest deliberately does not cover them, and this is recorded here so the divergence is a documented contract, not a discovered surprise. Profile-carried hook material would be new vocabulary with new secret-bearing surface (command paths, env) and is explicitly out of scope (§15.8).
R8 — Engine version is a recorded, surfaced fact; digest identity is scoped to it. No equality requirement.
BridgeCapabilities gains engine_version (the workspace/ContractVersion string) at bind; MemberMaterialized echoes it; HostStatus carries the refreshed capabilities record and therefore does not add an engine-version sibling (one carrier per message; A-weak9 — this is also realms R5’s binary-version observability). Build identity claims are always the pair (spec_digest, engine_version): embedded builtin skills (mob-communication, task-workflow, workgraph-workflow, build.rs:224-230) and canonical() model data (catalog.rs:142-156) are compiled into the binary, so equal specs on unequal binaries are honestly different builds. An ack engine_version differing from the bound record is a typed HostEngineVersionChanged materialize failure that routes through host rebind (the host restarted upgraded mid-flight). WHY hard equality is rejected: it would make every rolling upgrade a placement outage; the one failure class that matters operationally — model unknown on the member binary — is already caught precisely by R4 preflight (ModelUnresolvable).
15.3 PortableMemberSpec (contracts; deny_unknown_fields; every field enumerated against BuildAgentConfigParams build.rs:92-126, SpawnMemberSpec handle.rs:1844-1907, and the actor post-mapping actor.rs:9227-9235)
DeliverMemberInput, matching InitialTurnPolicy::Defer — build.rs:500-507); shell_env and MCP env/headers (R5); anything Arc-shaped (R6); hooks (R7); credentials (D3).
15.4 Machine deltas (catalog DSL; extends §6.1/§6.3)
MobMachine:- New field
spawn_profile_authority_resolved_spec_digests: Map<AgentIdentity, Option<String>>;AuthorizeSpawnProfileinput andSpawnProfileAuthorizedeffect gainresolved_spec_digest: Option<String>(exact-match echo extendsrequire_authorized_effect, spawn_profile_authority.rs:86-121). Invariant: remote placement ⇒SomebeforeSpawnadmits. ResolveSpawnMemberAdmission(§6.1) gains the portability validation — and placement/portability admission is enforced INSIDE the machine spawn-exec ladder atenqueue_spawn’sBeginSpawnExecopener, the one choke-point every spawn path crosses (agent tools, RPCmob/spawn*, embedderMobHandle::spawn_spec— the RPC/embedder paths bypass the tool-seam admission today, meerkat-mob-mcp/src/lib.rs:978-984; A4): typed reject causesNonPortableResource { kind: RustBundles | PerSpawnExternalTools | MobDefaultExternalTools | DefaultLlmClientOverride | HostSurfaceMcpAllowlist | WorkgraphTools }andSecretBearingField { field: ShellEnv | McpStdioEnv | McpHttpHeaders }, alongside §6.1’sMissingHostCapability. Schedule tools are portable under A3-final/§18.5: session-target schedules remain host-local and member-side mob-target schedules fail typed at fire.HostCapabilityFlags(§6.1) gainsmemory_store,mcp,engine_version.- Invariants:
RequestMemberMaterialization.spec_digest= the authorized resolved digest for that identity;CommitSpawnMembership(remote arm) requires the ack-echoedspec_digestto equal it and records the echoedengine_version; ackengine_version≠ boundhost_capabilities.engine_version⇒ typedHostEngineVersionChangedmaterialize failure feeding rebind.
- Materialized-members value extends to
{ generation, fence_token, session_id, spec_digest }. - New input
ResolveMaterializePreflight { agent_identity, generation, fence_token, spec_digest, observations: MaterializePreflightObservations }→ effectsMaterializeAdmitted/MaterializeRejected { cause }(dispositions declared forseam-inventory). Observations are shell-read facts (binary lookup, registry resolve outcome, realm backend probe) fed typed into the authority — the authority decides, the shell observes (RMAT read-seam discipline). - Dedup extension: same
(identity, generation, fence)+ samespec_digest⇒ replay returns the recorded result; same tuple + different digest ⇒ typedSpecDigestMismatch(one idempotency key can never name two builds). - Persistence: the host-side SQLite row gains the spec bytes + digest + engine-version-at-build (7.6.7).
SupervisorPendingRotationRecord precedent, meerkat-mob/src/store/mod.rs): the minted mob-operator authority facts per remote identity+generation, re-minted into a sealed context after each machine admission (R6); and the member-operator request ledger keyed (mob_id, agent_identity, requester_generation, requester_fence_token, request_id). Authority facts are covered by the resolved spec/machine transition witness. Request rows carry the typed-operation digest and move only Pending → Terminal; they are never capacity-evicted and are scrubbed only with the mob.
15.5 Wire deltas (meerkat-contracts; V4; regen-schemas/SDK/parity gates apply)
-
PortableMemberSpecas §15.3 (deny_unknown_fields);MaterializeMember.specbecomes this type. -
MemberMaterializedgainsspec_digest: String,engine_version: String. -
BridgeCapabilitiesgainsmemory_store: bool,mcp: bool,engine_version: String. -
BridgeRejectionCausegainsModelUnresolvable { model },AuthBindingUnresolvable { realm, binding },McpCommandMissing { server },RealmBackendUnavailable,SpecDigestMismatch,HostEngineVersionChanged(typed only;bridge-classifiergate applies, and every newBridgeReplyconsumer joinsBRIDGE_CLASSIFIER_FILESper Gotcha #1). -
New supervisor-addressed payload family over the
supervisor.bridgeintent:MemberOperatorRequest { agent_identity, requester_generation, requester_fence_token, request_id, op: MemberOperatorOp }/MemberOperatorReply—MemberOperatorOpis a closed enum mirroring the twelve operator tools (tools.rs:358-519). Direction is member-host → controlling host. The member runtime samples its current host binding stamp once per logical request; every transient resend is byte-identical, including generation, fence, request id, and operation. Admission is MACHINE-owned (A8):ResolveMemberOperatorAdmission { agent_identity, requester_generation, requester_fence_token, sender_peer_id, request_id }requires the signed generation+fence to equal the current identity binding, then checks roster peer-key binding, placement presence, and non-revoked host. A delayed envelope from a prior incarnation therefore rejectsStaleGeneration/StaleFenceeven whenResumereused the same session and peer key. Only after admission does the shell read the generation-pinned authority-facts record and re-mint the sealed agent context; no authority claim rides the wire. Idempotency is durable and fail-closed. The responder atomically insertsPending { op_digest }under(mob_id, agent_identity, requester_generation, requester_fence_token, request_id)before any effect. Same key + different digest is a stable request conflict and never overwrites the original. A matching immutableTerminal { reply }replays verbatim across reply loss and controlling-host restart. A recovered or concurrently observedPendingis never execution authority: it CAS-terminalizes to explicitupcall_indeterminateand is not re-executed (includingSpawnMember,SpawnManyMembers, andMobRunFlow). The actual completed reply is sent only after its terminal CAS succeeds or a durable terminal winner is reloaded. If terminal persistence fails after the effect, the caller receives non-success and the row remainsPending; the next delivery converges to the stable indeterminate terminal. If the initial Pending insert fails, no effect ran and the rejection is explicitly retryable. There is no capacity eviction; mob destroy is the sole scrub boundary. Reply-budget exhaustion on the member side remains typedupcall_timeout.
15.6 Preflight protocol and failure semantics (extends §9)
15.7 Durability and revival on the member host
The materialization spec is durable on the member host: theMobHostBindingAuthority SQLite row stores the spec bytes keyed by digest next to (generation, fence_token, session_id). Member-host restart revival (§9 row “Member host restart”) rebuilds the agent by re-running build_agent_config on the stored spec — profile.tools.mcp_servers recompose exactly as designed (their durability rationale, profile.rs:44-53), while the persisted transcript, including every ordered System message, is restored byte-for-byte and materialization injects nothing. No controlling-host contact is required. profile.resume_overrides (profile.rs:193-200) apply against the stored spec as profile truth; “updated profile” reaches a remote member only as a new MaterializeMember at a higher (generation, fence), which replaces the stored spec atomically with its dedup row. Realm provisioning: the member host creates/attaches realm mob.{id} locally through the single owner meerkat_core::mob_realm_id (connection.rs:261; build.rs:24-34); memory lives under that realm’s store path with unchanged session-scoped MemorySearchScope::for_session semantics (factory.rs:5644-5656) — session scoping means remote members’ memory behavior is byte-identical to local, just physically on the owning host.
15.8 v1 non-goals (typed degradation, never silent)
Listed in §12 spirit: no secret indirection vocabulary and no transport confidentiality (attempting to shipshell_env/MCP env/headers remotely ⇒ typed SecretBearingField admission reject — never silent stripping); no cross-host workgraph tools (⇒ typed NonPortableResource reject; a bridge-proxied dispatcher is a v2 seed), while schedule tools follow A3-final/§18.5 (session-target schedules are host-local and member-side mob-target schedules fail typed at fire); no remote rust bundles / per-spawn external tools / mob-default external tools / injected LLM clients (⇒ typed NonPortableResource rejects); no profile hook vocabulary (member hosts run their own hooks by declared policy R7 — the divergence is contractual, not accidental); no engine-version equality requirement (mismatch is recorded and surfaced; only ModelUnresolvable hard-fails); no cross-host memory or shared semantic index (memory is member-host-realm-local; session scoping preserves observable parity); no mobpack-based definition distribution (the spec is self-contained by R1; mobpack synergy is a v2 seed).
15.9 Phase assignments (§10)
- Phase 1 (catalog + contracts): §15.4 DSL deltas (resolved-spec digest field/input/effect echo, admission portability causes,
HostCapabilityFlagsextension,MobHostBindingAuthoritypreflight input/effects with dispositions), §15.5 wire types (PortableMemberSpec, ack echo fields, rejection causes,MemberOperatorRequest/Reply+MemberOperatorOp). TLC bounds unchanged (2 hosts × 3 members). - Phase 2 (host bind): capability advertising (
memory_store,mcp,engine_version) in the bind ceremony reply; recorded viaCommitHostBind. - Phase 3 (placement + materialization): the spec compiler on the controlling host (Path→Inline resolution, R3 prompt base resolution, authority mint + durable record), member-side preflight ladder + dedup, durable spec row + revival recompose, portability/secret admission rejects, and the member-operator upcall lane — it gates this phase’s own exit criterion (B2→B21/B22 on Host B).
- Phase 5 (scopes): pin the lane separation — member upcalls stay generated-authority-adjudicated; principal grants neither gate nor are satisfied by them (one owner per fact).
- Phase 6 (events): MCP
Failedlifecycle visibility for remote members falls out of the pump; no additional work beyond §7.4 itself. - Phase 7 (surfaces):
mob/member_statusMCP/tool-surface parity fields for remote members ride the existing projection work;rkat mob hostsurfaces preflight results in host status output.
15.10 Tests (extends §11; deterministic lanes are the ratchet)
unit: spec digest determinism + canonicalization; Path→Inline resolution fail-closed (unreadable path fails the spawn, build.rs:530-538 parity); R3 — compiler output for remote placement never containsInherit (prompt and tool policy); portability matrix — each NonPortableResource kind and SecretBearingField field produces exactly its typed cause; materialize dedup — same tuple+digest replays recorded result, differing digest ⇒ SpecDigestMismatch; upcall admission — unknown identity / sender-key mismatch / stale generation / stale fence rejected before authority lookup; request ledger — Pending-before-effect, conflicting digest preservation, same-key concurrency, post-effect store failure, immutable terminal replay, full generation+fence key, and retention beyond 1,024 rows; MemberOperatorOp covers exactly the operator tool surface (closed-enum pin).
int / e2e-fast (two-hosts-in-one-process harness): same-spec-two-hosts build equality — digest match ⇒ identical tool-catalog names and system-prompt bytes on both MeerkatMachines (same binary); skill-less remote member’s prompt contains the controlling host’s base and NOT the member host’s AGENTS.md; materialize preflight rejects each of ModelUnresolvable / AuthBindingUnresolvable / McpCommandMissing / RealmBackendUnavailable and each feeds member_materialization_failures + revival classification; MCP connect failure after passing preflight ⇒ member runs, Failed lifecycle event appears in the merged mob stream, materialize ack unaffected; member-host revival recomposes MCP router + prompt from the durable spec row with zero bridge traffic asserted; B2→B21 upcall spawn end-to-end (spawn, list, retire via upcalls); reply-loss + controlling restart replays the persisted terminal with one effect; an effect-before-terminal crash window recovers indeterminate without re-execution; a held old envelope rejects after a same-session generation/fence bump even though the member key is unchanged; real SQLite reopen preserves terminals and mob destroy scrubs the ledger; remote member with tools.mob=false exposes no upcall surface; resume_overrides re-apply only from a new spec at a higher generation; TLC green at 2×3 bounds for the new digest/preflight state.
e2e-system (real multi-process TCP): materialize against a real member host with the stdio MCP binary present vs absent; host restart revival from the SQLite spec row; engine version captured at bind and echoed at materialize; mid-flight host upgrade ⇒ HostEngineVersionChanged ⇒ rebind ⇒ retry succeeds.
e2e-smoke (live): remote member resolves a real provider via name-ref auth_binding on the member host; no credential material observed on the wire (payload capture assertion).
16. Live channels (realtime) for remote members
Scope: thelive/*realtime plane (audio/text live channels) for mob members whose sessions live on a member host. The v4 plan text was previously silent on this plane —live/*appears nowhere in §§1-13. This section closes that gap without re-litigating D1-D6.
16.0 Verified substrate (what exists, what fails today)
- Live is an RPC-process-only surface:
rkat-rpcflags--live-ws/--live-ws-scheme/--live-tool-timeout-ms(meerkat-rpc/src/main.rs:53-81); everylive/*dispatch arm is registered only when a transport is attached (live_enabled(),meerkat-rpc/src/router.rs:1064-1079; arms :1958-2077; pinned byassert_no_live_methods, router.rs:10232-10240). REST mounts no live endpoints; the CLI has no live wiring (NOT FOUND outside meerkat-live + rkat-rpc main). live/openresolves the target session strictly process-locally: the B17 gateruntime.session_state()consults staged sessions + the localSessionServicelist and returnsINVALID_PARAMS "session {id} not found"otherwise (meerkat-rpc/src/handlers/live.rs:871-886overmeerkat-rpc/src/session_runtime.rs:6902-6948). A consolelive/openagainst a remote member’s session fails today at this gate — and that behavior is correct under §3 (realms are not distributed; the session genuinely does not exist in the controlling host’s realm). We do not “fix”live/open; we add an identity-addressed mob surface (DL3).- The bootstrap indirection is already designed:
LiveOpenResult.transportcarriesWireLiveTransportBootstrap::Websocket { url, token }with an absolute URL (meerkat-contracts/src/wire/live.rs:155-206), built as"{base_url}{LIVE_WS_PATH}?token=..&channel=..&format=.."(handlers/live.rs:1196-1202) wherebase_url = "{scheme}://{listener.local_addr()}"(main.rs:511-519). SDK clients connect that URL directly — the SDK deliberately does not own the WS transport (sdks/typescript/src/client.ts:3073-3080). - Channel binding + one-active-channel-per-session are owning-session MeerkatMachine facts (
live_active_channel_by_session/live_channel_session_by_channel,meerkat-machine-schema/src/catalog/dsl/meerkat_machine.rs:2726-2760;ResolveLiveOpenAdmissionAcceptedguardssession_not_active/channel_not_bound, dsl:17402-17421). WS bearer tokens are machine-adjudicated in the same session’s machine: mint viaRecordLiveWebsocketTokenIssued, single-use/TTL/channel-pinned admission viaResolveLiveWebsocketTokenAdmission(dsl:17981-18069;TOKEN_TTL= 60s,meerkat-live/src/transport.rs:57). - The WS wire protocol carries only typed input inbound (Text =
LiveInputChunkWire, Binary = negotiatedpcm_24k_mono) and observations outbound (transport.rs:5-33). The control verbs (commit_input,interrupt,truncate,refresh,close,status) exist only as RPC methods (router.rs:1987-2077). - Realtime gates evaluate on the process that owns the session: B19
ModelNotRealtimefrom the catalogrealtimeflag (meerkat/src/session_runtime/live_orchestration.rs:46-70; exactly one catalog row isrealtime: true—gpt-realtime-2,meerkat-models/src/capabilities/openai.rs:312); B18 provider-adapter support is factory-owned (OpenAiRealtimeSessionFactory::supports_provider == Provider::OpenAI,meerkat-openai/src/live.rs:3043-3050; #302 no-factory fail-closed check,handlers/live.rs:888-901). Realtime credentials resolve per open from the owning process’s realm config chain (meerkat-rpc/src/live_wiring.rs:25-38, wired atmain.rs:376-390); the factory holds no credential material. - Live observations project into the owning session’s canonical stores via the mandatory projection sink (
main.rs:401-436;LiveAdapterHostrequires the sink at construction,meerkat-live/src/host.rs:1535-1546), so live-derived transcripts/status land in the owning host’s session/EventStore. - The supervisor bridge has no live/audio/realtime commands (
BridgeCommand,meerkat-contracts/src/wire/supervisor_bridge.rs:238-251— NOT FOUND for any streaming/media variant) and the bridge client is single-flight (request_lock,meerkat-mob/src/runtime/supervisor_bridge.rs:35-49).meerkat-livehas no production WS client (the onlytokio_tungstenite::connect_asynchits are insidemod tests,transport.rs:928+; the sole outbound WS client in the workspace is the OpenAI adapter’s provider connection). live/openalready anticipates mob-owned sessions locally:ensure_live_peer_ingressskips session-owned drain reconfiguration whenpeer_ingress_owner.is_mob_owned()(meerkat-rpc/src/session_runtime.rs:4344-4365, skip at :4357).- The
--live-wsbind bypassesTcpBindPolicy:validate_tcp_bind_policyis applied only to--tcp(main.rs:184-194); the live-ws listener binds any address unconditionally (main.rs:462-469) and is plaintext (wssis advertisement-only for a TLS-terminating proxy,main.rs:75-81).
16.1 Decisions
DL1 — Media plane: direct-to-owning-host bootstrap. The bridge carries the open and the control verbs; it never carries frames. The controlling host proxiesopen over the bridge; the owning host runs its local open pipeline unchanged (machine admission → host channel → B19/B18 prechecks → per-open credential resolution → token mint → bootstrap build) and the returned LiveOpenResult — owning host’s absolute WS URL + token — is passed back verbatim. The console’s client then connects the WS directly to the member host.
WHY: (a) the indirection is already designed in — absolute URL + direct-connect SDK contract (wire/live.rs:176-206; client.ts:3073-3080); pointing it at another host requires zero wire-shape change. (b) Token authority is owning-session machine state (dsl:17981-18069); direct bootstrap keeps mint, single-use admission, and listener co-located. A controlling-host-minted token for a member-host listener is unrepresentable without new shared-state machinery — we do not invent it. (c) The relay alternative has no substrate and is costed honestly: the bridge is single-flight request/response over signed envelopes with a 30s verified-ack budget (supervisor_bridge.rs:35-49), there are no streaming bridge channels, meerkat-live has no production WS client, and D5’s long-poll page primitive is latency-unfit for realtime audio. A relay would be a new transport plus a shadow channel registry beside LiveAdapterHost — parallel authority, doubled latency/jitter on the one plane where that is fatal. (d) This is a scoped, declared exception to D2’s single-endpoint property, for the media plane only. Control (open/close/verbs) and observation (transcripts/events/history) stay proxied through the controlling host exactly per D2/§7.4. §12 already treats direct observer→member-host streams as the v2 optimization because long-poll suffices there; for realtime audio the ordering inverts — the deferred thing becomes the only thing that works.
DL2 — Carrier: member-addressed V4 bridge commands, admitted by MeerkatMachine; MobMachine owns zero live-channel facts.
OpenMemberLiveChannel / CloseMemberLiveChannel / MemberLiveChannelStatus / ControlMemberLiveChannel extend the §6.4 member-addressed family (ResolveSupervisorBridgeCommandAdmission classification; runtime_alphabet_parity forces explicit coverage).
WHY: the channel binding and the one-active-channel invariant are session-machine facts (dsl:2726-2760, 17402-17421) — admission must land where the binding lives. A host-addressed adjudicator (MobHostBindingAuthority) would be a second owner for the same fact, violating §3’s no-parallel-authority constraint. The controlling host is a scope gate + conduit: it holds no live channel state, not even a projection map (a remote channel’s status is a proxied point read, DL3). Host-scoped live facts (the advertised WS endpoint) are bind-ceremony data (DL5), not command admission.
DL3 — Console surface: identity-addressed mob/member_live_open|close|status|control, placement-blind; live/* is untouched.
WHY: live/* is session-scoped and process-local by design; a remote member’s session_id names a session in another host’s realm, so the B17 not-found (handlers/live.rs:871-886) is honest, not a bug. Identity → (placement, session) resolution is MobMachine truth (member_placement §6.1 + materialization-ack session binding §7.3; MobMemberStatusResult.current_session_id already projects the binding, meerkat-contracts/src/wire/mob.rs:2024-2043). The mob handlers own MobMachine access and are the plane-(b) scope chokepoint (§8) — so the placement branch lives there: local placement resolves the member’s session and invokes the same internal pipeline the live/* handler uses; remote placement dispatches the bridge command. One verb family, same LiveOpenResult shape either way; the console never branches on placement.
DL4 — One canonical pipeline: complete the LiveOrchestrator extraction so RPC handler and member-host bridge responder share one open/close/control path.
WHY: the open sequencing today is welded into the RPC handler (handle_live_open, handlers/live.rs:842-1300) against meerkat_rpc::SessionRuntime, while its inner seams already live in the facade (LiveOrchestrator, meerkat/src/session_runtime/live_orchestration.rs:498; the RPC runtime delegates live_open_config_for_session/precheck_live_open to it, meerkat-rpc/src/session_runtime.rs:3058-3096; the facade already has the optional live feature pulling meerkat-live, meerkat/Cargo.toml:34,126). The member host is rkat mob host (D6), not rkat-rpc — it must execute the identical pipeline including the fail-closed open-failure cleanup (close_live_channel_after_open_failure, handlers/live.rs:725-830) and ensure_live_peer_ingress’s mob-owned skip (session_runtime.rs:4357). Extraction is the concrete seam that makes “one path” true rather than aspirational; without it the bridge responder would be a shadow copy of a 500-line handler. The extraction scope explicitly includes the projection-sink roles: SessionServiceProjectionSink is today the ONLY production implementation of LiveProjectionSink + LiveChannelCloseFeedback + LiveChannelStatusFeedback + LiveWsTokenAuthority and its constructor takes Arc<meerkat_rpc::SessionRuntime> (meerkat-rpc/src/live_projection_sink.rs:104-113, wired into all four roles at main.rs:406-418) — the member host cannot run token mint or projection without a facade-owned implementation over its own session service. Decoupling those four roles from the RPC runtime is 6b pre-work, scheduled in §16.7.
DL5 — Capability declaration: bind-time fact = the advertised live WS endpoint; model/provider realtime gates stay owning-host open-time typed rejects, round-tripped as typed bridge causes.
CommitHostBind (and HostRebound) carry an optional live endpoint; MobMachine records host_live_endpoints: Map<HostId, LiveWsEndpointUrl>. Absence = live-incapable — no boolean shadow flag. Before dispatching OpenMemberLiveChannel, the controlling host requires host_live_endpoints[placement] to be present; typed LiveTransportUnavailable reject otherwise, with no bridge dispatch.
WHY: the endpoint is the one fact the controlling host must know before proxying, and its presence is the capability (single fact, single owner — a separate live_websocket boolean would be dual truth). Provider-adapter support is deliberately factory-owned (“B18 is owned by the concrete realtime factory… because the factory mints the adapter”, live_orchestration.rs:44-49) and model realtime capability is catalog-owned per open (live_orchestration.rs:46-58) — the session’s model can be reconfigured after bind, so declaring provider/model sets at bind would duplicate those owners and drift. The listener→factory coupling is enforced at host boot exactly as rkat-rpc does today: a configured live transport without the openai-realtime feature is a typed startup refusal (main.rs:313-324, mirrored in the host role). D5’s declared-degradation discipline is satisfied: the console can label a live-incapable host honestly from bind-time truth, and the owning host’s typed open-time errors are the fail-closed backstop for drift.
DL6 — Advertised-URL correctness: the host role requires an operator-declared advertised base URL; local_addr derivation stays the single-host default.
rkat mob host --live-ws <addr> requires --live-ws-advertise <ws|wss absolute base URL> (typed startup error when missing); that URL (with its scheme) is what lands in the bind descriptor, in host_live_endpoints, and in the owning host’s LiveWsConfig.base_url — so minted bootstrap URLs are correct by construction. Standalone rkat-rpc keeps today’s scheme://local_addr derivation (main.rs:511-519).
WHY: local_addr-derived URLs break cross-host (0.0.0.0 binds, NAT, hostnames, TLS proxies). This mirrors host_endpoints (§6.1): advertised, never derived. The URL is a MobMachine bind fact — it must NOT be added to RuntimeHostInfo (anti-authority pin, meerkat-contracts/src/wire/host.rs:181-210, gotcha #2).
DL7 — Exposure posture: bind-policy parity now; wss-behind-proxy is the production posture, not a hard floor.
Close the pre-existing gap: apply validate_tcp_bind_policy("live-ws", ...) / --allow-remote to the live-ws bind in both rkat-rpc and the host role (today only --tcp is validated, main.rs:184-194 vs :462-469). Keep token-only auth on the WS upgrade; record the advertised scheme in the bind fact; document TLS-terminating-proxy fronting (wss) as the production posture.
WHY: this matches the shipped secure_rpc doctrine — transport exposure is an explicit opt-in, not an auth mechanism (main.rs:48-52). Hard-requiring wss would make every member host depend on external TLS termination (Meerkat ships none; the scheme is advertisement-only, main.rs:75-79) — an operational floor that defeats the LAN/dev topologies mobs target. The replay window is bounded by machine-adjudicated single-use, 60s-TTL, channel-pinned tokens (transport.rs:57; dsl:18024-18069). Behavioral change, named: standalone rkat-rpc --live-ws 0.0.0.0:x without --allow-remote, silently accepted today, becomes a typed startup error.
Wire exposure, named (A5): MemberLiveChannelOpened round-trips the WS bearer token over the controlling↔member comms link, which is Ed25519-signed but NOT encrypted (no cipher exists in meerkat-comms — verified NOT FOUND). An on-path reader of that link can read the token during its single-use / 60s-TTL / channel-pinned window. This is the ONE sanctioned exception to the secrets-never-on-bridge invariant — a short-lived capability token, never a durable credential — recorded in the threat model (§20) with the mitigations (token bounds; tunnel the inter-host link on hostile networks) and the v2 seed (encrypted comms transport) that closes it.
DL8 — Scope (plane b): new ControlScope::Live gates the entire live family; bootstrap issuance is itself the scope-gated act.
mob/member_live_open|close|status|control all require Live at the controlling host’s chokepoints (sealed ResolvedControlPolicy, §8); deny = ScopeDenied { required, presented }. Owner-implicit-full default unchanged.
WHY: a WS bootstrap is a duplex bearer capability — typed input inbound plus media/transcript outbound on one socket (transport.rs:5-33). It does not decompose into SendCommand/SubscribeEvents halves, and a composite check would misrepresent the grant: handing out the token is the authorization event, after which the controlling host is out of the loop by design (DL1). Barge-in interrupt on a live channel is media-plane conversation control and sits inside Live; lifecycle HardCancelMember stays under Cancel. The §8 ordering rule applies verbatim: no remote live surface ships before phase 5 lands.
DL9 — v1 transport scope: WebSocket only; transport=webrtc for a remote member is a typed reject.
WHY: WebRTC signaling is bound to the local RPC connection and local LiveWebrtcState with its own machine-adjudicated token family (handlers/live.rs:1204-1287; dsl:2748-2755, 17700+); proxying live/webrtc/answer doubles the proxied command family for a feature-gated transport and round-trips SDP through the single-flight bridge. WS covers the full capability matrix. Reject = LiveTransportUnsupported { requested: webrtc }; the signaling proxy is a named v2 seed (§12), mirroring how direct observer streams were scoped.
DL10 — Input plane: no bridge-carried input or media frames; turn-level control verbs only.
ControlMemberLiveChannel.verb ∈ { CommitInput, Interrupt, Truncate, Refresh } — the RPC-only turn-level verbs. Frame-level input (text chunks and audio) rides the direct WS the client already holds (Text = LiveInputChunkWire, transport.rs:22-28). There is no bridge SendInput.
WHY: the bridge is single-flight (supervisor_bridge.rs:35-49); per-chunk input would serialize against lifecycle commands — the exact substrate-unfitness that killed the relay option. This is not a surface rejection: the WS is the input plane for live channels on every placement, and live/send_input remains available locally by session id exactly as today.
16.2 Observation parity (nothing new to build)
Live transcripts, close feedback, and status project into the owning session’s canonical stores through the mandatorySessionServiceProjectionSink (main.rs:401-436; host.rs:1535-1546). On a member host running persistent sessions (D5), those facts land in its EventStore and flow to the console via the already-planned §7.4 PollMemberEvents pump and ReadMemberHistory pages. There is no separate live event proxy. Live tool calls raised mid-turn dispatch into the owning session’s dispatcher on the owning host (RuntimeLiveToolDispatcher, session-scoped, process-local) with the host-owned per-call timeout — unchanged.
16.3 Machine deltas (catalog DSL; amends §6)
- §6.1 MobMachine — one new field:
host_live_endpoints: Map<HostId, LiveWsEndpointUrl>(sealed newtype over a scheme-qualifiedws|wssabsolute base URL; ctor validates scheme+shape). Populated/refreshed only by the existingCommitHostBind/HostReboundinputs (payloads gain the optional endpoint); cleared byRevokeHost. No new inputs, no new effects. Invariants:keys(host_live_endpoints) ⊆ { h ∈ mob_hosts : host_bind_phase[h] = Bound }; rebind without the endpoint clears the entry (capability is re-declared each bind, per D5’s restart-truthfulness rule). - §6.4 MeerkatMachine — member-addressed admission classification coverage for the four new commands (
OpenMemberLiveChannel,CloseMemberLiveChannel,MemberLiveChannelStatus,ControlMemberLiveChannel) inResolveSupervisorBridgeCommandAdmission; the typed command manifests (runtime_alphabet_parity) force this. Fence posture matches the rest of the member-addressed family in v1: supervisor identity + epoch on delivery, supervisor-sideStaleFenceTokenon submit (§6.4’s recorded status quo — no new member-side fence check is smuggled in here). - §6.5 / §8 MobMachine grants —
ControlScopevocabulary gainsLive; flows through the existingGrantOperatorScopes/RevokeOperatorScopesinputs and the sealed policy unchanged (data extension, no machine-shape change). - Owning-side live authorities: explicitly unchanged.
ResolveLiveOpenAdmission*,RecordLiveWebsocketTokenIssued,ResolveLiveWebsocketTokenAdmission*, refresh/close/command authorities (dsl:17402-18069) run on the owning host exactly as shipped. The controlling host never mints or validates live tokens and holds no channel map — remote channel status is a proxied point read, not a projection cache.MobHostBindingAuthority(§6.3) gains nothing: live commands are member-addressed (DL2).
make machine-codegen / machine-check-drift / machine-verify, runtime_schema_parity, runtime_alphabet_parity, seam-inventory dispositions for the extended bind payload; TLC bounds unchanged (2 hosts × 3 members, one live-capable and one live-incapable host in the new-state instances).
16.4 Wire deltas (meerkat-contracts; rides the §7 V4 protocol bump)
NewBridgeCommand variants (member-addressed, V4-required, deny_unknown_fields):
OpenMemberLiveChannel(BridgeLiveOpenPayload { supervisor, epoch, protocol_version, turning_mode: Option<RealtimeTurningMode>, transport: Option<LiveOpenTransport>, seed_max_chars: Option<usize> })— mirrorsLiveOpenParamsminussession_id(wire/live.rs:123-129; identity addressing replaces it).CloseMemberLiveChannel(BridgeLiveChannelPayload { supervisor, epoch, protocol_version, channel_id })MemberLiveChannelStatus(BridgeLiveChannelPayload)ControlMemberLiveChannel(BridgeLiveControlPayload { supervisor, epoch, protocol_version, channel_id, verb: BridgeLiveControlVerb })withBridgeLiveControlVerb = CommitInput | Interrupt | Truncate(mirror of live/truncate params) | Refresh.
BridgeReply variants: MemberLiveChannelOpened { result: LiveOpenResult } (embeds the existing contracts type verbatim — same crate, wire/live.rs:155-160 — so the owning host’s absolute URL + token round-trip untouched), MemberLiveChannelClosed { status }, MemberLiveChannelStatusReport { ... mirrors live/status result }, MemberLiveChannelControlled { ... mirrors the per-verb results }.
New BridgeRejectionCause variants (typed only — the bridge-classifier gate forbids ResponseStatus reinterpretation): ModelNotRealtime { model, provider }, LiveAdapterUnavailable { provider }, LiveTransportUnavailable, LiveChannelAlreadyBound, LiveChannelNotFound, LiveTransportUnsupported { requested }. Every owning-host typed failure maps onto one of these — never a generic failure; the controlling host maps them back onto the same error classes the local path emits (handlers/live.rs:1021-1050 vocabulary).
Host bind payloads (§7.2): the host binding descriptor and BindHost/rebind replies gain the optional advertised live endpoint (DL5/DL6).
Console DTOs: mob/member_live_open { mob, member, turning_mode?, transport? } → LiveOpenResult, mob/member_live_close, mob/member_live_status, mob/member_live_control (RPC + generated SDKs; the CLI wraps the RPC family). REST and MCP intentionally expose no live mutation/status verbs in v1, matching §17.2 SD-2/SD-4. Gates: make regen-schemas, verify-schema-freshness, verify-version-parity, verify-sdk-codegen-freshness, verify-rpc-surface-alignment, verify-rest-surface-alignment. BRIDGE_CLASSIFIER_FILES (xtask/src/bridge_classifier.rs:27-31) must gain every new BridgeReply-consuming file — the mob-side live proxy module at minimum (gotcha #1).
16.5 Member-host live composition (amends D6 role recipe)
rkat mob host gains the live plane as composition, not authority: --live-ws <addr> + required --live-ws-advertise <url> (DL6) + --live-ws-scheme folded into the advertise URL; LiveAdapterHost with the mandatory projection sink over the host’s own session service (the main.rs:401-436 wiring shape); build_per_open_realtime_session_factory over the member host’s own realm config chain (live_wiring.rs:25-38) — consistent with §3, the OpenAI binding must exist in the member host’s realm; openai-realtime feature preflight (typed startup refusal without it, main.rs:313-324 pattern); live-ws bind gated by TcpBindPolicy (DL7). The bridge responder’s live arms call the extracted LiveOrchestrator pipeline (DL4), including ensure_live_peer_ingress — whose mob-owned-ingress skip (session_runtime.rs:4357) must hold for host-materialized members so a remote live open never reconfigures a mob-owned comms drain.
16.6 Failure semantics (amends §9 table)
16.7 Phase placement (amends §10)
- Phase 1 additions: §16.3 catalog deltas (bind-payload endpoint field, §6.4 classification coverage,
ControlScope::Live) + §16.4 contracts (commands, replies, causes, console DTOs) in the same change-sets. - Phase 2 additions: host-role live composition (§16.5);
--live-ws-advertise; bind descriptor + ceremony carry the live endpoint;TcpBindPolicygap closure in both binaries (DL7). - Phase 5 addition:
Livescope in the grant vocabulary + sealed-policy enforcement rows for the live verb family. - New Phase 6b — Remote live channels (gated on phase 5 per §8’s ordering rule AND on phase 6’s deliverables — the §7.4 pump carries live observation parity (§16.2) and the bridge single-flight relaxation is what keeps live control verbs from serializing behind long-polls; 6b therefore lands AFTER phase 6, with only the
LiveOrchestrator/sink extraction pre-work parallelizable):LiveOrchestratorpipeline extraction (DL4); member-host bridge responder arms; controlling-host proxy dispatch + placement branch + typed cause mapping; reply-loss reconciliation;BRIDGE_CLASSIFIER_FILESextension. Exit gates: the phase-wide machine/contracts gates plus the deterministic test rows below. - Phase 7 additions:
mob/member_live_*on RPC + generated SDKs (+rkat mob live ...CLI verbs); REST/MCP remain intentionally absent per §17.2 SD-2/SD-4; surface-alignment gates. - Phase 8: MobKit consumes the new surfaces; still no MobKit-owned live registry.
16.8 Test matrix additions (amends §11; deterministic lanes are the ratchet)
unit/int/e2e-fast (two-hosts-in-one-process harness; deterministic realtime via a scripted RealtimeSessionFactory fake — precedent: RealtimeMockSessionFactory, meerkat-openai/tests/mock_realtime_ws/server.rs:254-296, and RecordingProviderRealtimeFactory, meerkat/src/factory.rs:6792; lift into a shared fixture):
- V4 live command/reply/cause encode-decode + fail-closed decode on V2/V3 receivers; admission classification for all four commands (alphabet parity green)
- proxied open returns the owning host’s bootstrap verbatim: URL base = advertised URL (not local_addr), token minted by the owning session’s machine; direct WS connect to the member-host listener drives text+audio end-to-end against the scripted factory
- cross-host token misuse matrix: replay against wrong channel, expired token, double-consume, token minted by host A presented to host B — all fail closed
- duplicate proxied open →
LiveChannelAlreadyBound; open on non-realtime model →ModelNotRealtimeround-trip; open on live-incapable host →LiveTransportUnavailablewith zero bridge traffic (asserted); webrtc-on-remote →LiveTransportUnsupported - control-verb proxy: commit_input/interrupt/truncate/refresh round-trip and mutate the owning channel; long-poll pumps (phase 6) do not block live control verbs (single-flight relaxation covers both)
- scope matrix:
Livegrants exactly the live family;SendCommand+SubscribeEventswithoutLive→ScopeDenied{required: Live}; owner-implicit-full unchanged - observation parity: transcripts projected by the owning sink arrive at the console via the §7.4 pump/pages — no live-specific event path exists (asserted by absence)
- reply-lost open: drop the bridge reply after owning-side open; status discovers the channel; close clears it; retry opens cleanly
- mob-owned-ingress skip holds for a host-materialized member on remote open (no drain reconfigure)
- bind-policy: non-loopback
--live-wswithout--allow-remoterefused (both binaries); host role without--live-ws-advertiserefused; advertise-URL newtype rejects non-ws/wss schemes - pinning:
live/*RPC methods still not advertised without a local transport (router.rs:10232-10240 stays green);live/openwith a remote member’s session id still returns not-found (B17 unchanged)
e2e-system (real multi-process TCP): real rkat mob host with a live listener; console opens by identity, connects the real WS across processes (scripted factory — no provider); member-host restart with WS attached → socket closes, rebind re-declares the endpoint, reopen succeeds; partition during proxied open → typed Unavailable, reconcile via status+close.
e2e-smoke (live, ignored-by-default): remote member on gpt-realtime-2 — full audio turn cross-host with real OpenAI credentials in the member host’s realm (extends the existing live smoke family, tests/integration/src/e2e_lanes.rs:189-190,213).
16.9 Non-goals (v1) — typed degradation, never silent
- Remote WebRTC transport:
LiveTransportUnsupported { requested: webrtc }typed reject at the controlling host. v2 seed:live/webrtc/answersignaling proxy over the bridge. - Media relay through the controlling host: no relay path exists and none is silently substituted — an unreachable advertised URL surfaces as the client’s WS connect failure after a prompt typed-success open (never a hang, never proxied audio). v2 seed only if operator evidence demands it.
- Bridge-carried input/media frames: no
SendInputbridge verb; the WS is the input plane on every placement.live/send_inputremains a local-session RPC verb; for remote members’ session ids it honestly reports not-found in the controlling realm. - In-process TLS termination:
wssstays advertisement-only (main.rs:75-79); production posture is a TLS-terminating proxy fronting the member-host listener, recorded in the advertised URL’s scheme. mob/member_statuslive-channel enrichment: status assembly stays bridge-free; live channel state is the dedicatedmob/member_live_statuspoint read (avoids per-status bridge fan-out and any controlling-side channel cache).- Agent-facing live delegation tools: NOT FOUND today (
meerkat-mob-mcp/src/agent_tools.rshas no live tools) and none are added — live channels are a console/operator surface in v1; an agent asking for one gets the ordinary unknown-tool error. Live tool dispatch during a live turn is unaffected (owning-hostRuntimeLiveToolDispatcher, unchanged). - Cross-host token mint/validation: unrepresentable by construction — token state lives only in the owning session’s machine; there is no API through which the controlling host could mint one, so there is nothing to degrade.
- Aggregate cross-host live budget: per-session enforcement only, consistent with §7.3’s budget stance.
16.10 Implementer gotchas (extends §13)
- The
--live-wsbind-policy gap (main.rs:462-469 vs :184-194) is pre-existing; closing it changes standalonerkat-rpcbehavior for non-loopback binds (now requires--allow-remote). Ship the change with its own release note; do not quietly fold it in. - The advertised live endpoint is a MobMachine bind fact — do not put it on
RuntimeHostInfo(anti-authority pin, wire/host.rs:181-210). live/open’s B17 not-found for remote session ids is correct; the fix is the identity-addressed mob surface, never a placement branch insidelive/*.MemberLiveChannelOpenedmust embedLiveOpenResult(contracts-owned, wire/live.rs:155) rather than re-declaring fields — one wire shape for local and remote opens, and theUnknown{debug}transport floor (wire/live.rs:190-206) rides along for free.- Every new
BridgeReplyconsumer file goes intoBRIDGE_CLASSIFIER_FILES(xtask/src/bridge_classifier.rs:27-31) — the mob live proxy is a new consumer by construction. - The deterministic realtime fake lives in
meerkat-openai/teststoday (mock_realtime_ws); the cross-host lanes need it as a shared fixture — budget the lift, don’t copy-paste the mock.
17. Surface consequence matrix — RPC / REST / MCP-server / agent tools / SDKs / WASM / CLI
Referenced by phase 7 (§10). Doctrine for this section: surfaces stay thin skins overSessionService/MobHandle. Remote coverage lands UNDER existing methods whereverAgentIdentityalready keys the API — there are no parallel “remote-flavored” verbs. Only genuinely new nouns (hosts, grants, route installs, member history) get new methods, and those register in the generated catalogs so every ratchet fires. Every mob surface change falls into exactly one of three consequence classes: (a) transparently remote (zero wire change; placement resolves behind the existing method), (b) new noun (full catalog cascade), (c) structurally untouched (asserted, so drift into it is reviewable).
17.1 Class (a): transparently remote — zero wire change
WHY this class exists: every member-targeted mob method already keys onMobMemberParams { mob_id, agent_identity } (meerkat-contracts/src/wire/mob.rs:644-648) — identity, not session id, not host. Placement resolution therefore happens in exactly one place: MobCommand admission on the controlling host’s MobActor (local member ⇒ existing path; member_placement[identity] present ⇒ bridge proxy per §7). No surface learns routing. The only observable surface change in this class is behavioral (remote members now answer) plus the new typed error causes (§17.4) surfacing through existing methods.
Placement wire carriers (class (b)-lite; additive optional fields, every one enumerated because every DTO is
deny_unknown_fields): MobSpawnParams (wire/mob.rs:683-720), MobSpawnSpecParams (:733-753), MobMemberSpecWire via MobEnsureMemberParams (:1404-1411), agent-tool SpawnMemberArgs/DelegateArgs (agent_tools.rs:1973/:1940 — LLM-visible schema delta, SD-7), MCP MeerkatMobSpawnInput/MeerkatMobSpawnManyInput (public_mcp.rs:342-350), and wasm SpawnSpecInput (meerkat-web-runtime/src/lib.rs:2113-2133, which typed-rejects it, §17.6) each gain placement: Option<WireHostRef>; each lands with its schema regen in phase 3.
Streaming semantics are unchanged on the wire: remote member events re-emit on the existing mob/stream_event notification (rpc_catalog.rs:972-977) with the same StreamScopeFrame::MobMember and canonical mob:<agent_identity> scope_id (meerkat-core/src/event.rs:1905-1927, 1966-1983). The durable (generation, seq) cursor rides only in bridge page replies and the new DTOs — never in StreamScopeFrame, whose agent_runtime_id/fence_token/generation fields are skip_serializing today (event.rs:1917-1926), and never in the two non-durable sequence domains (gotcha §13.8).
17.2 Class (b): new nouns — decisions (each fork closed)
(v4.1.2 errata, ADJ-P7-1: two wire additions landed in phase 7 itself rather than earlier phases —MobHardCancelParams { mob, member, reason }/MobHardCancelResult (no DTO existed; reason is honest — the handle verb consumes it) and MobMemberLiveStatusParams { channel_id: Option } (the ADJ-P6B-2 reply-loss discovery read must not be amputated on the wire; close keeps the required-id payload). WireHostCapabilityFlags was amended to u64 + BTreeSet<String> to match domain truth — no silent caps — and gained tracked_input_cancel as the separately negotiated exact-cancellation capability required by durable tracked turns. MobHostStatus.endpoint/authority_epoch/capabilities are Option — a Requested-phase host has committed no bind facts, and required fields would fabricate empties.)
SD-1 — Member history is a NEW by-identity method mob/member_history; it does not overload session/history. (Naming reconciled per A13: the host projection family is mob/hosts; the bridge verb stays HostStatus.)
WHY: every session/* method is keyed by realm-local session id (rpc_catalog.rs:104-429); a remote member’s session id belongs to another host’s realm (§3 constraint 1), so overloading session/history would break the session-id-is-local invariant and make SessionService placement-aware. mob/member_history / MemberHistory currently exists on no surface (NOT FOUND across meerkat-contracts, meerkat-rpc, meerkat-rest, meerkat-mob-mcp, meerkat-mcp-server src). Params follow the MobMemberParams convention: MobMemberHistoryParams { mob_id, agent_identity, from_index: Option<u64>, limit: Option<u32> } (deny_unknown_fields). Result is a new MobMemberHistoryResult wrapping the same transcript page body the bridge BridgeReply::MemberHistoryPage carries (§7), plus typed provenance: { page, generation: u64, placement: Option<WireHostRef>, next_index: Option<u64>, complete: bool } — one shape local and remote (pinned by test, §17.11), and provenance has a typed home instead of leaking into folklore fields.
SD-2 — REST posture: observation GETs land on REST; ALL admin (host bind/revoke, grants) is RPC+CLI only, recorded in the catalog.
WHY: REST already serves member observation (/mob/{id}/members/{agent_identity}/status, SSE events — rest_catalog.rs:436-507) and full session history (/sessions/{id}/history, rest_catalog.rs:166-173); a REST console that can watch a remote member but not read its history is a visible regression. Conversely REST has never carried mob create/wire/retire admin, and the transcript-edit family is the documented RPC-only precedent (rest_catalog.rs:174-181). New REST paths (GET-only): GET /mob/{id}/members/{agent_identity}/history → MobMemberHistoryResult; GET /mob/{id}/hosts → MobHostsResult; GET /mob/{id}/route-installs → MobRouteInstallsResult. Zero new REST mutations. The posture is recorded as a NOTE: comment in rest_catalog.rs in the transcript-edit style so verify-rest-surface-alignment readers see intent, not accident; each path lands in the same change as its axum router() arm (meerkat-rest/src/lib.rs:2021) or the gate fails (scripts/verify_rest_surface_alignment.py:1-16).
SD-3 — New RPC methods (8), all under the existing mob_enabled catalog flag; no new RpcMethodCatalogOptions field.
WHY mob_enabled: multi-host is not a build feature; the router parity test drives the catalog from cfg!(feature = "mob") (meerkat-rpc/src/router.rs:10328) and already documents the pattern for advertised-but-runtime-unavailable methods — dispatch to a typed capability error, never a catalog hole (the skills/list precedent, router.rs:10337-10341). A single-host runtime answers mob/bind_host with typed CapabilityUnavailable, not silence.
Grant methods ship in phase 5 with enforcement (§8 ordering rule) and require the new
ControlScope::AdminGrants (A9 — AdminHost stays exactly ‘(bind/revoke hosts)’; grant administration is its own scope, so ScopeDenied { required: AdminGrants, presented } is well-formed for the most privileged verb family). A default-deny plane with no grant surface is unusable the moment a second principal appears. Method names mirror the §6.5 machine inputs (GrantOperatorScopes/RevokeOperatorScopes). Every method above triggers the full six-step cascade: contracts types → rpc_catalog.rs descriptor → router dispatch arm (forced same-change by the parity sweep, router.rs:10325-10382) → docs/api/rpc.mdx row (scripts/verify_rpc_surface_alignment.py:12-16) → hand-written TS and Python wrappers (scripts/verify_sdk_wrapper_freshness.py:34-41) → make regen-schemas.
SD-4 — MCP server: observation tools only; host/grant admin is deliberately NOT on the MCP surface in v1.
WHY: MCP servers are routinely wired into agent tool contexts; an LLM-reachable AdminHost/grant mutation is a plane-(b) escalation vector, and the sealed-policy chokepoints (§8) are easiest to reason about while the granting surface set is minimal. This is the first deliberate asymmetry in the otherwise near-full meerkat_mob_* mirror (public_mcp.rs:497-524) — recorded here and in a roster comment so it reads as intent. New tools (3): meerkat_mob_member_history, meerkat_mob_hosts, meerkat_mob_route_installs. Mechanism per tool: descriptor in public_tools_list() (public_mcp.rs:467) + DISPATCH_TOOL_NAMES test slice (:496-524, pinned by public_tool_surfaces_have_no_drift) + dispatch arm in handle_public_tools_call; request lifecycles are inherited automatically because contributed_tool_lifecycles keys off the surface’s own advertised list and unknown names fail closed (meerkat-mcp-server/src/lib.rs:1824-1870). Note: meerkat_history stays session-id keyed (lib.rs:1745-1750) and cannot serve a remote member — the by-identity tool is the only MCP path to remote history. There is no MCP alignment gate script (verify_mcp_* NOT FOUND in scripts/): advertise↔dispatch parity is enforced in-process by the roster tests, but docs/api/mcp.mdx must be hand-updated in phase 7 — named as a work item, not left to memory.
SD-5 — Reachability/placement reach the wire as typed optional DTO fields + polled projections; NO new AgentEvent types; extending external_member: Option<Value> is prohibited.
WHY: §7.5 defines reachability as an observer-local projection — persisting liveness churn into the durable mob event log would contradict that (projection-promotion), and any new WireEvent type fires verify_sdk_event_inventory across three SDKs for no benefit (scripts/verify_sdk_event_inventory.py:1-24 — gate scope is exactly WireEvent.known_event_types). MobMemberStatusResult (wire/mob.rs:2022-2045) gains typed optional fields, all #[serde(default, skip_serializing_if = "Option::is_none")] so released SDK parsers are untouched: placement: Option<WireHostRef>, control_reachability: Option<WireReachability>, comms_reachability: Option<WireReachability>, last_seen_ms: Option<u64>, freshness_reason: Option<String> — following the tri-state WirePeerConnectivity precedent already on this DTO (wire/mob.rs:1998-2020). The untyped external_member: Option<Value> field on the same DTO is the natural folklore dumping ground (Rule-4 pattern); stuffing placement there is forbidden and pinned by a test asserting placement facts appear only in the typed fields. MobHostStatus (new): { host_id, endpoint, bind_phase, authority_epoch, capabilities: WireHostCapabilityFlags (single enumeration owner: §6.1 — durable_sessions, autonomous_members, hard_cancel_member, tracked_input_cancel, live transports, engine_version, resolvable_providers, protocol range), control_reachability, last_seen_ms, freshness_reason, materialized_member_count } — this is where consoles read durable_sessions=false to label ephemeral-host cursor gaps honestly (§9 cursor-overrun row).
SD-6 — SDK wrappers: full TS + Python wrappers for every new method; internal_exclusions is not widened.
WHY: the exclusion set’s existing entries are transport-internal only (initialize, session/stream_open|close, mob/stream_open|close — scripts/verify_sdk_wrapper_freshness.py:35-41); admitting app-facing admin verbs would weaken the ratchet for every future method. TS and Python SDKs are rkat-rpc clients (sdks/typescript/src/client.ts:2, 276), so the console keeps exactly one endpoint (D2) with zero SDK transport work.
SD-7 — SpawnMemberSpec.placement is an LLM-visible schema change on mob_spawn_member/delegate; agent tools get NO host/grant verbs.
SpawnMemberSpec (meerkat-mob/src/runtime/handle.rs:1844) gains placement: Option<...> (§7.3); the agent-tool input schemas re-derive from it, so the delegation tools’ advertised schemas change — a deliberate, reviewed prompt-surface delta. AgentMobToolSurface (agent_tools.rs:102) is a named plane-(b) chokepoint (§8): scope enforcement gates it exactly like the RPC/REST/MCP handlers. Agents never administer hosts or grants — those verbs simply do not exist in the agent tool roster (agent_tools.rs:46-61 stays host/grant-free).
SD-8 — Capability advertisement: one new flag RuntimeHostFeatureFlags.multi_host_mobs: bool; no new CapabilityId.
WHY: CapabilityId (meerkat-capabilities/src/lib.rs:28-49) is the mobpack-manifest requirement vocabulary; mobpacks do not declare placement in v1 (non-goal), so minting a manifest capability would advertise a contract nothing consumes. RuntimeHostFeatureFlags (wire/host.rs:36-51, already carrying mobs/external_members) is the wire flag structure consoles read via runtime/capabilities (rpc_catalog.rs:207-211). The flag is a non-authority projection; per-host capability detail lives in mob/hosts (SD-5). RuntimeHostInfo stays anti-authority — the pin runtime_host_info_does_not_claim_topology_authority (wire/host.rs:181-210) forbids topology/registry vocabulary on it, and multi_host_mobs as a feature flag on RuntimeHostCapabilities carries none of the forbidden tokens.
17.3 Class (c): structurally untouched (asserted)
Allsession/*, turn/*, events/*, auth/*, config/*, schedule/*, workgraph/*, live/*, comms/send|peers, blob/artifact/approval methods (rpc_catalog.rs:104-429 and siblings) are keyed by realm-local ids and stay placement-blind — a remote member’s session is read only through mob/member_history and the merged mob stream. runtime/host_info / runtime/capabilities / runtime/health (rpc_catalog.rs:202-216) remain read-only projections; the anti-authority pin (wire/host.rs:181-210) must keep passing — placement facts never land on RuntimeHostInfo (placement_labels keeps zero writers, §12). rkat-rpc --tcp remains a console transport with loopback-default bind policy (meerkat-rpc/src/main.rs:45-52, 184-190); it is never a host↔host control path — the bridge over the comms acceptor is the only inter-host control plane (§10 phase 7 note, restated as a member-host posture test in §17.11).
17.4 Typed error rendering across surfaces
The §7 bridge causes surface to users through the closed wireErrorCode enum (meerkat-contracts/src/error/mod.rs:25-41), which centralizes all four renderings as const matches — extending it forces exhaustive updates at compile time (jsonrpc_code, from_jsonrpc_code, http_status, cli_exit_code; :44-129). New codes:
MCP tool rendering is NEW WORK, not an existing seam (the draft’s
WireError claim was refuted — zero consumers in the MCP crates): the actual envelope is McpToolError { code: i32, message, data } (meerkat-mob-mcp/src/lib.rs:3934-3980) and mob errors are string-laundered via invalid_params(err.to_string()) today (public_mcp.rs:541,566). Phase 7 adds typed constructors + a non-laundering MobError → ErrorCode → McpToolError mapping so ScopeDenied{required,presented} / HostUnavailable / StaleCursor / StaleFence carry stable codes + typed data over MCP. SDK error classes key on the same codes (TS + Python; regenerated errors.json via make regen-schemas). REST mapping flows through the existing ErrorCode::http_status() seam used at meerkat-rest/src/lib.rs:2249, 2263 — no per-handler status folklore.
17.5 SDK consequences (TS / Python / web)
- TS + Python:
Mob.member_history()wrapper (+hosts(),route_installs(),bind_host(),revoke_host(),grant_scopes(),revoke_scopes(),grants()) — forced by the wrapper gate the moment the catalog entries land (SD-6). No SDK Mob class has any history method today (NOT FOUND in sdks/web/src/mob.ts, sdks/typescript/src/mob.ts, sdks/python/meerkat/mob.py). - Generated types: all three
generated/dirs regenerate (scripts/verify-sdk-codegen-freshness:11-15); no newWireEventtypes meansverify_sdk_event_inventorydoes not fire (SD-5). - Released-SDK compat: every touched DTO extension is optional +
skip_serializing_if. The web SDK’s closed-key parsers (requireOnlyKeys, sdks/web/src/mob.ts:96-107) guard exactly the spawn-response entry shape (:466-495) and event-source shapes (:514-548) — neither shape is touched by this plan’s wire deltas;MobMemberStatusResultis parsed tolerantly by named fields. Rule recorded: any future field on arequireOnlyKeysshape must update the web SDK parser in the same change or released browsers throw on contract-valid payloads. - Web SDK is console-only and RPC-free:
sdks/web/srcmust stay free of RPC-shaped method literals (scripts/verify_rpc_surface_alignment.py:27-29). Its multi-host work is generated-type regeneration plus the handwritten placement/status and spawn-response parsers needed for those generated contracts; it still gains no RPC console methods. Nobind_host/grant/member-history additions to the web Mob class in v1 — the browser observes only what the embedded runtime owns.
17.6 WASM / browser runtime
meerkat-web-runtime is an entirely in-process runtime — no TCP acceptor is representable in a browser (wasm exports enumerate the full in-browser mob surface, meerkat-web-runtime/src/lib.rs:1941-2801; nothing listens). Consequences:
- Typed placement reject (phase 3, with the field itself):
mob_spawnparsesspecs_jsonin-browser (lib.rs:2077); when a spec carriesplacement, it returns a typed unsupported-capability error (D5 doctrine: typed degradation, never silent strip). Silently ignoring placement would be the exact silent-cap pattern §6.5 kills for event streams. - No host exports:
mob_bind_host/grant/member-history wasm exports are not added; browser mobs are declared single-host in §17.12. - Compile-time obligation: every new contracts type (HostRef, cursors, host DTOs, error codes) must stay wasm32-clean —
wasm-checkruns on every code change andwasm-contracton contracts/web paths (.github/workflows/cargo.yml:518-541).
17.7 CLI
- Phase 2 (ceremony operability):
rkat mob hostdaemon (D6) — flags--identity-dir,--listen-tcp,--advertise-tcp,--descriptor-out, reusing the shippedrkat runacceptor flag family and descriptor mechanics (--comms-listen-tcp/--comms-binding-out, meerkat-cli/src/main.rs:1727-1790) with the host-flavored descriptor{kind: "host", address, public_key, bootstrap_token}(§7.2). Plus the controlling-side ceremony verbsrkat mob bind-host <mob> --descriptor <file>,rkat mob revoke-host,rkat mob hosts— the CLI drives mob APIs in-process, so these can precede the RPC methods and make the phase-2 exit gate operable end-to-end. - Phase 5:
rkat mob grant,rkat mob revoke-grant,rkat mob grants(land with enforcement; owner-implicit-full-scope keeps single-user CLI behavior unchanged, §8). - Phase 7:
rkat mob member-history <mob> <identity>,rkat mob route-installs <mob>. - Unchanged verbs (remote-covering for free):
MemberStatus,ForceCancel,Respawn,Logs,Attach,Status,WaitKickoff,SpawnHelper,ForkHelper(main.rs:2666-2890 — no host/grant/member-history verbs exist today, NOT FOUND in theMobCommandsenum;granthas zero hits in main.rs). - New exit codes 45-48 require NEW WORK, not an existing seam:
ErrorCode::cli_exit_code()has zero consumers inrkattoday (the binary exits EXIT_SUCCESS=0 / EXIT_ERROR=1 / EXIT_BUDGET_EXHAUSTED=2, main.rs:89-91, and mob handlers anyhow-stringify). Phase 7 adds a typedMobError → ErrorCode → exitpath for the mob verb family; the pre-existingEXIT_BUDGET_EXHAUSTED=2vscli_exit_code(BudgetExhausted)=21inconsistency is recorded and NOT silently changed for non-mob verbs.
17.8 Member-host surface posture (stated, non-contractual)
The member host runs nothing user-facing by default: no RPC, REST, or MCP listener — the supervisor bridge over the host comms acceptor is the only control plane (D6;rkat mob host must not embed rkat-rpc --tcp, whose bind policy is transport exposure, not principal auth — meerkat-rpc/src/main.rs:49-52 and secure_rpc.rs:3-4). Because member sessions materialize into the host’s realm-local stores (§3), a local rkat session list/rkat session show against that realm incidentally displays them. This is unsupported-but-harmless local observability, not a contract: no gate, no docs promise, and mutating a materialized member’s session locally is UNDEFENDED in v1 — stated plainly, not laundered: nothing validates fence/epoch member-side on delivery (§6.4), the new StaleFence checks gate only host-addressed materialize/release, and local realm writes sit entirely outside the fence path. A host-local operator with realm access can mutate placed members’ sessions; that is inside the compromised/trusted-host boundary recorded in the threat model (§20). Daemon lifecycle (start/stop/restart-rebind) is the OS/service manager’s job, not a Meerkat surface; restart-rebind semantics are §9’s member-host-restart row.
17.9 CI gate cascade per addition class
17.10 v1 non-goals for surfaces (typed degradation, never silent)
- REST admin (bind/revoke host, grant/revoke): not served; recorded
NOTE:in rest_catalog.rs (SD-2). Absent path = honest 404 by documented intent, not phantom catalog entry. - MCP host/grant admin tools: absent from the roster; unknown tool names fail closed as unknown tools (mcp-server lib.rs:1824-1870). Symmetry may return post-v1 behind proven
AdminHostscope. - Browser mobs are single-host: placement-bearing spawns get a typed unsupported reject in
meerkat-web-runtime; no host/grant wasm exports; no RPC literals in sdks/web (gate-enforced). - No direct observer→member-host endpoints (restates D2); consoles always talk to the controlling host.
- No new MCP alignment gate in v1: in-process roster parity is the enforcement; docs/api/mcp.mdx is a named hand-update.
- Member-host local surfaces non-contractual (§17.8): visible sessions are a realm-local side effect, typed-undefined for mutation, never advertised.
- No aggregated cross-mob host registry endpoint: hosts are per-mob MobMachine facts;
mob/hostsis mob-scoped. A fleet view is a MobKit projection concern. - No mobpack-manifest placement capability (
CapabilityIduntouched); packs that need placement fail closed at spawn admission with the §7.3 typed reject, not at pack load.
17.11 Surface test additions (lane-mapped; deterministic lanes are the ratchet — e2e-system/e2e-smoke do not run in GitHub CI)
unit:ErrorCode round-trip for the 4 new codes across all four const maps (+from_jsonrpc_code); new DTO serde round-trips incl. deny_unknown_fields rejects; MobMemberStatusResult decodes pre-extension payloads (absent optional fields) and placement never serializes into external_member (SD-5 pin); runtime_host_info_does_not_claim_topology_authority still green with multi_host_mobs flag.
int (meerkat-rpc): catalog dispatch-parity sweep auto-covers the 8 new methods; scope matrix per method — ReadHistory-only principal reads but cannot cancel, SendCommand-only drives but cannot read, deny carries {required, presented}; single-host runtime answers mob/bind_host with typed CapabilityUnavailable.
int (meerkat-rest): three new GET handlers; 403/410/503/409 mappings flow through ErrorCode::http_status(); alignment gate covers catalog↔router↔OpenAPI.
int (meerkat-mob-mcp / mcp-server): roster parity auto-pins the 3 new tools; meerkat_mob_member_history remote==local page shape; host/grant tool names dispatch as unknown (fail-closed asymmetry pin); agent-tool schema snapshot: mob_spawn_member exposes placement, spawn to unbound host surfaces the typed admission reject.
e2e-fast (two-hosts-in-one-process harness, §11): console flow on the controlling host — mob/member_history page-walk across a member respawn (generation bump), mob/hosts labels durable_sessions=false, mob/route_installs drains to complete; mob/stream_open{agent_identity} for a remote member delivers mob:<identity>-scoped events with unchanged notification shape.
e2e-system (local/nightly): rkat mob host daemon + rkat mob bind-host descriptor hand-off + rkat mob member-history over real TCP; assert the member-host process exposes no RPC/REST/MCP listener (§17.8); CLI exit codes 45-48 observed.
e2e-smoke (live): A/B/C kitchen-sink incl. console history/status of remotely-spawned B21.
SDK lanes: TS/Py wrapper tests in the sdk jobs; web wasm-contract covers the placement typed-reject; sdk-web suite re-runs closed-key parser tests unchanged (proof the touched DTOs avoid requireOnlyKeys shapes).
17.12 Surface gotchas (append to §13)
verify_sdk_wrapper_freshnessrequires the method literal in hand-written SDK source — generated mirrors do not count (verify_rpc_surface_alignment.py:20-24). Budget real wrappers, not codegen output.- REST catalog entries and
router()arms must land in the same change (phantom-404 precedent, rest_catalog.rs:174-181). - MCP has no external alignment gate — the roster tests catch dispatch drift, but
docs/api/mcp.mdxdrift is caught by nobody; it is a named phase-7 work item. external_member: Option<Value>onMobMemberStatusResultis a folklore trap — placement/reachability go in the typed fields only (SD-5 pin test).- The web SDK’s
requireOnlyKeysshapes are breaking-on-extend for released browsers — check every touched DTO against sdks/web/src/mob.ts:466-548 before adding fields.
18. Orchestration-adjacent systems: flows, workgraph, schedules, approvals, tool policy, budget, taint
Scope: how every orchestration-adjacent subsystem behaves when a mob spans hosts. Builds on D1–D6 (settled). Decisions in this section are numbered O1–O8. Every “today” claim is anchored at HEAD (0.7.22 / da467fca8).
18.1 Flows across remote members (O1, O2)
Substrate. A flow step dispatches throughFlowEngine::execute_single_target_with_retries → FlowTurnExecutor::dispatch and awaits a per-dispatch terminal (meerkat-mob/src/runtime/flow.rs:840-881). ActorFlowTurnExecutor::dispatch branches on runtime mode: AutonomousHost uses the in-process interaction_event_injector.inject_with_subscription (meerkat-mob/src/runtime/actor_turn_executor.rs:281-316); TurnDriven calls provisioner.start_flow_step with a local event_tx mpsc channel in StartTurnRequest (:334-365). Completion is a per-dispatch subscription classifying terminal AgentEvents — RunCompleted / ExtractionSucceeded / ExtractionFailed / InteractionComplete / InteractionCallbackPending / RunFailed / InteractionFailed; channel close without a terminal is a typed step failure (:98-217, close at :211-215). Steps target roles and carry allowed_tools/blocked_tools (meerkat-mob/src/definition.rs:393, :411-414) lowered into a TurnToolOverlay (flow.rs:1919-1930; type at meerkat-core/src/service/mod.rs:1379); the overlay is typed-rejected for AutonomousHost members because the injector seam cannot carry it (actor_turn_executor.rs:275-279; inject_with_subscription has no overlay parameter — meerkat-core/src/event_injector.rs:80-86). Remote members have no working step path today: the provisioner trait default forwards start_flow_step → start_turn (meerkat-mob/src/runtime/provisioner.rs:277-286), and the peer-only start_turn rejects any event_tx with UnsupportedForMode (“tracked turn event streams are not supported for peer-only members in phase 1”, :3328-3333) — pinned as a whole-run failure discovered only after the run starts (meerkat-mob/src/runtime/tests.rs:23945-24018). This section supersedes that failure shape. Frame loops (repeat_until) and hard-timeout disposition stay controlling-host-local and machine-owned (ClassifyTurnTimeoutDisposition, catalog mob_machine.rs:900; shell realization actor_turn_executor.rs:409-457) — no frame-engine distribution.
O1 — Remote flow steps ride the one delivery command, upgraded with a typed turn directive; dispatch disposition is machine-classified before delivery.
WHY: the lead constraint is one member-delivery path, and DeliverMemberInput already carries input_id, content, handling mode, and injected context (provisioner.rs:3366-3390); minting a second “flow turn” command would fork admission, dedup, and epoch handling.
BridgeDeliveryPayloadgains one additive optional field:turn: Option<BridgeTurnDirective { correlation: { run_id, step_id }, tool_overlay: Option<WireTurnToolOverlay> }>(absent-omitted, theinjected_contextbyte-compat precedent,meerkat-contracts/src/wire/supervisor_bridge.rs:852-859). Silent overlay drop is structurally impossible: the payload isdeny_unknown_fields(:844), so a pre-V4 receiver rejects a directive-bearing delivery at decode — a typedBridgeCommandRejectedstep failure, never a turn that runs ungated. Directive-bearing deliveries require protocol V4.- Member-side, a directive-bearing delivery is admitted as a tracked turn:
TurnDrivenmembers admit through the host’s session turn admission carryingStartTurnRuntimeSemantics { handling_mode, tool_overlay }and a host-localevent_tx— the local TurnDriven mechanics relocated to the host that runs the loop;AutonomousHostmembers admit viainject_with_subscription. Overlay +AutonomousHoststays typed-rejected regardless of placement (same seam limitation on every host). - Dispatch disposition becomes machine-owned: new MobMachine input
ClassifyFlowStepDispatch { run_id, step_id, target, overlay_present }→ effectFlowStepDispatchClassified { dispatch: Local | RemoteTurnDirective | RejectedOverlayAutonomous | RejectedHostIncapable }, adjudicated against machine-owned facts ONLY: the guard readsself.member_runtime_modes[target](machine state written atCommitSpawnMembership),self.member_placement, andself.host_capabilities— runtime mode is NEVER a shell-supplied input (a roster-cache claim would be shadow truth on the exact seam this decision promotes to machine ownership);overlay_presentis the one legitimately shell-observed request fact (theClassifyTurnTimeoutDispositionshape). The executor consults it before any delivery, so an unsupported combination fails the step typed at dispatch instead of surfacing as today’s mid-runUnsupportedForMode. A run-open whole-flow sweep is deliberately rejected: role→member resolution happens at dispatch and membership mutates mid-run — a sweep would fabricate certainty; per-dispatch classification is the honest gate.
EventEnvelope gains no field.
WHY: EventEnvelope carries no input/turn correlation (meerkat-core/src/event.rs:94-103) and the envelope/page rows are frozen; watermark-window attribution is unsound under queued concurrent inputs and idempotent redelivery (BridgeDeliveryOutcome::Deduplicated, supervisor_bridge.rs:866-875). Only the terminal needs attribution — display events don’t.
- At directive admission, the member host atomically creates the per-turn subscription (the remote twin of the admission-time atomicity at
actor_turn_executor.rs:299-316/:334-365— today’s receiver discards the completion handle,meerkat-runtime/src/comms_drain.rs:2701) and, at terminal, records(agent_identity, generation, input_id) → RecordedTurnOutcome { terminal_seq, outcome }durably inMobHostBindingAuthority(§6.3). BridgeReply::MemberEventsPage(§7.4) gains a boundedturn_outcomes: Vec<BridgeTurnOutcomeRecord { input_id, generation, terminal_seq, outcome }>page plusoutcomes_complete;PollMemberEventscarries the exact prior-page(generation, input_id)acknowledgements and an independentmax_outcomes.terminal_seqis pinned to the durableStoredEvent.seqdomain (never the session-task or per-stream counters, gotcha 8). The host prunes only acknowledged durable rows; unknown/already-pruned acknowledgements are no-ops and create no negative memory. Lost replies replay unacknowledged rows, while a delayed journal commit remains visible on a later page. Both one-record admission and the combined encoded reply are byte-bounded below transportMAX_PAYLOAD_SIZE: each failed terminal carriesdetail: { text, original_utf8_bytes, truncated }, and the member host retains the largest UTF-8 prefix that keeps the complete journal row at or below 64 KiB beforerecord_turn_outcome. Pathological input-id framing that cannot leave room even for an empty detail is rejected before runtime acceptance; an accepted turn is never rejected after its effect because its provider detail was large. The terminal discriminant, exact acknowledgement key, and original UTF-8 byte length remain unchanged/typed. Retention is also hard-bounded while a controller is offline: each member admits at most 256 retained-plus-in-flight directed outcomes (at most 64 KiB each). The 257th directive rejects before runtime acceptance with typedOutcomeJournalFull { retained, limit }; no unacknowledged completion authority is evicted. Hosts withoutdurable_sessionsnever receive directives at all (RejectedHostIncapableat dispatch classification), so the field is always well-defined. The controlling host’s poll pump (one loop, one command — no second path) feeds outcome records to the remote flow ticket registry; unattributed event rows keep flowing to the merged stream for display. If an immutable terminal event row itself exceeds the bridge page budget, the pump advances that exact typed omission and resolves failed turns from the bounded sidecar instead of waiting for the ordinary turn timeout.- Pump lifecycle is decoupled from observation (A17): dispatching a directive-bearing delivery records a MobMachine obligation
pending_remote_turn_outcomes: Set<(AgentIdentity, InputId)>(recorded atFlowStepDispatchClassified{RemoteTurnDirective}realization, resolved when the pump consumes the outcome record or the timeout ladder disposes the step). The pump for a member runs while EITHER an event subscription is authorized OR obligations are outstanding — a flow step never depends on a console watching (the Rule-8 dropped-observation shape is unrepresentable). - Terminal classification is extracted into a single shared
meerkat-coreturn-terminality classifier consumed by both the local executor bridge (actor_turn_executor.rs:98-217) and the member-host journal writer. WHY: two hand-maintained terminal-event lists are exactly the split-terminality failure shape; one authority, zero divergence window. - Semantics: keyed on
(generation, input_id)— redelivery converges on the one journal row; a generation bump (rematerialize) scopes the journal, so an old generation’s outcome can never attribute to a new turn and the step fails typed on generation change. Member-host restart mid-turn: the turn is dead, no record ever appears, and the step resolves through the existing timeout ladder (machine-owned disposition) — typed, honest. A directive-bearing delivery the member runtime cannot host as a tracked turn is rejected at admission with new causeTurnDirectiveUnsupported(extends#[non_exhaustive]BridgeDeliveryRejectionCause,supervisor_bridge.rs:885-896).
18.2 Tool access policy: Inherit locality and the sealed wire form (O3)
Substrate.ToolExecutionPolicy::resolve fails closed on Inherit (meerkat-core/src/tool_execution_policy.rs:92-102, UnresolvedInherit :33-45, sealed private ctor :57-75). The factory resolves before persisting metadata and installs ExecutionPolicyGatedDispatcher as the outermost gate (meerkat/src/factory.rs:5421-5432, persist :5461-5463, gate :5818-5822) — so a persisted effective policy is never Inherit. The agent-facing spawn seam resolves Inherit to the parent’s persisted policy via the local session service (meerkat-mob-mcp/src/agent_tools.rs:86-94, :657-697; local read :671-680; metadata read fault fails the spawn closed :681-691). The trap: the mob compiler maps residual Inherit/None to None = unrestricted (meerkat-mob/src/build.rs:332-341), and this path never reaches the factory’s UnresolvedInherit gate — so any wire shape that can carry Inherit resolves to unrestricted on the compiling host. Explicit policy presence is a MobMachine-privileged admission fact requiring manage scope (catalog/dsl/mob_machine.rs:1996-2039), computed from the requesting args at the tool seam (meerkat-mob/src/runtime/tools.rs:585-586, handle.rs:398-401, mapped at handle.rs:6550-6564).
O3 — Inherit resolves in the process that owns the parent session’s realm; the materialize wire carries a resolved-only sealed type; admission facts and resolved policy travel as separate fields.
WHY: the parent’s persisted metadata is realm-local (§3: realms are not distributed) — for B2-spawns-B21 the parent’s realm is on Host B, so a controlling-host resolution would need a cross-host metadata read in the spawn critical path, racing parent policy changes and duplicating the D2 read plane into the control plane. The existing seam already resolves locally and fail-closed; keep it, relocate nothing.
MaterializeMember.spec.tool_access_policy: Option<WireResolvedToolAccessPolicy>whereWireResolvedToolAccessPolicy = AllowList(names) | DenyList(names)—Inheritis unrepresentable at decode;None= unrestricted (host authority). Decode-time rejection is the only sound gate givenbuild.rs:332-341; the member-host factory’sUnresolvedInheritseal (factory.rs:5421-5432) remains the backstop, and the member host persists the received policy into its ownSessionMetadata.tooling.tool_access_policyso B21-spawns-B211 inherits correctly on Host B with zero controlling-host involvement.- Member-host-originated spawn requests ride the ONE member-upcall lane —
MemberOperatorRequest(§15, exact-generation/fence machine admission plus durable request-id ledger; A8: there is exactly one member→controlling command family, and this is it) — carrying two separate facts:requested_tool_access_policy_present: bool(the admission fact computed at the tool seam from the agent’s args and checked through the re-minted generated agent authority) andresolved_tool_access_policy: Option<WireResolvedToolAccessPolicy>(the resolution output). PrincipalControlScopegrants are not consulted on this lane (§15.2). WHY: resolvingInheriton Host B yields an explicit policy the agent never requested; deriving the privileged-presence fact from the resolved spec would launder an inherited policy into a stronger authority demand (or worse, the inverse). Two facts, two owners, no laundering. - Containment invariant (tested both directions): a restricted parent on Host B spawning onto Host B or Host A yields a child whose sealed gate equals the parent’s effective policy; no host pair produces an
Inherit-derived unrestricted child.
18.3 Budget (O4)
Substrate. Onlybudget_limits is real end-to-end: spec → CreateSessionRequest.build.budget_limits (meerkat-mob/src/runtime/actor.rs:793-803) → per-session Budget enforcement in the agent loop (meerkat-core/src/budget.rs:12-20). budget_split_policy (meerkat-mob/src/launch.rs:63-77; spec field handle.rs:1874; wire meerkat-contracts/src/wire/mob.rs:713) is functionally unconsumed: every actor spawn handler underscore-discards it (actor.rs:8869, :9783, :14430) and no split computation exists anywhere at HEAD (NOT FOUND workspace-wide outside a wire round-trip test). §7.3’s sentence “the spawn’s budget split is computed on the controlling host and seeded in the materialize payload” overstates HEAD and is corrected here (see the §7.3 delta below).
O4 — PortableSpawnOverlay.budget_limits is the single budget carrier; BudgetSplitPolicy is deleted.
WHY single-carrier pass-through: the overlay is digest-covered and survives member-host revival, while enforcement is loop-internal per-session state and therefore naturally member-host-local. A payload-level sibling would create two authorities for one budget fact. Cross-host aggregate budget is already a §12 non-goal; implementing split would require a new seam reading a live session’s remaining budget (core Budget is loop-internal atomic state, not service-readable) — scope this plan explicitly defers. WHY delete: an accepted-then-discarded budget instruction is a fail-quiet containment gap — an agent that sets Fixed(1000) believes containment exists and gets nothing. Pre-1.0 dogma: no dead vocabulary, no shims. BudgetSplitPolicy (launch.rs:63-77), SpawnMemberSpec.budget_split_policy (+ builder, handle.rs:1874/:2032-2033), the tool-schema/args plumbing (tools.rs:378, :586, :903-904, :970-971), the RPC param (meerkat-rpc/src/handlers/mob.rs:436-438, :502-503), WireBudgetSplitPolicy (wire/mob.rs:713, emit.rs:1392), and the machine admission fact privileged_budget_split_policy_present (catalog dsl mob_machine.rs:1997/:2018/:2031) are all removed in one change-set. If split computation ever lands (v2 seed), it returns as controlling-host-only planning that computes BudgetLimits — policy never crosses the wire even then.
- §7.3 sentence replacement: “Budget: the spawn’s
budget_limitsridePortableSpawnOverlay.budget_limitsinside the digest-covered spec; the member host applies them toCreateSessionRequest.build.budget_limits(the shippedwith_spawn_budget_limitspath, actor.rs:793-803) and enforcement is local to the member’s session (aggregate mob budget is a non-goal, §12). Budget-split vocabulary and payload-level budget siblings do not exist on the wire.”
18.4 WorkGraph (O5)
Substrate. The workgraph store is a host-local SQLite file keyed by a realm-id string: the factory (absent a supplied dispatcher) fails closed without a typedRealmId, then opens realm_scope_root/workgraph.sqlite3 with WorkGraphService::with_scope(store, realm_id, namespace) (meerkat/src/factory.rs:4907-4913, :4919-4935; with_scope meerkat-workgraph/src/service.rs:39-49; reads keyed (realm_id, namespace) meerkat-workgraph/src/store.rs:91-96; realm bundle row meerkat/src/persistence.rs:362-364). Mob members build with realm_id = mob.{id} (meerkat-mob/src/build.rs:24-34, :198, :207), workgraph tools per profile (:243-244), and the workgraph-workflow skill preloaded (:224-229) — which tells members they share one task list. The hazard: a member materialized on Host B under the same mob.{id} realm string opens Host B’s workgraph.sqlite3 — the shared task list silently splits into disjoint stores while the skill keeps asserting unity. Fail-quiet split-brain.
O5 — Cross-host shared workgraph is a typed v1 non-goal, made fail-closed at spawn admission: remote placement of a workgraph-tooled profile is machine-denied.
WHY: bridging the store means a new bridge command family with CAS/ordering semantics over the network (update_item_cas, store.rs:75-80) for every tool call, cutting against §3’s realm-locality constraint and widening phase scope for a coordination pattern that degrades acceptably (operators split roles or disable workgraph on remote profiles). The settled alternative is not “silent divergence” — it is a machine reject.
- The denial is guarded INSIDE the machine spawn-exec ladder at the
BeginSpawnExecopener — the one choke-point every spawn path crosses (A4; the tool-seam-onlyResolveSpawnMemberAdmissiondispatch is bypassed today by RPCmob/spawn*and embedderMobHandle::spawn_spec, meerkat-mob-mcp/src/lib.rs:978-984, so guarding only there would leave the split-brain reachable). Input factworkgraph_required: boolis the EXPLICIT resolvedprofile.tools.workgraphassertion captured before inherited category opening. Explicit workgraph emits the mergedSpawnMemberAdmissionResolved { admission: NonPortableResource { kind: WorkgraphTools } }; a remote spawn carrying the non-serializable inherited tool-visibility authority is hard-denied first asNonPortableInheritedToolFilter. No skill suppression logic is needed because neither rejected combination builds, andnon_portable_disabledremains a typed empty record in v1. Upcall spawns are unaffected becauseMemberOperatorSpawnSpeccarries no inherited-filter field. - Degradation is operator-explicit: place the member locally, or set
tools.workgraph = falseon the remote profile (the member then preloadstask-workflowperbuild.rs:227-229— session-local builtin tasks, no cross-host semantics). v2 seed: a bridge-proxied workgraph service (controlling-host store as the single owner), designed only if a machine decision ever depends on shared work state.
18.5 Schedules (O6)
Substrate.TargetBinding = Session | Identity | Mob | HostRunnable (meerkat-schedule/src/types.rs:1147-1152); MobTargetBinding covers Member/Flow/SpawnHelper/ForkHelper (:1397-1425). The schedule driver is a per-process loop (spawn_schedule_host, meerkat/src/surface/schedule_host.rs:943-983); surfaces wire NoopScheduleMobHost by default — mob targets probe Missing and deliveries fail typed MobRejected (:290-341) — and only mob-binding surfaces attach MobMcpScheduleHost, which delivers Member via runtime.member_send and Flow via run_flow, with completion polled from flow_status (meerkat-mob-mcp/src/schedule_host.rs:263-313, completion future :552-562; CLI production wiring meerkat-cli/src/main.rs:10517). Schedule stores are realm-local (meerkat/src/persistence.rs:359-361). HostRunnable targets dispatch through a process-local registry attached via with_runnable_host; no registry ⇒ typed TargetMissing (meerkat-schedule/src/runnable.rs:108-117 trait, :126-129 registry; schedule_host.rs:364-371, :665-700). Scheduled spawn/fork helpers capture the creating session’s effective tool access policy at schedule creation because they later fire under host authority (meerkat/src/factory.rs:4848-4875).
O6 — Mob-target schedules exist only on the controlling host; member hosts run realm-local drivers with the Noop mob host; HostRunnable never crosses hosts; scheduled deliveries to remote members inherit phases 3–6 verbs with zero schedule-specific bridge work. (Adjudication A3-final: tools.schedule IS permitted on remote members — session-target schedules are realm-local facts that fire correctly on the owning host, and the one cross-host shape, a mob-target schedule created member-side, fails TYPED at fire as pinned below. The earlier blanket NonPortableResource::ScheduleTools reject is superseded; only workgraph stays admission-denied because its failure shape is silent split-brain with no typed fire-time backstop.)
WHY: MobMcpScheduleHost binds the local mob runtime — the single MobMachine owner (D6); a second mob scheduler on a member host would be a second command authority. Scheduled Member/Flow delivery goes through identity-addressed runtime verbs (member_send, run_flow), so remote-member reachability for schedules is exactly as good as those verbs get in phases 3–6; scheduled-flow completion polls flow_status, which is controlling-host-local flow-run state — already correct.
rkat mob host(D6, phase 2) does spawn a schedule host: realm-local stores, real session-target delivery (member sessions may carrytools.schedule,build.rs:245-246, and their session-target schedules must fire on the host owning the session realm),NoopScheduleMobHost, and no runnable registry in v1. A mob-target schedule created on a member host fails typed at fire (probeMissing→MarkMisfiredper theMissingTargetPolicydefault, types.rs:1138-1142; deliveryMobRejected) — pinned as correct behavior, not fixed.- HostRunnable host-locality is a typed invariant: the registry is process state;
host_runnabletargets never route across hosts; absent registry keeps failingTargetMissing. Pinned by test. - Scheduled-helper containment: the captured creator policy is already resolved (the factory refuses to build with unresolved
Inheritand persists resolved-only metadata — factory.rs:5421-5432/:5461-5463), so when a scheduled SpawnHelper/ForkHelper resolves to remote placement in phases 3+, the captured policy ridesMaterializeMemberasWireResolvedToolAccessPolicyunchanged — the same sealed-resolved rule as every other spawn (O3). No schedule-specific policy path.
18.6 Approvals (O7)
Substrate. Approvals surface only on the RPC plane (approval/request|list|get|decide, meerkat-rpc/src/router.rs:1878-1889) and persist per-RPC-runtime at store_path/approvals.json (meerkat-rpc/src/session_runtime.rs:1402-1435); REST advertises approvals: false (meerkat-rest/src/lib.rs:3328). The lifecycle authority is the catalog ApprovalLifecycleMachine generated into core (meerkat-machine-schema/src/catalog/dsl/approval_lifecycle.rs:32+), owned per-process by ApprovalService. There is no bridge approval vocabulary (NOT FOUND: zero approval hits in meerkat-contracts/src/wire/supervisor_bridge.rs and meerkat-mob/src/runtime/bridge_protocol.rs), no in-loop producer, and the mob-side forwarding hooks are Declared-only (“this slice does not execute forwarding”, meerkat-mob/src/runtime/handle.rs:867-893). §8’s ControlScope has no Approve verb.
O7 — Approvals stay host-local in v1; the gap is a bind-declared capability, never silence.
WHY: the lead’s conditional resolves against proxying — approvals are not event-carried, so they cannot ride the D5 pump; proxying would need a new command family, an Approve ControlScope verb, and a single-host decision-authority fencing story (the generated lifecycle authority is per-process — split-brain on decide otherwise), all for a surface with no in-loop producer at HEAD.
- v1 semantics: an approval record lives on the host whose RPC runtime created it.
HostCapabilityFlags(§6.1) gainsapproval_forwarding: bool, advertisedfalseby v1 member hosts at bind and projected to the console so remote members’ approval state is labeled unavailable, mirroring the D5durable_sessionsdegradation pattern — typed capability, not silent difference. - v2 seed (recorded so it is not re-invented):
ApprovalOwnerRef::ExternalMember { mob_id, member_ref }(meerkat-core/src/approval.rs:169-172) + the Declared forwarding hooks are the reserved vocabulary; pulling it forward requires addingApproveto §8’s enum, bridgelist/decideproxy commands,BRIDGE_CLASSIFIER_FILESentries, and placing decision authority on exactly the controlling host.
18.7 Outbound content taint (O8)
Substrate.BridgeCommand::DeclareMemberOutboundTaint is fully shipped (meerkat-contracts/src/wire/supervisor_bridge.rs:250, payload :284, additive-V3 note :163): the actor installs on a local member’s comms runtime directly or relays over the bridge for external-bound members (meerkat-mob/src/runtime/actor.rs:12043-12061 local, :12063-12087 relay); the member-side receiver validates supervisor authority then calls set_outbound_content_taint fail-closed — an unsupported runtime rejects rather than drops (meerkat-runtime/src/comms_drain.rs:3268-3311). The declaration rides inside the signed envelope region (meerkat-comms/src/types.rs:54-61) and does not survive respawn (meerkat-mob/src/runtime/handle.rs:4838-4841). No RPC/MCP surface exposes it — embedder-facing MobHandle/MobCommand only (NOT FOUND in meerkat-rpc/src, meerkat-mob-mcp/src).
O8 — Taint generalizes over D1 with one branch extension and stays member-addressed; re-declaration remains embedder-owned; the mob caches no declaration.
- A host-materialized member is the bridge case of the existing local-vs-bridge branch:
provisioner_commsreturns nothing (the member’s comms runtime lives on Host B), so the relay path applies — extendruntime_binding_for_entry/peer-spec derivation (actor.rs:12065-12072) to the newHostMaterialized { host }binding (peer = member pubkey @ host acceptor address, D1 demux). Admission classification is pinned member-addressed (member-side MeerkatMachine/comms-drain admission, exactly the shipped receiver) — never host-addressed; the taint fact is per-member. - Ownership: the embedding application at the controlling host owns the outbound-taint fact for all mob members regardless of placement and is the only declarer; member hosts never self-declare, and the member host’s runtime keeps owning its own session-content bookkeeping as today. Lapse-on-rematerialize is the shipped, documented semantic (fresh context ⇒ no declaration, handle.rs:4838-4841) and stays symmetric across hosts: the embedder re-declares keyed on the member lifecycle signal (
MemberMaterializedat the new generation, §7.3). The mob deliberately keeps no declaration cache to replay — that would be shell shadow state over an embedder-owned fact. Covered by an e2e-fast re-declaration test (§18.11).
18.8 Failure semantics (additions to §9)
18.9 Machine deltas (folds into §6; catalog first, dispositions and TLC bounds per §6 preamble)
- MobMachine —
ClassifyFlowStepDispatch { run_id, step_id, target, overlay_present }input →FlowStepDispatchClassified { run_id, step_id, target, dispatch: Enum<FlowStepDispatchKind> }effect (Local | RemoteTurnDirective | RejectedOverlayAutonomous | RejectedHostIncapable), guarded onmember_runtime_modes+member_placement+host_capabilities(all machine facts;RejectedHostIncapablecovers adurable_sessions=falsehost — tracked turns need the durable outcome journal); dispositionlocal seam SurfaceResultAlignment(theFlowFrameTerminalStatusClassifiedprecedent, catalog mob_machine.rs:1412). - MobMachine —
ResolveSpawnMemberAdmissiongainsworkgraph_required: bool+placement_remote: boolinput facts and a Denied transition guardedplacement_remote && workgraph_required, emittingMobSpawnMemberAdmissionKind::NonPortableResource { kind: WorkgraphTools }(merged vocabulary, A4), evaluated on the spawn-exec ladder choke-point. - MobMachine —
privileged_budget_split_policy_presentdeleted from theResolveSpawnMemberAdmissioninput and both privileged-args guards (catalog mob_machine.rs:1997/:2018/:2031) as part of the O4 vocabulary removal. - §6.1 field extension —
HostCapabilityFlagsgainsapproval_forwarding: bool(v1 member hosts advertisefalse). - MobHostBindingAuthority (§6.3) — new facts:
turn_outcomes: Map<(AgentIdentity, Generation, InputId), RecordedTurnOutcome { terminal_seq, outcome }>withRecordTurnOutcomeand exactAcknowledgeTurnOutcomeinputs. Redelivery never creates a second row; acknowledgement removes only an existing exact row and an absent acknowledgement retains no tombstone. Acknowledged rows are pruned immediately, with remaining generation-scoped retention also pruned atReleaseMember/generation retirement; admission classTurnDirectiveUnsupportedadded to the §6.3 vocabulary. Persisted in the same host-realm SQLite table family. - No Schedule/Occurrence machine changes; no ApprovalLifecycle changes; no new ControlScope verbs (explicitly:
Approveis NOT added in v1).
18.10 Wire deltas (folds into §7; V4; regen-schemas + parity gates per §10 preamble)
BridgeDeliveryPayload+turn: Option<BridgeTurnDirective { correlation: BridgeTurnCorrelation { run_id, step_id }, tool_overlay: Option<WireTurnToolOverlay> }>(absent-omitted; fail-closed on old receivers via existingdeny_unknown_fields).BridgeDeliveryRejectionCause+TurnDirectiveUnsupported { detail }.PollMemberEventsexactoutcome_acks+ independentmax_outcomes;BridgeReply::MemberEventsPageboundedturn_outcomes: Vec<BridgeTurnOutcomeRecord { input_id, generation, terminal_seq, outcome: WireFlowTurnOutcome }>+outcomes_completesidecar metadata (EventEnvelopeand event page rows unchanged — restating the §7 envelope freeze).MaterializeMember.spec.tool_access_policy: Option<WireResolvedToolAccessPolicy { AllowList | DenyList }>—Inheritunrepresentable;MaterializeMember.spec.overlay.budget_limits: Option<BudgetLimits>is the sole digest-covered budget carrier.- Cross-host spawn request carries
requested_tool_access_policy_present: boolseparately fromresolved_tool_access_policy(O3 no-laundering rule). BridgeCapabilities/HostCapabilityFlags+approval_forwarding: bool.- Removals:
WireBudgetSplitPolicy(wire/mob.rs:713, emit.rs:1392), RPCbudget_split_policyparam (handlers/mob.rs:436-438) — clean break, no tombstones.
18.11 Phase and test additions (folds into §10/§11)
Phase assignments (no new phase; the flow work is the completion-consumption line item §10.6 already reserves, made concrete):- Phase 1: all §18.9 catalog deltas + §18.10 contracts deltas + the §7.3 budget-sentence correction; TLC bounds unchanged (2 hosts × 3 members) now covering flow-dispatch classification and the workgraph-placement denial.
- Phase 2:
rkat mob hostspawns the schedule host (realm-local stores, Noop mob host, no runnable registry). - Phase 3: sealed policy + budget seed in materialize (member host persists received policy and applies budget seed via the
with_spawn_budget_limitsshape); presence-fact/resolved-policy split on member-host-originated spawns; workgraph placement gate live; BudgetSplitPolicy code removal (launch.rs, spec/builder, tools, RPC); taint relay branch extended toHostMaterialized. - Phase 5: no deltas from this dimension (recorded: no
Approvescope). - Phase 6: remote flow dispatch end-to-end — directive send, member-side tracked-turn admission + shared terminal classifier + durable journal, pump sidecar consumption, remote flow ticket,
ClassifyFlowStepDispatchwiring, deletion of the shell-side autonomous-overlay reject (actor_turn_executor.rs:275-279) in favor of the machine classification; supersede the pinned peer-only failure test (tests.rs:23945-24018) with the new typed shapes;BRIDGE_CLASSIFIER_FILES(xtask/src/bridge_classifier.rs:27-31) extended with every newBridgeReplyconsumer (journal/pump/flow-executor files). (v4.1.2 errata, ADJ-P6-13: the shell-side overlay reject was deleted in phase 6 as planned — the guards moved into the machine (mob_machine.rsClassifyFlowStepDispatch), so theactor_turn_executor.rs:275-279anchor is superseded.) - Phase 7: console
approval_forwardinglabeling; docs.
- unit:
WireResolvedToolAccessPolicydecode rejectsinherit(serde, fail-closed); shared terminal classifier covers all seven terminal events + channel-close; journal dedup/generation-scoping;ClassifyFlowStepDispatchkernel matrix; workgraph-denial guard; TLC green at 2×3 with the new inputs. - int / e2e-fast (two-hosts-in-one-process harness): flow run with a remote TurnDriven member completes via journal+sidecar, overlay enforced member-side (denied tool ⇒
access_deniedinside the remote turn); remote AutonomousHost step without overlay completes; overlay+autonomous rejected at dispatch classification (local and remote, same cause); attribution correct under interleaved queued inputs (peer send racing a flow step);Deduplicatedredelivery converges on one outcome; generation bump mid-step fails typed; pre-V4 directive decode-reject shape; restricted parent on Host B spawns onto Host A and Host B — both children contained, wire never carriesInherit; budget seed enforced member-side (max_tool_calls⇒ typed budget stop); workgraph remote placement denied / local allowed /tools.workgraph=falseremote allowed withtask-workflowpreload; scheduledmember_sendandrun_flowto a remote member deliver via runtime verbs, scheduled-flow completion viaflow_status; mob-target schedule on member host fails typed (Noop pin);host_runnableabsent-registryTargetMissingpin; taint re-declaration after remote respawn (declare → rematerialize → verify lapse → re-declare → envelope carries claim); console labelsapproval_forwarding=false. - e2e-system (real multi-process TCP): flow run across
rkat mob hostwith member-host restart mid-step ⇒ typed step timeout, run ledger honest; member-host schedule driver fires a session-target schedule from the host’s realm store. - e2e-smoke (live): the §11 kitchen-sink A/B/C run includes one cross-host flow with an overlay-bearing step.
18.12 Non-goals additions (folds into §12) and gotchas (folds into §13)
Non-goals (each with typed degradation, never silence): cross-host shared workgraph (remote placement of workgraph-tooled profiles is machine-denied at spawn admission; degrade by local placement ortools.workgraph=false); approval forwarding (approvals stay host-local; approval_forwarding=false declared at bind, console-labeled; v2 seed = ApprovalOwnerRef::ExternalMember + Declared hooks + Approve scope); budget split computation (vocabulary deleted; budget_limits is the only budget primitive, enforced member-locally; aggregate mob budget stays a non-goal); mob-target scheduling on member hosts (Noop host, typed Missing/MobRejected, pinned); cross-host host_runnable dispatch (never; typed TargetMissing pinned).
Gotchas: 11. BridgeDeliveryPayload is deny_unknown_fields (supervisor_bridge.rs:844) — the turn directive is fail-closed against old hosts by construction; do not “help” by tolerating unknown fields. 12. inject_with_subscription has no overlay parameter (event_injector.rs:80-86) — autonomous members cannot enforce overlays on any host; keep the typed reject, do not smuggle overlays through injection. 13. Workgraph realm scope is a string keyed into a host-local sqlite (store.rs:91-96; factory.rs:4919-4935) — the same mob.{id} on two hosts is two disjoint stores; the spawn-admission gate is the only thing standing between you and silent split-brain. 14. Only surfaces binding MobMcpState get real mob schedule delivery; NoopScheduleMobHost on member hosts is correct, not a bug — do not wire a second mob scheduler. 15. Terminal classification must have exactly one owner (the shared core classifier) — a second terminal-event list on the member host is the split-terminality bug class by construction.
19. Member session lifecycle across hosts: launch modes, disposal, transcript edits, compaction, memory, claims
Scope rule for this whole section: everything that executes inside the member’s agent loop or its session service runs on the host that owns the session’s realm (§3: realms are not distributed). Compaction, memory indexing, durable archive adjudication, transcript revisions, and session-identity claims are all realm-local by construction — verified below. The only new cross-host machinery this dimension needs is (a)launch_mode semantics inside MaterializeMember, (b) a truthful durable-disposal contract for ReleaseMember, and (c) one typed field on an existing MobMachine signal so the machine stops recording archives that never happened.
Substrate at HEAD, verified:
MemberLaunchMode=Fresh | Resume { bridge_session_id } | Fork { source_member_id, fork_context };ForkContext=FullHistory | LastMessages { count }(meerkat-mob/src/launch.rs:20-35,49-61). Fork is notSession::fork()copy-on-write: the source history is read via the controlling host’s local session service (SessionServiceHistoryExt::read_history, actor.rs:9179-9191), rendered to text (render_fork_context, actor.rs:911), and prepended to the child’s initial prompt as a plainContentBlock::Textbefore provisioning (actor.rs:9240-9248). The child gets a fresh session.- The fork source resolve is local-only: roster →
MemberRef::bridge_session_id()(meerkat-mob/src/event.rs:114-119); a source without a session yields untypedMobError::Internal("fork source ... has no session")(actor.rs:9146-9150);LastMessagesreads localview.state.message_count(actor.rs:9158-9176). Resumevalidation is entirely realm-local:is_member_activefast-path (actor.rs:8978-8987),AutonomousHostrequires a local interaction-event injector (8988-8997), comms-runtime presence when wiring exists (8999-9012), inactive fallback =load_persisted_session+build_resumed_agent_config(9038-9074); the persistent-unsupported/missing-snapshot exits are untypedInternaltoday (9044-9048, 9128-9130).- Local member retire archives durably and loudly: disposal pipeline
StopHostLoop → NotifyPeers → ArchiveSession(meerkat-mob/src/runtime/disposal.rs:28-32),ArchiveSessionfailure is critical (actor.rs:14257-14275), realized asSessionBackend::retire_member → archive_with_authority_then_unregister → archive_with_mob_lifecycle_authority(provisioner.rs:2185-2195, 857-936) with a deliberate carve-out: adopted host-owned sessions (aResumeover a store the mob does not own) get runtime-retire + binding release only (provisioner.rs:868-897).dispose_archive_sessiontoleratesNotFoundas success (actor.rs:15697-15716). - Remote (peer-only) retire today is runtime-retire only: controlling side sends
BridgeCommand::RetireMemberthenmark_member_retired(provisioner.rs:3197-3248); the receiver executesadapter.retire_runtime(session_id)— no SessionDocumentMachine archive (meerkat-runtime/src/comms_drain.rs:2788-2825);DestroyMemberlikewise maps toRuntimeControlPlane::destroy(comms_drain.rs:2827-2864). - And yet the machine records an archive either way:
ObserveMemberRetirementArchivedis signalled whenever theArchiveSessionstep “completed” (actor.rs:15285-15289, 15725-15737) — which is true for the host-owned carve-out and for the remote runtime-only retire. The archived fact is a fiction in both cases. - Archive truth is owned by the canonical SessionDocumentMachine:
Archivedis the absorbing terminal, re-archive is the explicitAlreadyArchivedverdict (meerkat-machine-schema/src/catalog/dsl/session_document.rs:343-383), realized fail-closed durable-document-commit-FIRST / runtime-retire-SECOND byMachineSessionArchiveProtocol(meerkat-session/src/persistent.rs:1521-1600) and read back viasession_archived_by_authority/reject_if_archived_session(persistent.rs:1608-1650). All per-session, realm-local. - Transcript edits (
session/fork_at,fork_replace,rewrite_transcript,transcript_revision,transcript_revisions,restore_transcript_revision) are RPC-only (meerkat-contracts/src/rpc_catalog.rs:135-160; REST deliberately dropped them — rest_catalog.rs:174-181), implemented only byPersistentSessionService(persistent.rs:5652-5655; the core trait defaults are typedUnsupported,meerkat-core/src/service/mod.rs:2016-2062), and admitted runtime-locally (staged-session check +resolve_transcript_edit_admissionon the localMeerkatMachine,meerkat-rpc/src/session_runtime.rs:7563-7614). - Compaction runs inside the agent loop of whichever host runs the session (
CheckCompactioneffect →run_compaction,meerkat-core/src/agent/state.rs:1233-1265);DefaultCompactor+ optional curator are wired atAgentFactorybuild time (meerkat/src/factory.rs:5700-5716);CompactionCuratoris an in-process trait with no wire form (meerkat-core/src/compact.rs:175-192). Compaction discards index intoMemoryIndexScope::for_sessionin the same host’s realm (agent/state.rs:1365-1373). MemoryStore::drop_scope/enumerate_scopedhave zero production callers at HEAD: workspace grep hits onlymeerkat-core/src/memory.rs,meerkat-memory/src/simple.rs,meerkat-memory/src/hnsw.rs. NOT FOUND: any caller in mob/session/runtime/cli/rpc/rest/facade/mcp-server/mob-mcp. Nobody cleans member memory on retire — locally or remotely. Pre-existing, symmetric.- Session-identity claims are process-scoped by design:
SessionClaimHandle+DefaultSessionClaimRegistry::global()(meerkat-core/src/handles.rs:2096-2148), one registry perMeerkatMachine(meerkat-runtime/src/meerkat_machine/mod.rs:2365-2373; re-exportmeerkat-runtime/src/handles/session_claim.rs:1-17), acquired at comms-runtime construction and held RAII (meerkat-comms/src/runtime/comms_runtime.rs:1573,1890-1893). - Whole-mob resume is controlling-realm-local:
reconcile_resumecensus over the local session service (builder.rs:2638-2652), orphan archive (2662-2675), recreate-from-snapshot for members withmember_session_bindingsentries (the skip at 2680-2687 applies only to members without a binding). Ops-owner anchoring for members without a local session goes throughowner_bridge_session_idwith a silent skip when it isNone(actor.rs:8541-8562, skip at 8553-8554) and a fail-closed divergence check (8582-8600). - Owner-archive cascade runs controlling-host-local:
destroy_bridge_session_mobs+ orphan scavenging (meerkat-mob-mcp/src/lib.rs:1784-1816, 1826-1863), remote teardown bounded byremote_destroy_cleanup_deadline(actor.rs:15816-15823).
19.L1 — MaterializeMember.launch_mode carries Fresh | Resume only; Fork is unrepresentable on the wire; Resume executes on the member host.
Current amendment: the shipping private wire isWire enum (V4):Fresh {}orResume { session_id, resume_from_role? }. The optional predecessor role is one-request authority for an exact cold durable role migration. Omission remains strict same-role resume.Forkis still structurally absent. The following section records the original v4.1 launch design before that additive trusted-host field.
MaterializeLaunchMode { Fresh, Resume { session_id } }. WHY the split: fork collapses to prompt text on the controlling host before any provisioning happens (actor.rs:9240-9248) — the rendered context travels inside the materialize spec’s initial prompt with zero member-side machinery — while the entire resume validation chain is realm-local by nature (live-session check, injector capability, comms presence, persisted snapshot: actor.rs:8978-9074) and the session being resumed lives in the member host’s realm. A Fork payload on this wire would mean shipping one host’s history into another host’s admission path — exactly what §3 forbids. The exclusion is structural at BOTH carriers: MaterializeMember.launch is the closed {Fresh, Resume} wire enum, and PortableMemberSpec (§14.2/§15.3) has NO launch-mode field at all — the domain SpawnMemberSpec.launch_mode: MemberLaunchMode (serde-representable including Fork, handle.rs:1870, launch.rs:20-35) never serializes into the spec, so the second launch-mode channel the domain type would otherwise open is closed by construction (one semantic fact, one wire owner). deny_unknown_fields + per-command V4 decode make a fork payload a fail-closed decode error; if a future variant ever reaches admission, MobHostBindingAuthority rejects it with typed LaunchModeUnsupported.
Member-side Resume realization mirrors the local durability chain against its realm, but a newly admitted higher generation never adopts a live runtime in place: any live incarnation is quiesced and its event projection drained before load_persisted_session + build_resumed_agent_config. AutonomousHost still requires interaction-event-injector capability and wiring still requires comms presence. The ack is where the typed causes live — the local path’s untyped Internal exits (actor.rs:9044-9048, 9128-9130) must NOT be inherited: ResumeSessionNotFound (no snapshot), CapabilityMissing { capability } (injector absent for autonomous mode; durable_sessions absent for snapshot restore), LaunchModeUnsupported. MemberMaterialized.launch_outcome retains the wire vocabulary Fresh | ResumedLive | ResumedFromSnapshot for compatibility with already-recorded replies, while new higher-generation same-session admissions record ResumedFromSnapshot. Idempotency is the existing tuple rule (§7, MaterializeMember): replay on the same (agent_identity, generation, fence_token) returns the recorded result including the recorded launch_outcome.
Controlling-side admission: ResolveSpawnMemberAdmission placement validation (§6.1) additionally rejects a Resume whose session id is bound (in member_session_bindings + member_placement) to a different placement than the spawn requests — typed LaunchModePlacementMismatch. A resume id the machine has no placement fact for crosses the wire and is adjudicated member-side (member-side validation is authoritative; the controlling check only catches provable mismatches early).
19.L2 — Cross-host fork: remote TARGET works at phase 3 by construction; remote SOURCE is a proxied ReadMemberHistory read landing with phase 6; the untyped no-session error is retyped now.
This deliberately revises the “cross-host fork is a v1 non-goal” prior, on evidence: the prior priced a FullHistory CoW that does not exist in this codebase — mob fork never touches Session::fork(); it is a history read + text render (launch.rs:49-61 doc contract; actor.rs:9179-9191, 9240-9248). A remote source therefore uses the already-planned paged ReadMemberHistory bridge command (§7.4); fork adds no new wire surface. Decision:
- Remote target (fork from a local source onto a remote host): works as soon as phase 3 lands — the rendered fork text is already inside the materialize spec’s initial prompt. Nothing to build.
- Remote source: the fork context read routes through the same placement switch as
mob/member_history(§7.4) — local member ⇒ localread_history; remote member ⇒ReadMemberHistorypages, accumulated controlling-side against the first page’s pinnedmessage_count, then rendered by the samerender_fork_context.LastMessagesis tail-addressed in its first request; normally that one byte-bounded page is complete, while large rows may require monotonicnext_indexcontinuation. A single row that cannot fit the transport-safe reply budget fails typed rather than timing out or silently dropping fork context. Ships with phase 6. - Typed reject, both eras:
MobError::ForkSourceUnavailable { source_member_id, cause: NoSession | RemoteReadUnavailable }replaces the untypedMobError::Internal("fork source ... has no session")(actor.rs:9146-9150).NoSessioncovers peer-only external sources (unchanged semantics, now typed);RemoteReadUnavailablecovers a remote source before phase 6 lands or when the owning host is unreachable (Unavailablebridge outcome). The retype lands in phase 3 so the phase-3→6 window never ships the untyped shape.
ReadHistory scope; the mob is reading its own member’s history to provision its own member).
19.L3 — ReleaseMember performs full mob-owned durable disposal on the owning host; disposal class is a typed reply fact. (Amends §7 host-addressed commands and §7.3.)
For HostMaterialized members the mob minted the session, so the mob archives it — the exact ownership discriminator that already exists locally: archive_with_authority_then_unregister’s authority-ownership read (provisioner.rs:868-897) separates mob-owned durable records (archive) from adopted host-owned sessions (runtime retire + binding release only). ReleaseMember realization on the owning host, in order:
MobHostBindingAuthorityadmission:(agent_identity, generation, fence_token)validated; replay at the recorded tuple returns the recorded disposal (dedup memory extended to remember release outcomes, mirroring materialize dedup, §6.3); a lower tuple is typedStaleFence.- Two-layer disposal order, mirroring the local realization EXACTLY (an implementer following a flat ‘archive-first’ rule would archive a still-running session and race): OUTER quiesce first — cancel any active turn → control-plane runtime retire → drain — the
retire_runtime_before_archiveshape (provisioner.rs:866 → :627-686); THEN the archive protocol, whose commit-first ordering is protocol-INTERNAL (MachineSessionArchiveProtocol, persistent.rs:1521-1600: the SessionDocumentMachineArchiveSessionDocumentdurable commit lands in the member host’s realm before the machine-owned documentRetireterminal). A failure in either layer fails the release with a typed cause. - Idempotency mirrors local tolerance:
NotFoundandAlreadyArchived(session_document.rs:369-383) resolve to success-class outcomes (actor.rs:15708-15716 precedent). - Claims/memory: the member’s comms-runtime teardown releases its session-identity claim via RAII (comms_runtime.rs:1573) — no bridge involvement; no memory cleanup (L6).
BridgeReply::MemberReleased { disposal: MemberSessionDisposal } with MemberSessionDisposal = Archived | AlreadyArchived | RuntimeReleasedOnly { cause: HostOwnedSession | NoDurableSessions }. An ephemeral member host (durable_sessions=false at bind) performs runtime retire + binding/claim release and answers RuntimeReleasedOnly { NoDurableSessions } — typed capability degradation, declared at bind, never silent.
One disposal verb per membership class: HostMaterialized members are released only via host-addressed ReleaseMember (fence-validated, host-dedup’d); the member-addressed RetireMember/DestroyMember commands are never sent to them and keep their exact current semantics for legacy peer-only External members (comms_drain.rs:2788-2864 unchanged). ReleaseMember doubles as the orphan-reconciliation verb (§7.3, §9) — same admission, same reply.
Cascade mapping: single-member retire of a HostMaterialized member keeps ArchiveSession criticality (actor.rs:14257-14275) — no MemberReleased, or a reply the tolerance rules don’t absorb, fails the retire loudly with the typed bridge cause. The owner-archive destroy cascade (destroy_bridge_session_mobs, mob-mcp lib.rs:1784-1816; scavenging :1826-1863) stays bounded best-effort under remote_destroy_cleanup_deadline (actor.rs:15816-15823) with per-member failures recorded; anything missed is reclaimed by the next HostStatus reconciliation at a stale fence.
19.L4 — ObserveMemberRetirementArchived AND its destroy sibling stop lying: both signals gain the typed disposal field, total over local and remote. (Amends §6.1 machine deltas.)
Today the shell signals it whenever the ArchiveSession step completed (actor.rs:15285-15289) — which includes the local host-owned carve-out (no mob archive happened, provisioner.rs:868-897) and, if remote retire shipped as-is, the runtime-only bridge retire (comms_drain.rs:2802). Catalog delta: BOTH DSL signals — ObserveMemberRetirementArchived (meerkat-machine-schema/src/catalog/dsl/mob_machine.rs:1071, transitions :4909, :4938) and ObserveDestroyMemberRetirementArchived (mob_machine.rs:1072, transitions :5039-:5161; the destroy path drives it from the identical step-completion gate via dispose_member_for_destroy → dispose_archive_session, actor.rs:15356-15372) — gain disposal: MemberSessionDisposal; fixing retire while the destroy sibling stands would leave the same fiction reachable through destroy-over-remote-members, so both land in one change-set (no divergence window) — the same vocabulary as the wire reply. The shell may populate it only from a typed disposal outcome it actually observed: the local provisioner path classifies mob-archived vs host-owned-released; the remote path copies MemberReleased.disposal from the ack (AlreadyArchived folds into Archived for the machine fact — both mean the durable terminal holds). No inference, no default. mob/member_status projects the recorded class so consoles can distinguish “durably archived” from “released, durable lifecycle owned elsewhere”. Alphabet/schema parity and seam-inventory dispositions per §6 rules.
19.L5 — Whole-mob resume and ops-owner anchoring are placement-gated. (Amends §7.3 recovery and §6.1 invariants.)
reconcile_resume’s census and orphan-archive only ever see the controlling realm (builder.rs:2638-2675) — correct for local members, and remote session ids inmember_session_bindingsare harmless there (the archive set isactive_local − machine). But the missing-session recreate loop is a live trap:HostMaterializedmembers DO havemember_session_bindingsentries (from the materialize ack, §6.1), are never in the localactive_ids, and would enter local snapshot restore (builder.rs:2680-2687 only skips members without a binding) — producing a bogus local recreate of a remote member. The loop gains a placement gate:member_placementpresent ⇒ skip local recreate entirely; remote members reconcile via the (re)bindHostStatussweep +ReleaseMember-at-stale-fence (§7.3) and, where the host reports the session live, plain rebinding — Resume of a host-materialized member is the rebind/recovery path, not a new mechanism.- Ops-owner anchoring extends the owner-bridge-session invariant (commit
093af8cf1; actor.rs:8541-8562, fail-closed check 8565-8633):HostMaterializedmembers bind their controlling-side ops owner to the MobMachine-ownedowner_bridge_session_idexactly like peer-only members. The silent skip whenowner_bridge_session_idisNone(actor.rs:8553-8554) does not extend to them: §6.1 gains the invariantmember_placement ≠ ∅ ⇒ owner_bridge_session_id ≠ None, enforced atResolveSpawnMemberAdmission(typed placement reject when a remote spawn is requested on a mob created without owner-bridge authority), so the restore path can fail closed instead of skipping.
19.L6 — Host-local by construction: compaction, memory, claims. (Statements + §12/§13 amendments; zero new machinery.)
- Compaction is member-host-local because it is loop-internal (agent/state.rs:1233-1265): the member host’s provider credentials fund
DefaultCompactor’s summarization call (wired at itsAgentFactorybuild, factory.rs:5700-5716).CompactionCuratoris an in-process trait with no wire form (compact.rs:175-192) — a curator configured on the controlling host cannot apply to remote members. v1 decision: curated compaction for remote members is configured on the member host process (rkat mob hostcomposition), or not at all;DefaultCompactoris always the member-host default. There is no silent difference to type: curator presence is a host build choice, never a mob fact, and compaction failure semantics (CompactionFailedevent, history preserved) are identical on every host. - Memory: compaction discards index under
MemoryIndexScope::for_sessionin the owning host’s realm (agent/state.rs:1365-1373). Cleanup is a pre-existing repo-wide gap —drop_scope/enumerate_scopedhave zero production callers (NOT FOUND outsidemeerkat-core/src/memory.rsandmeerkat-memoryimpls/tests) — and multi-host does not widen it, only multiplies where orphaned scopes live. v1 records the binding constraint so the future fix cannot be built wrong: memory cleanup executes on the owning host inside its release/archive path (the L3 realization is the insertion point), never as a bridge-served remote store handle (§3). - Claims:
SessionClaimHandlestays process-scoped — one registry perMeerkatMachine(mod.rs:2365-2373), process-global default for bare factory callers (handles.rs:2125-2148). The member host’s own machine registry claims each materialized session at comms construction and releases via RAII on teardown (comms_runtime.rs:1573, 1890-1893) — including onReleaseMember. Cross-host duplicate-materialization protection isMobHostBindingAuthority’s(identity, generation, fence)dedup + fence monotonicity (§6.3), which is the distributed analog of the in-process claim. New gotcha (§13): the host acceptor registry andMobHostBindingAuthoritymust not grow into a distributed claim registry — “this session id is active” remains a per-process fact.
19.L7 — Transcript edits on remote member sessions are a typed v1 non-goal. (Amends §12.)
The admission chain is irreducibly host-local three times over: staged-session materialization check +MeerkatMachine transcript-edit admission on the local runtime_adapter (session_runtime.rs:7563-7614), sole implementation in PersistentSessionService (persistent.rs:5652-5655) over the owning realm’s revision store, and no ControlScope for rewrite exists (§8 enum). Shipping console parity would mean a five-command bridge family with revision paging, CAS-parent semantics, and a new scope — none of which has a named consumer. Decision:
- v1: the family is owning-host-local. Raw session-id RPC verbs keep realm-local
NotFound— truthful under §3, the id names no session in this realm. The by-identity console surface never dangles the verbs:mob/member_statusalready gainsplacement(§7 projections); it additionally carries the host’sdurable_sessions-derived lifecycle capability flags so consoles label transcript editing (and revision browsing) unavailable for remote members instead of attempting and mis-readingNotFound. Precedent for a deliberately absent surface: the REST catalog’s recorded drop of this same family (rest_catalog.rs:174-181). - v2 seed:
RewriteMemberTranscript/ListMemberTranscriptRevisions/RestoreMemberTranscriptRevisionbridge commands gated on a newControlScope::RewriteTranscript, member-side admission reusing the existing machinery — only after phase 5, per the §8 ordering rule.
19.F Failure semantics (rows to add to the §9 table)
19.P Work items by phase
- Phase 1 (catalog + contracts):
MaterializeLaunchModewire enum +launch_outcomeonMemberMaterialized;MemberReleased { disposal }reply +MemberSessionDisposalvocabulary; rejection causesResumeSessionNotFound,CapabilityMissing,LaunchModeUnsupported,LaunchModePlacementMismatch;disposalfield on theObserveMemberRetirementArchivedDSL signal + transitions;HostCapabilityFlags.durable_sessions; §6.1 invariantmember_placement ≠ ∅ ⇒ owner_bridge_session_id ≠ None;MobHostBindingAuthorityrelease-dedup memory;mob/member_statuslifecycle capability flags;MemberHistoryPage.message_countpinned in the page mirror. - Phase 3 (placement + materialization): member-side
Resumerealization + typed ack causes;ForkSourceUnavailableretype (replaces actor.rs:9146-9150Internal); remote-target fork (falls out of spec-carried prompt text — covered by tests, no new code);ReleaseMemberfull disposal realization (archive-first order, idempotency, ephemeral degradation) + controlling-side truthfuldisposalclassification for local AND remote retire;reconcile_resumeplacement gate on the recreate loop; ops-owner anchoring fail-closed forHostMaterialized; member-host claim acquisition/release through its machine registry. - Phase 6 (events/history): remote-source fork context via
ReadMemberHistorypages (shared placement switch withmob/member_history);LastMessagesoffset math over the mirroredmessage_count. - Phase 7 (surfaces): console labeling of per-member lifecycle capabilities (transcript edits, revisions, resume-after-restart) from
mob/member_status;rkatmob status output includes disposal class and placement.
19.N Non-goals (rows to add to §12)
- Remote transcript-edit family (rewrite/revisions/restore/fork_at/fork_replace over the bridge) — typed degradation: realm-local
NotFound+mob/member_statuscapability labeling; v2 seed: bridge command family behindControlScope::RewriteTranscript. - Cross-host
CompactionCuratorpropagation — curators are member-host process composition;DefaultCompactoris the universal default; no wire form exists or is added. - Memory-scope cleanup on retire — pre-existing zero-caller gap, symmetric local/remote; constraint recorded: future cleanup is owning-host-only inside the L3 release path.
- Distributed session-claim registry — claims stay process-scoped; the cross-host analog is
MobHostBindingAuthorityfencing/dedup. - Cross-host
Session::fork()copy-on-write — does not exist even locally (fork is a rendered-context spawn); nothing degrades because nothing is removed.
19.G Gotchas (rows to add to §13)
ObserveMemberRetirementArchivedfires today onArchiveSessionstep completion even when the realization was runtime-retire-only (host-owned carve-out, provisioner.rs:868-897) — when adding thedisposalfield, classify from the typed provisioner/bridge outcome, never from step completion.reconcile_resume’s recreate loop keys onmember_session_bindingspresence (builder.rs:2680-2687) —HostMaterializedmembers HAVE bindings; forgetting the placement gate silently recreates remote members locally.- The ops-owner
None-skip (actor.rs:8553-8554) is load-bearing for legacy peer-only mobs; do not delete it — fence it off forHostMaterializedvia the admission invariant instead. MemberHistoryPagemust mirrorSessionHistoryPage.message_count(service/mod.rs:1497-1505) or remoteLastMessagesfork needs a second round-trip that nobody will remember to add.
20. Threat model and security posture
This section answers the question no chapter owned: what does an attacker get at each trust boundary, and what is the containment claim. Every statement here is consistent with the mechanisms in §§6-19; where v1 is deliberately undefended, it says so plainly (A22 discipline — never laundered).20.1 The containment invariant
A compromised member host controls exactly its placed members — never the mob. Proven against each channel it holds:- Peer impersonation — YES, bounded to placed members. The host holds every placed member’s private key (§7.3 key custody: minted and stays on the host), so it can sign envelopes as those members to every peer that trusts them. It holds NO other keys: not the supervisor’s (the bridge authority keypair lives in the controlling host’s
SupervisorAuthorityRecord), not other hosts’ members’, not the mob owner’s. Its reach is exactly the wiring graph its members legitimately have. - Observation fabrication — YES, labeled. The host is the only source of its members’ history/events (D2 bridge-served projections carry no cryptographic provenance in v1). It can fabricate console-visible transcripts for its own members. Mitigation, not prevention: every remote-served projection is typed
HostClaimed(vsControllingHostVerified, §7 projections), so consoles and MobKit can render the trust distinction; signed event provenance is a v2 seed. - Ack lying — bounded by machine guards. A lying
MemberMaterializedcannot commit arbitrary state:CommitSpawnMembershipguards the digest echo against the machine-authorized value (A12), the fence tuple against machine facts, and membership/roster/wiring mutations only ever happen on the controlling machine. A host can lie aboutlaunch_outcomeor health — observation-grade facts,HostClaimedby construction. - Upcall abuse — bounded by generated agent authority and incarnation fencing.
MemberOperatorRequestfirst requires the envelope signer, requester generation, and requester fence to match the MobMachine’s current member binding; it then re-mints only the controlling-host-recordedMobToolAuthorityContextfacts for that identity+generation. A compromised host can invoke exactly the operator capabilities that member has as an agent, and no principalControlScopegrant is minted or consulted. It cannot mint grants, bind hosts, touch fences, replay an old incarnation through a reused peer key, or expand its capability set through wire claims (there are none). - Authority — NO. Fences, generations,
topology_epoch, roster, placement, grants, and rotation state advance only through the controlling MobMachine. A member host presents observations and requests; it decides nothing mob-semantic.MobHostBindingAuthorityis host-LOCAL state about its own bindings, not mob authority. - Credentials — NO. No credential, token, lease, or provider secret ever arrives on the wire (A2/A5/A10, type-level). A compromised host exposes ITS OWN realm’s credentials — the ones its operator provisioned — which is the pre-existing local-machine trust boundary, not a new one.
20.2 Undefended in v1 (stated, not laundered)
- Member-side fence validation on delivery does not exist (§6.4/A22): a stale supervisor is rejected by epoch, but delivery commands carry no member-fence check. Exposure: a controlling host racing its own respawns, not a third party (delivery is supervisor-signed).
- Host-local realm writes: an operator (or attacker) with filesystem/process access on a member host can mutate placed members’ sessions outside the fence path (§17.8). This is inside the host trust boundary — hosts are TRUSTED EXECUTORS for their placed members, admitted by explicit operator ceremony.
- Transport confidentiality: the comms link is Ed25519-signed plaintext (no cipher exists in meerkat-comms — verified NOT FOUND). On-path readers see envelope contents: peer messages, bridge payloads, event pages, and live WS tokens (A5; single-use, 60s TTL, channel-pinned). New host binding does not add another bearer:
BindHost.bootstrap_proofis tuple-bound HMAC output. The older member-binding protocol still exposes its rawBindMember.bootstrap_token; that pre-existing residual is intentionally not expanded into the new host wire and requires a separately versioned remediation. Operational posture for hostile networks: tunnel inter-host links (WireGuard/TLS); v2 seed: encrypted comms transport, which closes this whole row. - Acceptor pre-auth surface: the TCP acceptor peeks/classifies before signature verification (pairing branch). §21.2 bounds it operationally; it is not authenticated-only ingress.
20.3 Ceremony material custody
- Host binding descriptor (contains the one-time bootstrap token): written 0600 by
rkat mob host, transferred out-of-band by the operator, used locally to derive a tuple-bound HMAC, and consumed single-use when that proof admitsBindHost; the raw bearer never enters the remote command and is invalid thereafter (thecanonicalize_bridge_addressstrip precedent applies — tokens never persist in advertised addresses). A leaked descriptor after bind is inert; a leaked descriptor before bind can let an attacker claim the host — revocable viaRevokeHost, visible inmob/hosts. - Host identity dir: 0700 dir / 0600 key, same discipline as member identity dirs (
Keypair::load_or_generate). - Host revocation and key rotation (v1):
RevokeHostis the compromise-recovery verb — clears the bind, live endpoint, and capability record; placed members enter the revival ladder (§9 row). Key rotation = revoke + fresh ceremony with a new descriptor. Online host-key rotation is a v2 seed. - Grants: issued only through
AdminGrants-scoped surfaces (owner-implicit in v1, A16); persisted in the controlling host’s mob store; never on member hosts.
21. Operations, packaging, and governance
21.1 Rolling upgrades and protocol skew
BridgeCapabilities is deny_unknown_fields, so a NEWER host’s bind reply with new capability flags is a decode-breaking read for an OLDER controlling host. The upgrade order is therefore fixed: controlling host first, member hosts after — a controlling host must understand every capability shape it can receive; member hosts advertise version RANGES and the controlling host speaks the highest common version per host. Rules:
- Spawn/placement admission onto a bound host whose
supported_protocol_versionsexcludes V4 is a typed reject (protocol range is an admission-gating flag, §6.1). - Rebind re-records capabilities (
HostRebound); a range/capability regression routes affected members through revival classification with a typed cause (§9 row) — never silent degradation. - Mid-flight upgrade: an ack
engine_versiondiffering from the bound record is typedHostEngineVersionChanged→ rebind → retry (§15 R8). - Procedure for a live mob: upgrade the controlling host (members keep running — the bridge tolerates supervisor restarts via durable authority records); then per member host: drain new placements (operator choice), upgrade, restart → rebind → capability re-record → host-autonomous member revival (A20); respawn members only where
engine_versionsensitivity matters (operator policy — equal specs on unequal binaries are honestly different builds, §15 R8).
21.2 Ingress hardening and resource governance
- Acceptor pre-auth bounds (the peek/classify + pairing parse run before signature verification): connection cap + per-connection read deadline + the existing
MAX_PAYLOAD_SIZEframe bound + a pairing-attempt rate limit per source. These are host-role config (§21.3), mechanical shell limits — no machine facts. - Poll-pump fan-out (v1): one pump per remote member per mob on the controlling host, under a fixed per-mob ceiling of eight concurrent polls. Six permits serve observation-only 10s long-polls; two permits are reserved for machine-owned remote-turn / placed-kickoff / completion-cleanup custody, whose polls use a 250ms window. Before contending for its global class permits, every pump crosses a one-per-host weak semaphore for that class, so hundreds of blackholed members on host A can occupy or queue at most one global observation slot and one global custody slot while a healthy host B remains independently eligible. A queued pump reclassifies when its liveness facts change, releasing the old host-class gate before it competes in the new lane; the combined in-flight ceiling remains eight. Per-host batched polling and a configurable cap require a new observation transport/config seam and remain v2 work alongside outbound connection reuse (§12); v1 does not claim either one.
- Placed cleanup fan-out (v1): placed kickoff and ordinary-completion reconcilers admit at most one due row per host into a rotating eight-host window. Per-host row cursors prevent a blackholed low row from starving sibling custody, and the host cursor advances beyond the last admitted host so large fleets do not repeatedly overlap the same window. Mob Stop is two-phase: first persist
KickoffCancelRequestedand join every volatile kickoff producer, then—only after exact kickoff custody drains—run a bounded rotating window of exact fenced interrupts off the actor loop. No controller-wide mutex spans bridge I/O; same-key host cancellation tombstones arbitrate cancellation racing a delayed kickoff delivery. Retire/Complete/Destroy instead proceed to authenticated host release/revoke and dispose exact kickoff custody, so a blackholed ordinary cancel cannot deadlock teardown. - Ephemeral live buffer (
durable_sessions=falsehosts): bounded ring, size in host config; overrun is the typedStaleCursorpath — sizing changes the window, never the semantics. - One periodic host poll, two consumers: the reachability probe (§7.5) and orphan reconciliation share the SAME
HostStatuspoll loop — reconciliation is therefore continuous, not rebind-only (a healthy host that never restarts still gets its orphans released at stale fences). Mass-rebind after a controlling restart staggers polls with jitter.
21.3 Host daemon configuration (the layered-config row for the new role)
[mob_host] in .rkat/config.toml, per the Config::default → file → env → flags doctrine (flags override file):
MobHostBindingAuthority record — nothing about the daemon’s identity, bindings, or members depends on process memory.
21.4 Crate placement and release blast radius
No new publishable crate in v1 — the 38-crate publish order and
check-rust-release-packaging are untouched. Every change-set that adds test files or dependencies regenerates Bazel BUILD metadata + MODULE.bazel.lock (pre-commit hook; skipping it cascade-fails the BuildBuddy lanes on stale locks). Feature-gating keeps test-minimal / test-surface-modularity green: the host role rides the existing mob feature; live rides live/openai-realtime.
21.5 Docs and governance-gate inventory
Documentation that encodes facts this plan changes, all underdocs-check: docs/reference/capability-matrix.mdx (capability degradation rows), docs/reference/mob-architecture.mdx, docs/reference/comms-reference.mdx (acceptor, trust), docs/reference/session-contracts.mdx (remote observation semantics), docs/reference/runtime-architecture.mdx (host role), docs/reference/machine-authority.mdx (new state/inputs/effects + the scoped-authority note below), docs/api/rpc.mdx (regenerated — 8+ new methods), docs/api/mcp.mdx (HAND-updated; no script gate exists for MCP — named work item, §17 SD-4). The dogma-skill mirror (sync-meerkat-dogma-skill-docs) fires if doctrine text changes.
MobHostBindingAuthority governance treatment (decided here, not ad hoc): it follows the session_persistence_version_authority precedent exactly — catalog DSL source, generated production module, registered for production-schema parity, an ownership-ledger entry for each fact it owns, not in canonical_machine_schemas() and therefore no poster requirement (verify-machine-poster-coverage keys on the canonical alphabet); machine-authority.mdx gains one paragraph naming it as a scoped authority. Its effects carry seam-inventory dispositions like any machine’s.
21.6 MobKit consequence pass (expands phase 8)
- Ordering: Meerkat release with phases 1-7 → MobKit bumps its pin → MobKit consumes. No MobKit change ships against an unreleased wire shape.
- Unaffected by default: MobKit talks local RPC to the controlling host = owner principal (A16) — default-deny grants change nothing for it. Its per-turn injection lane (
injected_context) already bridges to remote members viaDeliverMemberInput(shipped, §15.1). Its resume-strand logic operates on controlling-host session projections — placement-blind by construction. - New consumption: placement + reachability + lifecycle-capability fields on
member_status;mob/member_history;mob/hosts; grants surfaces (as anAdminGrants-scoped operator UI, later);HostClaimedprovenance labeling in console rendering (§20.1 — MobKit must not render host-claimed transcripts as verified). - Fleet view across mobs is a MobKit projection concern (hosts are per-mob MobMachine facts; there is no cross-mob host registry endpoint, §17.10).
21.7 RPC/REST principal identity (the recorded v2 seed)
v1 leaves local RPC/REST/stdio as owner-principal surfaces (A16). The v2 seed, named so it is not re-invented: bearer-token →PrincipalId authentication on rkat-rpc --tcp, layered on secure_rpc’s TcpBindPolicy (whose own doc already says post-connect authorization “belongs to auth/grants”), with tokens minted/revoked through the same AdminGrants-scoped surfaces and mapped into the ONE grant vocabulary (§8). No parallel ACL system.