gemma-4-31b behaves like any other model ID in the runtime. Tool visibility and multimodal behavior still depend on model capabilities, so provider/model differences can affect the effective tool surface.
This page is the concept layer for provider abstraction. Use Auth and Self-hosting models for setup workflows, and use reference pages for exact capability and contract details.
Instruction projection
Provider independence does not make unlike wire formats identical. Meerkat keeps the exact orderedSystem / SystemNotice sequence in the durable
Session transcript and derives each provider request from it:
- Standard OpenAI Responses and OpenAI-compatible Chat Completions preserve each instruction at its position among user and assistant messages.
- OpenAI Realtime preserves the same ordering with in-conversation System
items; its separate session
instructionsfield is reserved for provider-owned configuration. - Cataloged Anthropic Fable 5, Opus 4.8, and Opus 5 support
mid-conversation System messages. Meerkat lowers a turn-scoped canonical
System -> Userboundary to Anthropic’s legalUser -> System -> Assistantplacement without changing the durable transcript. Other Anthropic models and Gemini can represent only a leading System prefix. The private ChatGPT Responses wire can represent at most one leading System message. A limited provider projection returns a typed error for an unrepresentable transcript instead of hoisting or merging messages. The ordinary System row remains authored in the durable Session; selecting a capable wire can execute that Session without rewriting it.
Provider setup
- Anthropic
- OpenAI
- Gemini
- Self-hosted
config.toml (active realm)
Environment variables
The
RKAT_* variants take precedence over provider-native names, so you can run Meerkat with dedicated keys separate from other tools.provider = "self_hosted" (auth_method of none, api_key, or static_bearer). See Self-hosting models for the realm binding shape.
Image generation providers
Thegenerate_image builtin uses provider-specific image profiles behind a single Meerkat request shape. The active session model does not have to be an image model; image operations can route to a provider default or a forced image target while preserving the original session identity.
Request
provider_params are provider-specific and do not replace Meerkat’s universal image fields. Use top-level size, quality, format, and intent; the OpenAI adapter lowers format to provider-side output_format. For the current gpt-image-2 default, public callers should only need background, output_compression, moderation, hosted-tool-only action, hosted-tool-only reasoning_effort, and hosted-tool-only web_search; use background: "auto" or "opaque" (not "transparent"), use output_compression only with format: "jpeg" or "webp", usually omit action, and omit input_fidelity because Meerkat rejects unknown OpenAI image provider params. Gemini accepts aspect_ratio and image_size. See Image generation for the exact request shape and troubleshooting.
SDK feature flags
When using Meerkat as a Rust library, enable only the providers you need:Provider parameters
Use--param for common top-level overrides such as temperature. Provider-native
options use the canonical provider_params.provider_tag envelope; pass that shape
with --params-json in the CLI or provider_params in an SDK. For OpenAI catalog
models, params_schema describes the fields inside the provider_tag object.
Provider-native web search is on by default for catalog models that support it.
Disable it in config with the matching provider_tools.<provider> search toggle,
or for a single CLI run with rkat run --no-web-search "...".
- Anthropic
- OpenAI
- Gemini
Automatic caching uses Anthropic’s five-minute ephemeral cache and advances
the breakpoint as the conversation grows. A qualifying cold write bills
input tokens at 1.25x the base rate; a cache hit bills cached input tokens
at 0.1x. Automatic lookup scans backward at most 20 cacheable blocks, so
adding more than 20 blocks between requests can still miss. Set
cache_control: "disabled" to opt out, or system_prefix when only the
stable system prompt should be cached. Amazon Bedrock rejects automatic
locally but continues to support system_prefix.Model catalog
Meerkat ships a curated built-in model catalog in themeerkat-models crate (meerkat-core owns only the vocabulary types and ModelCatalog mechanics) and merges it with any configured self-hosted aliases into one effective runtime registry used for capability detection, provider resolution, and catalog responses.
Query the catalog programmatically from any surface:
- CLI:
rkat models - RPC:
models/catalog - REST:
GET /models/catalog - MCP:
meerkat_models_catalog
self_hosted provider group and include their backing server_id.
For Gemma 4 specifically, prefer chat_completions as the default OpenAI-compatible interface. It is the clearest common path for tool calling across Ollama, LM Studio, and vLLM, while reasoning-trace semantics still vary by server.
Provider resolution
The provider for a model is resolved by exact match against the config-backed model registry (the compiled-in catalog merged with any configured self-hosted aliases). There is no name-prefix inference: a model id resolves to the provider recorded for its catalog entry (or its self-hosted alias config), and an uncatalogued id — even a prefix-shaped one likegpt-unknown-preview or
claude-unknown-preview — is rejected rather than guessed at.
A configured self-hosted alias such as gemma-4-31b resolves by exact model ID
match, so it works without --provider.
You can still override this with --provider on the CLI or provider in API requests.
Session creation resolves model, provider, and credentials as one identity on
CLI, RPC, REST, and MCP. An explicit model wins. An explicit provider constrains
all omitted-model choices, and a named auth_binding is resolved through the
effective realm inheritance chain to infer and validate its provider. The
binding’s default_model wins when the model is omitted; otherwise Meerkat
uses a compatible configured global model, the configured per-provider model,
and finally that provider’s catalog default. With no provider or binding hint,
the configured global model wins, followed by configured provider defaults in
catalog priority and the global catalog default.
A nonempty agent.model is always operator intent, including a still-supported
model that appeared in an older generated config. Meerkat does not reinterpret
model IDs through a frozen legacy-default list. Explicit provider/binding
mismatches, malformed bindings, and binding defaults owned by another provider
fail before agent construction; inherited bindings retain the realm that owns
their configuration.
Model fallback chain
Factory-built agents can use an ordered model fallback chain when the active model reaches a typed, recoverable LLM failure boundary. This protects long sessions from provider-side disruptions such as removed models, rate limits, provider overload, auth failures, and context-window overflow. Fallback is enabled by default. With no explicit chain, Meerkat builds a catalog-owned backup order from the configured provider defaults and the global catalog default, excluding the active model and duplicate targets. You can replace that order in realm config:config.toml
provider is optional when the model exists in the effective registry.
auth_binding is optional; when omitted, the target resolves through the same
provider-runtime registry used for normal session creation. The same
model/provider with a different auth_binding is a distinct fallback target,
so custom chains can fail over to another credential realm. For the catalog
default chain, Meerkat keeps candidates inside the selected non-env realm when
that realm has a matching provider binding; unavailable catalog-default
candidates are skipped rather than silently bleeding into another realm. For an
explicit custom chain, an unavailable target is a configuration error.
When a switch is applied, Meerkat validates a capability-profile witness minted
by the agent’s exact captured effective-registry authority, then freshly
resolves the target profile, context window, output ceiling, and capability
filter from that registry. An independently constructed registry cannot
authorize the switch even when it contains an identical provider/model row;
an unresolved target is rejected, and the public fallback proposal carries no
client-owned capability or limit fields. Meerkat seals the accepted identity,
profile, filter, and retry attempt in an opaque one-shot activation proof that
only the core agent loop can mint; holding a public runtime routing handle is
not enough to fabricate a commit. Meerkat
prepares the next session/request policy and complete tool-visibility state,
then asks generated authority to preauthorize an exact one-shot routing commit.
After reversibly activating the exact prebuilt target client and rotating the
auth lease, the runtime supervisor compare-and-swaps a control-only copy of the
last committed session snapshot: only LLM identity and typed visibility change;
the in-flight prompt, fallback notice, transcript history, and usage do not.
The preauthorized token then atomically recommits the generated runtime’s LLM
identity, capability surface, routing baseline, visibility filters, and
visibility revision before the agent publishes prepared local state. Lost store
acknowledgements are resolved by reading back the exact old or target bytes, and
a rejected machine realization restores the old bytes before client/auth
compensation. An unprovable third outcome forces canonical executor teardown;
it never enters ordinary failed-batch retry. Runtime routing status and
image-generation Auto therefore observe the same sticky fallback provider as
later agent turns, including the new provider parameters, output-token ceiling,
and capability base filter. Later turns read the registry-minted active profile
and canonical ToolScope projection, never client-local capability or limit
projections. The immediate visibility commit rebases any pending
staged revision, so a later normal turn boundary preserves pending intent but
cannot roll the active revision backward. The supervised operation outlives a
dropped run future, so hard interrupt settles the same result before the live
session can be discarded. Once the control commit succeeds, the fallback stays
sticky even if that provider retry is terminal; cold recovery loads the target
identity with the pre-turn transcript, so the failed input can be retried once
without duplicating its prompt or notice. The agent receives a hidden system notice with the
source model, fallback model, failure reason, skipped targets, active model
limits, and any tools hidden by the fallback model’s capabilities.
For structured-output extraction turns, fallback keeps the extraction request
deterministic: before changing any active state, Meerkat compiles the schema
through the exact inactive target client, injects that provider-lowered
representation into the target request, and keeps provider-native web
search/grounding disabled on the retry. Target compilation failure leaves the
client, lease, runtime identity, visibility, and session fallback state
unchanged.
Capability changes are expected. For example, falling back from a 1M-context
vision model to a 128K local model can clamp output tokens, hide image-result
tools, and skip the local target entirely if the failure was a context overflow
larger than that target’s context window. Later turns remain sticky to the
active fallback model until the session is explicitly hot-swapped or rebuilt.
Meerkat does not fallback on call/network timeouts, and it suppresses
cross-model fallback for any retryable error after user-visible text or
reasoning stream output has been emitted. Ordinary same-model retry policy can
still apply when the recovery authority permits it.