Ingesting Data

Ingestion turns supplied evidence into a source, an immutable rendered revision, and, when requested, extracted graph claims. It is asynchronous: a successful submission establishes that input was staged and work was queued. Use the returned IDs to inspect the result. For a complete runnable example with saved receipts, follow From a Source to Evidence-Backed Claims.

Choose an input path

What you have	Use	Important distinction
Text and a title	`ingest_doc_simple` or `POST /v1/docs/ingest`	Creates a source-backed document
Known provenance plus supplied content	`ingest_source_simple` or `POST /v1/sources/ingest`	Lets you name the origin and choose ingest depth
A URL to fetch	MCP `ingest_url` on `/mcp/full`	Source ingest’s `uri` field alone does not fetch content
UTF-8 files on the client computer	Optional `elephant-local-helper`	Reads client files and sends text
Media on the server or inline attachments	Server ingestion with `media`	Paths must be accessible to the server
Several independent documents	`batch_ingest_docs` or `POST /v1/docs/batch`	Inspect each item for staging failure

Check readiness and access

Call get_runtime_status or GET /v1/runtime/status first. Ingest only when semantic_ingest_ready is true. If not, inspect issues and the individual storage, worker, and extractor readiness fields. There is no degraded semantic ingestion mode that silently substitutes rule-based extraction. The MCP examples below are arguments for the named tool. Use /mcp/full to follow every ingest and inspection step; the default profile exposes convenient ingest tools but omits document pipeline status and some other inspection tools. Availability also depends on the principal and current runtime. HTTP examples assume explicit loopback development with SKIP_AUTH. On an authenticated server, add -H "Authorization: Bearer $TOKEN" to each request. The principal selects the space and must be allowed to write the source and document. Extraction uses your configured model provider.

Ingest a text document

MCP
REST

Call ingest_doc_simple:

{
  "title": "Cedar architecture",
  "text": "Cedar API depends on Cedar Storage.",
  "collection": "architecture-notes",
  "extraction_skill": "extraction/technical-docs"
}

Save the returned document ID and source link from the response’s data.

curl -fsS http://127.0.0.1:3000/v1/docs/ingest \
  -H 'Content-Type: application/json' \
  -d '{
    "title": "Cedar architecture",
    "text": "Cedar API depends on Cedar Storage.",
    "collection": "architecture-notes",
    "rendition_key": "cedar-architecture-v1",
    "extraction_skill": "extraction/technical-docs"
  }'

Save doc_id, source_id, and work_item_id from the response. sync: true is rejected; follow the asynchronous status flow.

The convenience inputs generate base fields. Use metadata for application context and extraction_purpose, extraction_skill, or extraction_domain for extraction hints. Space configuration takes priority over those hints; inspect resolved guidance when the result differs from your intent.

Ingest a named source

Use the source-first endpoint when the origin of the content is meaningful to your application. This example stages the supplied text for graph extraction:

curl -fsS http://127.0.0.1:3000/v1/sources/ingest \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Cedar architecture note",
    "uri": "https://docs.example.com/cedar/architecture",
    "source_type": "web",
    "title": "Cedar architecture",
    "text": "Cedar API depends on Cedar Storage.",
    "collection": "architecture-notes",
    "ingest_mode": "graph"
  }'

The same argument fields are accepted by MCP ingest_source_simple. The result contains source lifecycle information and, when content was staged, a doc_id.

Ingest mode	Requested depth
`metadata`	Record and normalize source metadata
`retrieval`	Materialize supplied evidence for retrieval
`graph`	Continue through extraction and validated graph commit
`skip`	Record the source decision without deeper processing

With supplied content the default is graph; without it the default is metadata. For a metadata-only source, omit title, text, media, and other staged-document fields. A URI is provenance metadata, so supply actual content when asking for retrieval or graph ingestion.

Track the result

Pass the returned document ID to get_doc_pipeline_status:

{
  "id": "<returned document ID>"
}

The REST equivalent is GET /v1/docs/{id}/pipeline-status.

Field	What to inspect
`extraction_state`	`pending`, `running`, `completed`, `failed`, or `not_triggered`
`revision`	The selected evidence revision, when available
`chunks.total`, `chunks.embedded`	Structural and embedding progress
`assertions_count`	Extracted assertion count
`entities_created`	Distinct resolved entities counted from assertions
`extraction_guidance`	Resolved purpose, skill, and resolution trace

Read the source with get_source or GET /v1/sources/{id} as well. Its lifecycle_status, current_ingest_mode, and ingest decision explain whether processing reached the requested depth or stopped as skipped or rejected. Work-item failures are reported separately. Document status summarizes processing; it does not certify the meaning of the generated claims. Inspect assertions and their evidence next.

Retry deliberately

Save submission receipts before starting further work. If a connection drops after the server accepted an ingest request, blindly resubmitting a convenience request can create another document. Those inputs normally generate fresh IDs. rendition_key participates in document processing identity; it is not a global idempotency key for every normalized HTTP submission. Sending the same key with newly generated document IDs does not guarantee one document or one work item. Integrations that require stable updates should retain source/document identity and use the appropriate complete-record ingestion contract after checking the persisted result. For failed work, inspect its error and fix the input or runtime cause before using an authorized work-item retry. A status check should never itself submit another copy of the source.

Batch documents

Use REST for normalized document inputs:

curl -fsS http://127.0.0.1:3000/v1/docs/batch \
  -H 'Content-Type: application/json' \
  -d '{
    "items": [
      {"title":"Cedar API","text":"Cedar API depends on Cedar Storage."},
      {"title":"Cedar Worker","text":"Cedar Worker processes queued jobs."}
    ]
  }'

MCP batch_ingest_docs takes a docs array. It normalizes omitted base fields and returns succeeded, failed, and a job_id. REST returns succeeded and failed. Inspect failures by their input index and retain the successful receipts; retrying the entire batch can duplicate successful inputs. These are staging results. The MCP batch job can be marked done while each source’s extraction is still queued. Inspect each document’s pipeline state to establish processing completion. Request-shape validation can reject the whole request before item-level processing starts.

Fetch a URL

On /mcp/full, call ingest_url:

{
  "url": "https://example.com/report.txt"
}

The server fetches the URL and handles the supported content according to its response type. There is no equivalent REST fetch-URL route. REST clients fetch the content themselves and submit it using document or source ingest, retaining the original URI as provenance.

Files and media

The optional elephant-local-helper provides client-side ingest_file and ingest_directory tools. It reads UTF-8 text; it is not a binary PDF, image, or video decoder. For example, its ingest_file accepts:

{
  "path": "/path/to/report.md",
  "source_type": "text/markdown"
}

The Full profile also has server-side tools with those names. Confirm which server owns the tool: a path on your laptop is not necessarily a path on the Meerkat Recall host. Mount intended server files explicitly; the supplied Compose file does not create an /ingest mount. For image/video content, use the media inputs in the document API. Ordered attachments belong to one document and are rendered together. Text submitted alongside media provides sidecar context; the rendered projection becomes the canonical searchable evidence.

Troubleshooting

Symptom	Next check
Ingest tools are absent	Runtime readiness and endpoint profile
Source is accepted but no document exists	Requested mode and whether content was supplied
Extraction stays pending	Worker readiness, queued work, and lease/error state
Extraction uses an unexpected skill	`extraction_guidance.resolution_trace` and space overrides
Search cannot find the new document	Caller authorization, processing state, collection, and search mode
Batch says done but graph is empty	Per-document status and source lifecycle

Continue with Searching to retrieve evidence or Truth slots to inspect how committed claims are reconciled.

Getting started

Core concepts

By example

Guides

Choose an input path

Check readiness and access

Ingest a text document

Ingest a named source

Track the result

Retry deliberately

Batch documents

Fetch a URL

Files and media

Troubleshooting

​Choose an input path

​Check readiness and access

​Ingest a text document

​Ingest a named source

​Track the result

​Retry deliberately

​Batch documents

​Fetch a URL

​Files and media

​Troubleshooting

Choose an input path

Check readiness and access

Ingest a text document

Ingest a named source

Track the result

Retry deliberately

Batch documents

Fetch a URL

Files and media

Troubleshooting