> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Manifests

> Declarative resource configuration with YAML manifests

Manifests let you define Mixpeek resources in YAML files and apply them in a single operation. This enables version-controlled, reproducible infrastructure across environments.

## Quick Start

```yaml theme={null}
# mixpeek.yaml — a complete pipeline: ingest → index → search
version: "1.0"
metadata:
  name: "video-search-env"

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1

buckets:
  - name: raw_videos
    namespace: video_search
    schema:
      properties:
        video: { type: video }

collections:
  - name: video_index
    namespace: video_search
    source:
      type: bucket
      bucket: raw_videos
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v1

retrievers:
  - name: video_search_tool
    namespace: video_search
    collections: [video_index]
    input_schema:
      query:
        type: text
        required: true
    stages:
      - stage_name: search
        stage_type: filter
        config:
          stage_id: feature_search
          parameters:
            searches:
              - collection_identifiers: [video_index]
                query:
                  input_mode: text
                  text: "{{INPUT.query}}"
                feature_uri: "mixpeek://multimodal_extractor@v1/embedding"
                top_k: 10
            final_top_k: 10
```

Every collection needs a `source` and a `feature_extractor`; every retriever needs `namespace`, `collections`, and its `stages` pipeline. `validate` names any missing field, warns on unknown top-level keys (a typo like `retreivers:` won't silently drop your search pipeline), and checks that references (`namespace`, `bucket`, collection names) resolve within the manifest.

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

### Applying over resources that already exist

`apply` takes a `mode` query parameter. It defaults to `create_only`, which **fails if any
resource in the manifest already exists**, because the manifest is treated as a closed world:
a reference to a bucket that exists but is not declared is a validation error.

Use `create_missing` to add to a namespace that already has resources. It creates what is
missing, leaves what exists alone, and resolves references against the resources already in
the namespace.

```bash theme={null}
# Add these collections over buckets that are already there
curl -X POST "https://api.mixpeek.com/v1/manifest/apply?mode=create_missing" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@collections.yaml"
```

`apply` plans before it creates anything. It runs the same checks as `validate`, and any
error refuses the whole manifest: nothing is created, and each error names the resource, the
field and the fix. The plan refuses:

* an extractor that no longer exists or is turned off on this deployment
* a retriever stage that is not registered, such as a retired stage type
* a feature URI that no collection in the manifest writes, such as one naming a renamed model
* two collections in one namespace that write the same vector at different sizes, such as one
  `multimodal_extractor` v2 collection with `output_dimensionality: 1536` and another at the
  default 3072. A namespace holds one index per vector, so give both the same size:

  ```yaml theme={null}
  feature_extractor:
    name: multimodal_extractor
    version: v2
    parameters:
      output_dimensionality: 1536   # every collection writing gemini-embedding-2 in this namespace
  ```
* an alert application that does not map an input its alert's retriever requires
* a resource its create request refuses: the namespace, bucket, collection, retriever, cluster, taxonomy, trigger, webhook, alert and storage connection are each built into the same request a real create sends, so a cluster with a non-boolean `hierarchical_grouping` or a taxonomy with an unknown `taxonomy_type` is refused here and reads `cluster:<name>: <field>: <reason>`
* a bucket sync whose `sync_mode` is not `initial_only` or `continuous`
* a `seed` that names a sample key the Mixpeek sample bucket does not hold (a read-only listing; if the bucket cannot be listed, `validate` adds a warning and refuses nothing)

`POST /v1/manifest/validate` runs every check above, so a manifest it passes is one `apply` accepts. `validate`, `diff` and `lint` each take the `parameters` form field that `apply` takes.

A namespace `budget` above your organization's budget is set to the organization's budget, and
apply reports the change in `warnings`.

`dry_run=true` returns that plan and creates nothing. If a resource fails partway through,
`apply` rolls back the ones it already created, and if the API is stopped in the middle of an
apply, it rolls that apply back when it starts again.

## Core Operations

| Operation | Description |
| - | - |
| **Apply** | Create all resources defined in the manifest. `mode=create_only` (default) fails if anything exists; `mode=create_missing` fills the gaps |
| **Validate** | Check syntax, schema, and references without changes |
| **Export** | Generate a manifest from existing resources |
| **Diff** | Compare manifest against current state |

## Resource Types

Manifests support these resource types, applied in dependency order:

1. **Namespaces** - Isolation boundaries with feature extractors and a `budget`
2. **Extractors** - The extractors collections run, declared under `extractors:`
3. **Storage connections** and **buckets** - Object storage with schemas
4. **Bucket syncs** - Scheduled pulls from a connection
5. **Collections** - Document stores with indexing
6. **Retrievers** - Search pipelines
7. **Alerts** - Retrievers or system conditions that notify you
8. **Clusters** and **taxonomies** - Grouping and classification
9. **Triggers** and **webhooks** - Automation

## Extractors

An `extractors:` section declares the extractors the manifest's collections run. Each entry has a `name`, a `namespace`, a `version` (default `v1`), a `source` and an optional `compute_profile`. Collections reference an entry by `name` in `feature_extractor`, and namespaces in `feature_extractors`. Once a manifest has an `extractors:` section, every extractor a collection or namespace names must be declared in it. Extractors apply before collections, so a collection never names an extractor that does not exist yet.

`source` is one of:

* `marketplace`: a feature extractor name from the registry, for example `text_extractor`. This is the default when `source` is omitted (`{marketplace: <name>}`).
* `model`: a model your organization uploaded through `POST /v1/models/uploads`, as `org:<model_id>`. Apply validates the model if it has not been validated, then serves it as an extractor at zero replicas until the first call.
* `huggingface`: a model repository on the Hugging Face Hub, as `{repo, revision, framework, token_secret}`. Apply imports the snapshot into your organization's model registry through the same checks an upload and confirm run, then serves it exactly like `model`. No manual upload is needed.

```yaml theme={null}
extractors:
  - name: text_extractor            # marketplace (the default when source is omitted)
    namespace: video_search
  - name: brand_classifier          # a model you uploaded
    namespace: video_search
    source:
      model: org:mdl_7f3a9c
  - name: minilm                    # a model from the Hugging Face Hub
    namespace: video_search
    source:
      huggingface:
        repo: sentence-transformers/all-MiniLM-L6-v2
        revision: "<40-character commit sha, or a tag>"
        framework: sentence-transformers   # the default; transformers is also accepted
        token_secret: HF_TOKEN             # only for a gated or private repo
```

A `huggingface` source is checked against the Hub at validate (read-only: repo info, refs and the file list), and apply refuses the same things:

* `revision` is required and must be a full 40-character commit sha or a tag. A branch such as `main` is refused because it moves, and the same manifest has to import the same weights on every apply.
* Weights must be safetensors. The import takes `.safetensors`, `.json`, `.txt` and `.model` files and skips pickle weights (`pytorch_model.bin`, `.pt`, `.ckpt`), because loading a pickle file can run code. A repo with only pickle weights is refused.
* The files the framework needs must be at the repo root: `config.json` and `modules.json` for `sentence-transformers`, `config.json` for `transformers`. An extractor can serve `sentence-transformers` models today, so `framework: transformers` is refused on an extractor.
* The snapshot must fit the model upload limit (5 GB). The limit is checked from the file list and again while downloading.
* A gated or private repo needs `token_secret`, the name of an organization secret holding a Hugging Face access token (`HF_TOKEN` or `${{ secrets.HF_TOKEN }}`). A public repo needs none.

The import records the repo, revision, resolved commit and the model card's license on the registry entry (`GET /v1/models/{model_id}` shows `license`). Applying the same repo and revision again in the same organization reuses that entry with no Hub call and no download, and a tag that points at an already imported commit reuses it too. While the import runs, the apply's task shows the extractor as `importing_model`; after that it moves through `validating_model`, `deploying_model` and `warming` or `ready`, the same states as a `model` source.

`compute_profile` carries the fields of a compute profile (`resource_type`, `batch_size`, `max_concurrency`, memory). Apply stores them as the namespace's batch override for that extractor, merged over the extractor's built-in profile when a batch is built. Two nested blocks are checked rather than stored:

* `batch.enabled`: only `true` is accepted. A collection that names the extractor runs it in batch.
* `serve`: the real-time serving block (`enabled`, `min_replicas`, `max_replicas`, `min_reason`, `idle_timeout_s`). Validate and apply check it; realtime still runs under the extractor's built-in serve settings, and `warnings` says so. A `min_replicas` above 0 is refused with the monthly cost of the warm replicas named, because a template adds no always-on floor. Leave it at 0 and the first call after idle waits for a replica to start.

```yaml theme={null}
extractors:
  - name: text_extractor
    namespace: video_search
    version: v1
    compute_profile:
      resource_type: cpu
      batch_size: 32
  - name: brand_classifier
    namespace: video_search
    source:
      model: org:mdl_7f3a9c

collections:
  - name: transcripts
    namespace: video_search
    source: {type: bucket, bucket: raw_videos}
    feature_extractor:
      feature_extractor_name: text_extractor
      version: v1
```

A collection naming an extractor the section does not declare is refused at plan with the name. Applying the same manifest again is a no-op for the extractors it declares.

## Parameters

A `parameters:` section declares the values a manifest needs from whoever applies it, so a template ships without a hand edit or a secret inside it. Reference a parameter anywhere in the document as `${{ params.<name> }}`. A reference that is the whole value takes the parameter's type; one inside a longer string is formatted into it as text. An `array` parameter is a list and can only be a whole value.

| Field | Type | Default | What it does |
| - | - | - | - |
| `name` | string | required | The name you reference as `${{ params.<name> }}` |
| `type` | `string`, `integer`, `number`, `boolean`, `array` | `string` | The type a supplied value must match |
| `items` | `string`, `integer`, `number`, `boolean` | `string` | The type of each item when `type` is `array` |
| `enum` | list of values | none | The allowed values. For an `array`, every item must be one of them |
| `required` | boolean | `true` | Whether a value must be supplied. Ignored when `default` is set |
| `default` | matches `type` | none | Used when no value is supplied. A list for an `array` |
| `description` | string | none | Shown next to the input in Studio |

```yaml theme={null}
parameters:
  - name: camera_url
    type: string
    description: Your camera's rtsp:// url
  - name: polling_seconds
    type: integer
    default: 300

buckets:
  - name: camera_feed
    namespace: site_cameras
    sync:
      source_url: ${{ params.camera_url }}
      interval_seconds: ${{ params.polling_seconds }}
```

Pass the values on validate and apply as the `parameters` form field, a JSON object of name to value:

```bash theme={null}
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@camera.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'
```

An array parameter with allowed values:

```yaml theme={null}
parameters:
  - name: search_collections
    type: array
    items: string
    enum: [products, reviews, faqs]
    default: [products, reviews]
  - name: tier
    enum: [free, pro]
    default: free

retrievers:
  - name: site_search
    namespace: shop
    description: "${{ params.tier }} plan search"   # a scalar can sit inside a string
    collections: ${{ params.search_collections }}   # an array must be the whole value
    stages: [...]
```

Supply an array as JSON: `-F 'parameters={"search_collections": ["faqs"], "tier": "pro"}'`. A value outside `enum` is refused with the allowed values listed, and an array written inside a longer string (`"types: ${{ params.search_collections }}"`) is refused.

A missing required parameter, a value of the wrong type, a reference to an undeclared parameter, or a name declared twice is refused before any resource is created, naming `parameters.<name>`.

## Budget

A namespace entry takes a `budget` with `daily_usd` and `monthly_usd`, the same fields the namespaces API takes as `spend_budget`. Omit it to use your organization's namespace default. A value above your organization's budget is set to the organization's budget, and apply reports the clamp in `warnings`.

```yaml theme={null}
namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1
    budget:
      daily_usd: 5
      monthly_usd: 100
```

See [Budgets](https://docs.mixpeek.com/docs/platform/budgets.md) for what happens when work reaches the limit.

## Alerts

An `alerts:` section creates alerts beside the retrievers they watch. An alert mirrors the create request of the alerts API, plus `namespace` and `retriever` by name.

| Field | Type | Default | What it does |
| - | - | - | - |
| `name` | string | required | Unique alert name |
| `namespace` | string | required | Namespace by name |
| `source` | `retriever` or `system` | `retriever` | Run a retriever, or watch a built-in condition |
| `retriever` | string | none | A retriever declared in this manifest, by name (`source: retriever`) |
| `retriever_id` | string | none | An existing retriever's id instead of `retriever`; set one of the two |
| `trigger_on` | `results` or `no_results` | `results` | When a retriever alert fires |
| `system_condition` | object | none | The built-in condition to watch (`source: system`) |
| `notification_config` | object | required | How and where to notify |
| `enabled` | boolean | `true` | Whether the alert runs |

```yaml theme={null}
alerts:
  - name: no_results_for_brand_queries
    namespace: video_search
    retriever: video_search_tool
    trigger_on: no_results
    notification_config:
      channels:
        - channel_type: email
          config:
            to_addresses: [ops@example.com]
```

Plan refuses an alert that does not map an input its retriever requires. The alert fields are described on [Alerts](https://docs.mixpeek.com/docs/enrichment/alerts.md).

## Track an apply

`apply` returns a `task_id` (absent on a dry run). `GET /v1/tasks/{task_id}` returns every resource of the plan with its states in order (`planned`, `creating`, `created`, `exists`, `updated`, or `failed` with the finding; a collection also passes `warming` and `ready`), each with the time it was reached. The same events arrive on the [change feed](https://docs.mixpeek.com/docs/operations/change-feed.md) and on a [webhook](https://docs.mixpeek.com/docs/operations/webhooks.md).

```bash theme={null}
curl https://api.mixpeek.com/v1/tasks/$TASK_ID \
  -H "Authorization: Bearer $API_KEY"
```

The apply response also carries `readiness`: for each collection whose extractor needs a real-time serving app, whether that app is running. An app scaled to zero is running; its first call answers 503 with `Retry-After` while it warms.

## Secret References

Use `${{ secrets.NAME }}` to reference organization secrets in the fields that hold credentials:

```yaml theme={null}
storage_connections:
  - name: external_data
    provider: s3
    config:
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
```

Apply fills in the value in a storage connection's `config`, an alert's `notification_config`, a webhook's `channels` and a bucket sync's `connection`. In a retriever, the reference stays a reference: apply writes it as `{{secrets.NAME}}`, which the stage resolves each time it runs, so the key is never stored on the retriever. A reference in any other field (a description, metadata, a collection or namespace setting) is refused, because those fields are stored as written and anyone with a read key can see them.

Secrets must exist before applying. Use the validate endpoint to check for missing secrets and misplaced references.

## Seed Data

A `seed:` section gives a namespace sample objects to answer its first query. After the resources are created, apply ingests them into the named bucket with one batch, bounded by `max_objects` (default 50, at most 500). Applying the manifest again finds that batch and ingests nothing more.

Each object names a source for each blob property under `blobs`. The blob type comes from the bucket schema property. A source takes one of three forms.

A Mixpeek sample key, from the sample set `samples/starter/<file>`:

```yaml theme={null}
seed:
  - bucket: ad-creatives
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}
```

A public `https://` URL (or `http://`, which object create also accepts). Mixpeek fetches it the way `POST /v1/buckets/{id}/objects` fetches a URL blob, with the same checks: the host must resolve to a public address, redirects are re-checked, and the response must be reachable and match the property's type.

```yaml theme={null}
seed:
  - bucket: ad-creatives
    objects:
      - blobs: {creative_asset_url: "https://cdn.example.com/ads/launch.mp4"}
```

One object in your own storage, given as `{connection, path}`. `connection` names a storage connection declared under `storage_connections` in the same manifest, or one that already exists in the organization. `path` is the object's key, written the way a bucket sync's `source_path` is. Apply reads the object through the connection with the same provider client a bucket sync uses and copies it into the bucket's storage. The connection's provider must be `s3`, `tigris`, `gcs`, `azure_blob`, `backblaze` or `supabase`; Google Drive and SharePoint name files by id and Snowflake yields rows, so those connections are refused here and belong in `bucket_syncs`.

```yaml theme={null}
storage_connections:
  - name: brand_s3
    provider: s3
    config:
      bucket_name: brand-assets
      region: us-east-1
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

seed:
  - bucket: ad-creatives
    objects:
      - blobs:
          creative_asset_url: {connection: brand_s3, path: ads/launch.mp4}
```

Validate and apply refuse the same seeds before anything is created: a connection that is not declared and does not exist, a connection whose provider cannot read one object by path (the error names the provider), a blob property the bucket schema does not declare, a URL that object create would refuse (no host, a private or internal address, unreachable, or the wrong content type), and any other `s3://`, `gs://` or `file://` string, since object storage is read only through a connection.

A seed's batch is billed to your organization when it is submitted, like any batch you create, whichever form its blobs take.

When the bucket declares other properties, set them on the object beside `blobs`, the way the object create API takes them:

```yaml theme={null}
buckets:
  - name: ad-creatives
    namespace: my_namespace
    schema:
      properties:
        creative_asset_url: {type: video}
        ad_id: {type: string, required: true}
        brand_slug: {type: string, required: true}

seed:
  - bucket: ad-creatives
    max_objects: 2
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}
        ad_id: sample-norelco
        brand_slug: norelco
```

Validate and apply check every object against the bucket's schema before anything is created. A missing required property, a property the bucket does not declare, a value of the wrong type, or a blob property given outside `blobs` is refused, and the error names the bucket, the object and the property.

### Seed or bucket sync

Use `seed` for a curated handful of objects, up to 500, that the first query should find. Use `bucket_syncs` with `sync_mode: initial_only` to import everything under a prefix of a storage connection once. The sync lists the prefix, applies `file_filters` and `schema_mapping`, and ingests the files in pages, with retries and a dead-letter queue for files that fail. `sync_mode: continuous` keeps polling for new files.

```yaml theme={null}
bucket_syncs:
  - name: brand-archive
    bucket: ad-creatives
    connection: brand_s3
    source_path: brand-archive/ads/2025/
    sync_mode: initial_only
    file_filters:
      include_patterns: ["*.mp4"]
```

`sync_mode` takes exactly `initial_only` or `continuous`; any other value is refused at validate.

For an `s3`, `tigris` or `backblaze` connection, `source_path` starts with the bucket: `brand-archive/ads/2025/` or `s3://brand-archive/ads/2025/`. The connection holds credentials, region and endpoint. Validate refuses a `bucket` key in such a connection's `config`, because the sync never reads it, and refuses a `source_path` that names no bucket.

## Taxonomy Labels

A flat taxonomy tags each document with its closest label. Write the labels inline on the taxonomy and name the feature to match on:

```yaml theme={null}
taxonomies:
  - name: creative-format
    namespace: ads
    type: flat
    match_on: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
    target_field: creative_format
    labels:
      - {label: testimonial, description: "A customer speaks to camera about results they saw."}
      - {label: tutorial, description: "A step by step routine that teaches while it sells."}

collections:
  - name: ad-scenes
    namespace: ads
    source: {type: bucket, bucket: ads}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {video: video}
      parameters: {run_multimodal_embedding: true, output_dimensionality: 1536}
    taxonomy_applications:
      - {taxonomy: creative-format, execution_mode: materialize}
```

| Field | Type | Default | What it does |
| - | - | - | - |
| `labels` | list | none | Each entry has `label` and an optional `description`. Any other key (a string, number or boolean) is stored on the label and written to the document as `<target_field>_<key>` |
| `match_on` | feature URI | inferred | The feature the classified collections write. The labels are embedded into the same feature. Optional when every collection applying the taxonomy writes exactly one feature that can embed text |
| `target_field` | string | taxonomy name, `-` as `_` | The document field the matched label is written to |

The manifest generates the resources the taxonomy runs on, and validate, the dry run and apply treat them like declared ones:

* a bucket `<taxonomy>-labels` with `content` (text) and `label` (string) properties
* a collection `<taxonomy>-labels` that embeds each label's text, `<label>: <description>`, with the extractor and feature `match_on` names. It copies `output_dimensionality` from the collections applying the taxonomy, so labels and documents share one vector space
* a retriever `<taxonomy>-lookup` that returns the single closest label for a document's vector
* a `seed:` entry for the label bucket, one object per label
* the taxonomy's `config`, wiring the retriever and the label collection, with `label` written to `target_field`

Each generated resource's description starts with `Generated for taxonomy '<name>'`, and each row of the dry run and apply response for one carries `generated_for: <taxonomy>`. A resource you declare under a generated name is refused; rename yours.

Validate refuses:

* labels on a taxonomy that is not flat
* a label given twice, or an empty label list
* labels together with `config.source_collection` or `config.retriever`
* a `match_on` whose extractor cannot compute that feature from text, such as a transcription embedding, which is computed from audio
* a collection applying the taxonomy that does not write the `match_on` feature

The older `type: flat` with `nodes` reads the same way: each node's `name` becomes a label and its `description` the label's description. It needs `match_on` unless it can be inferred as above, and is refused until it has one.

The seed runs once per label bucket, like any seed. To change the labels of a namespace that is already applied, add or edit objects in the `<taxonomy>-labels` bucket directly. An export writes the generated resources and the taxonomy config out in full, since the label text lives in the bucket's objects; the exported manifest applies as it is.

## Acceptance

A manifest can carry an `acceptance:` block: the steps that show its use case works and what each must return. Apply creates nothing from it. Apply runs the steps when you ask with `run_acceptance=true`, and Mixpeek's template acceptance harness runs them after every template apply.

| Field | Type | What it does |
| - | - | - |
| `steps` | list, at least one | The steps, run in order. Step names must be unique |
| `steps[].name` | string | Shown when the step fails |
| `steps[].kind` | `query`, `documents`, `alert` or `realtime` | What the step does |
| `steps[].retriever` | string | `query`: a retriever declared in this manifest |
| `steps[].collection` | string | `documents`: a collection declared in this manifest |
| `steps[].alert` | string | `alert`: an alert declared in this manifest |
| `steps[].feature_uri` | string | `realtime`: the `mixpeek://` feature URI to call |
| `steps[].inputs` | object | `query` and `realtime`: the inputs sent |
| `steps[].expect` | object | The outcome the step must produce, below |
| `max_cost_usd` | number | The most one acceptance run may cost, read from the meter |

Each kind reads its own `expect` fields:

| Kind | What runs | `expect` fields |
| - | - | - |
| `query` | the retriever, executed with `inputs` | `min_results`; `top_field` (a dotted path into the top result, such as `metadata.brand_slug`) with `top_equals` or `top_contains` |
| `documents` | the collection's document count | `min_documents` |
| `alert` | the alert's executions that triggered | `min_fires` |
| `realtime` | `POST /v1/inference` with `feature_uri` and `inputs` | `dims`, the length of the returned vector |

One step of each kind:

```yaml theme={null}
acceptance:
  max_cost_usd: 0.5
  steps:
    - name: seeded documents written
      kind: documents
      collection: transcripts
      expect: {min_documents: 2}
    - name: golden query
      kind: query
      retriever: video_search_tool
      inputs: {query: "electric shaver"}
      expect: {min_results: 1, top_field: metadata.brand_slug, top_equals: norelco}
    - name: shaver alert fires
      kind: alert
      alert: shaver_spotted
      expect: {min_fires: 1}
    - name: text embedding answers
      kind: realtime
      feature_uri: mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1
      inputs: {text: "electric shaver"}
      expect: {dims: 1024}
```

Validate refuses a step whose retriever, collection or alert this manifest does not declare, an `expect` field the step's kind does not read, and a step with no outcome, each naming `acceptance.steps.<name>`.

### Wait for the seed and run the steps on apply

Two query flags on `apply` make it wait until the namespace is usable and then prove it:

| Flag | What it does |
| - | - |
| `wait_for_seed=true` | After the resources exist, waits until every `seed:` batch has finished. The response lists each in `seed_batches` with its `status` and `documents_written`. A batch that failed, or did not finish in time, is also named in `errors` |
| `run_acceptance=true` | Implies `wait_for_seed`, and also waits for the serving apps listed in `readiness`. Then runs each step in order and returns the outcome in `acceptance`. A manifest with no `acceptance:` block is refused with 400 |

Both wait within `deadline_seconds`, or 900 seconds when you omit it. One clock covers the serving apps, the seed and the polling of `documents` and `alert` steps, which re-read until their count is reached or the time runs out.

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/manifest/apply?run_acceptance=true&deadline_seconds=900" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

```json theme={null}
{
  "success": true,
  "seed_batches": [
    {"bucket": "creative_assets", "batch_id": "btch_123", "status": "COMPLETED",
     "terminal": true, "usable": true, "documents_written": 4}
  ],
  "acceptance": {
    "passed": false,
    "steps": [
      {"name": "seeded documents written", "kind": "documents", "passed": true,
       "expected": {"min_documents": 2}, "observed": {"documents": 4}, "error": null},
      {"name": "golden query", "kind": "query", "passed": false,
       "expected": {"min_results": 1, "top_field": "metadata.brand_slug", "top_equals": "norelco"},
       "observed": {"results": 3, "top_value": "philips"},
       "error": "top metadata.brand_slug is 'philips', expected 'norelco'"},
      {"name": "max_cost_usd", "kind": "cost", "passed": true,
       "expected": {"max_cost_usd": 0.5}, "observed": {"spent_usd": 0.012, "metered_from": "namespace creation"}, "error": null}
    ]
  },
  "errors": ["acceptance failed: golden query (top metadata.brand_slug is 'philips', expected 'norelco')"]
}
```

How to read the result:

* `success` reports resource creation only, exactly as without the flags. Read `acceptance.passed` for the use case.
* A step passes only when it ran and every expected outcome held. A step that could not run is `passed: false` and its `error` starts with `not run:` and says why: a seed batch failed or was still processing at the deadline, a serving app was not running, the apply rolled back, or the cost cap was reached.
* A failed run is also named in `errors`, so a caller that only reads `errors` sees it.
* On a dry run both flags do nothing, and `warnings` says so.

**Billing.** Acceptance steps on `apply` are normal executions in your organization: each query and realtime call is billed like the same call made directly. `max_cost_usd` is read from your usage meter (the per-namespace credits in `GET /v1/organizations/billing/usage/breakdown`). Before each step after the first, a run that has reached the cap stops, and the remaining steps report `not run`. A closing `max_cost_usd` row compares the spend with the cap. When this apply created the namespace, the spend counts from its creation, seed ingest included. On a namespace that already existed it counts from the first step. Metering of very small usage can lag a few minutes, so the reading is a lower bound right after the run.

**Re-running the steps.** Re-apply the same manifest with `mode=create_missing&run_acceptance=true`. Every resource reads `exists`, nothing is created, the seed is not ingested again (its earlier batch is reused), and the steps run against the existing resources. Under the default `create_only` a re-apply is refused because the namespace exists, and the steps report `not run`.

## Dependency Resolution

Resources are created in topological order. The manifest engine:

* Resolves cross-resource references automatically
* Detects circular dependencies
* Rolls back all changes if any creation fails

## Authoring manifests with an LLM

An LLM can write a complete manifest on the first or second try if it reads what the API publishes and lets `validate` check the result. The loop has six steps.

1. **Read the schema.** `GET /v1/discovery/schema` returns:

   * `schemas`: a JSON schema for every section (`parameters`, `namespace`, `extractor`, `storage_connection`, `bucket`, `bucket_sync`, `collection`, `retriever`, `alert`, `cluster`, `taxonomy`, `trigger`, `webhook`, `seed`, `acceptance`) and for the whole `manifest`.
   * `dependency_graph`: for each type, the types whose names it references. A bucket sync needs a bucket and a storage connection, a seed needs a bucket, an acceptance step needs a retriever, collection or alert.
   * `examples`: a short YAML block for each type.
   * `example_manifest`: the complete manifest shown below.
   * `extractor_catalog`: every extractor and version this plane serves, with its input types, the feature URIs it writes and its parameters with defaults.
   * `stage_catalog`: every retriever `stage_id` with its category and parameter names.
   * `authoring_loop`: these steps in short form.

   Use `GET /v1/discovery/extractors` and `GET /v1/discovery/stages` for the full parameter schemas and stage examples.

2. **Write the manifest.** Reference resources by name. Copy feature URIs from `extractor_catalog`. A collection writes a feature URI only when the extractor parameters that produce it are on, for example `run_transcription_embedding: true` for the transcript embedding.

3. **Validate.** `POST /v1/manifest/validate` returns `valid`, `errors`, `warnings`, `missing_secrets`, `resources` (a count per type) and the declared `parameters`. Validate refuses what apply would refuse, so fix every entry in `errors` and rerun. Send parameter values in the `parameters` form field as a JSON object.

4. **Lint.** `POST /v1/manifest/lint` returns suggestions that each carry a code, a location, a suggestion and a `fix_example`. Three rules catch wiring that validates but cannot work: `UNUSED_BUCKET` (no collection reads the bucket), `TAXONOMY_INPUT_NOT_WRITTEN` (the taxonomy reads a feature that the collection it is applied to does not write) and `RETRIEVER_FEATURE_NOT_IN_COLLECTIONS` (the retriever searches a feature that none of its own collections write).

5. **Dry run.** `POST /v1/manifest/apply` with `dry_run=true` returns the plan: what would be created, in dependency order, and any refusal. Nothing is created.

6. **Apply.** `POST /v1/manifest/apply` creates the resources and returns the result, or the task to poll for progress.

To have the API write the first draft, call `POST /v1/manifest/generate` with a `description`. The model is given the live extractor and stage catalogs and the example below. The answer goes through the same checks as `validate`. If they find errors, the model is shown them and asked to fix its answer, up to two times. The response keeps `manifest`, `format`, `manifest_name` and `description`, and adds `validation` (`valid`, `errors`, `warnings`) and `repair_rounds`. When `validation.valid` is false the errors are still listed, so you can fix them yourself and continue from step 3.

### Complete example

This manifest syncs ad videos from S3 once, seeds a few sample objects, indexes the ads in two collections, searches them with a hybrid retriever, tags each scene with a creative format, groups scenes by look and states what a working setup returns. It validates once `ads_s3_bucket` is supplied and the secrets `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` exist.

```yaml theme={null}
version: "1.0"
metadata:
  name: ad-creative-search
  description: Search video ads by moment, tag each scene with a creative format, group scenes by look

parameters:
  - name: ads_s3_bucket
    type: string
    required: true
    description: Name of the S3 bucket that holds your ad videos
  - name: ads_prefix
    type: string
    default: ads/
    description: Folder inside that bucket to sync

namespaces:
  - name: ad-creative
    description: Ad creative search
    feature_extractors:
      - name: multimodal_extractor
        version: v2
    payload_indexes:
      - {field_name: metadata.brand, type: keyword}

storage_connections:
  - name: ads-s3
    provider: s3
    description: Read access to the bucket that holds the ad videos
    config:
      region: us-east-1
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

buckets:
  - name: ads
    namespace: ad-creative
    description: Video ads, plus the creative format labels (kind=label)
    schema:
      properties:
        video: {type: video}
        metadata: {type: object}
        content: {type: text}
        label: {type: string}
        kind: {type: string}

bucket_syncs:
  - name: ads-from-s3
    bucket: ads
    connection: ads-s3
    source_path: ${{ params.ads_s3_bucket }}/${{ params.ads_prefix }}
    sync_mode: initial_only
    file_filters:
      include_patterns: ["*.mp4", "*.mov"]

seed:
  - bucket: ads
    max_objects: 5
    objects:
      - blobs: {video: samples/starter/videos/norelco-speed.mp4}
        kind: ad
        metadata: {brand: norelco}
      - texts: {content: "A real customer speaks to camera about the results she saw."}
        kind: label
        label: testimonial
      - texts: {content: "Skin shown before use and after use, side by side."}
        kind: label
        label: before-and-after

collections:
  - name: creative-formats
    namespace: ad-creative
    description: Creative format labels, embedded in the same space as the ad scenes
    source:
      type: bucket
      bucket: ads
      filters: {field: kind, operator: eq, value: label}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {text: content}
      field_passthrough:
        - {source_path: label}
      parameters:
        run_multimodal_embedding: true
  - name: ad-scenes
    namespace: ad-creative
    description: Every ad split into scenes, with a visual embedding and a transcript embedding
    source:
      type: bucket
      bucket: ads
      filters: {field: kind, operator: ne, value: label}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {video: video}
      field_passthrough:
        - {source_path: metadata.brand}
      parameters:
        split_method: scene
        run_multimodal_embedding: true
        run_transcription: true
        run_transcription_embedding: true
    taxonomy_applications:
      - {taxonomy: creative-format, execution_mode: materialize}

retrievers:
  - name: format-lookup
    namespace: ad-creative
    description: Finds the closest creative format label for a scene. Used by the creative-format taxonomy.
    collections: [creative-formats]
    input_schema:
      query_embedding: {type: text, required: true}
    stages:
      - stage_name: match
        stage_type: filter
        config:
          stage_id: feature_search
          parameters:
            searches:
              - feature_uri: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
                query: {input_mode: vector, vector: "{{INPUT.query_embedding}}"}
                top_k: 1
            final_top_k: 1
  - name: ad-moment-search
    namespace: ad-creative
    description: Describe a moment and get the matching ad scenes
    collections: [ad-scenes]
    input_schema:
      query:
        type: text
        required: true
        description: Describe the moment, the claim or the format
    stages:
      - stage_name: recall
        stage_type: filter
        config:
          stage_id: feature_search
          parameters:
            searches:
              - feature_uri: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
                query: {input_mode: text, value: "{{INPUT.query}}"}
                top_k: 50
                weight: 1.0
              - feature_uri: "mixpeek://multimodal_extractor@v2/multilingual_e5_large_instruct_v1"
                query: {input_mode: text, value: "{{INPUT.query}}"}
                top_k: 50
                weight: 0.6
            final_top_k: 20

taxonomies:
  - name: creative-format
    namespace: ad-creative
    description: Tags every scene with its creative format
    type: flat
    config:
      taxonomy_type: flat
      retriever: format-lookup
      input_mappings:
        - {input_key: query_embedding, source_type: payload, path: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"}
      source_collection:
        collection: creative-formats
        enrichment_fields:
          - {field_path: label, target_field: creative_format, merge_mode: replace}

clusters:
  - name: visual-styles
    namespace: ad-creative
    description: Groups scenes by how they look
    source_collections: [ad-scenes]
    cluster_type: vector
    vector_config:
      feature_uris: ["mixpeek://multimodal_extractor@v2/gemini-embedding-2"]
      clustering_method: kmeans
      algorithm_params: {n_clusters: 6}
    llm_labeling:
      provider: google
      model_name: gemini-2.5-flash
      labeling_inputs:
        input_mappings:
          - {input_key: text, source_type: payload, path: transcription}
    enrich_source_collection: true

acceptance:
  max_cost_usd: 5
  steps:
    - name: scenes are indexed
      kind: documents
      collection: ad-scenes
      expect: {min_documents: 1}
    - name: texture query finds a scene
      kind: query
      retriever: ad-moment-search
      inputs: {query: close-up of foundation being blended on skin}
      expect: {min_results: 1}
```

## Workflows

### Environment Replication

```bash theme={null}
# Export from production
curl https://api.mixpeek.com/v1/manifest/export \
  -H "Authorization: Bearer $PROD_KEY" \
  -o prod-manifest.yaml

# Apply to staging
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $STAGING_KEY" \
  -F "manifest_file=@prod-manifest.yaml"
```

### Describe what is in a namespace

Add `detail=stats` to an export to see the data as well as the configuration. The YAML gets a read-only `status` section with each collection's document count, field types and most common field values, and each cluster's groups with their labels and sizes. Apply ignores `status`, so the file still applies as-is. `detail=stats` covers one namespace, so pass `namespace_id`.

```bash theme={null}
curl "https://api.mixpeek.com/v1/manifest/export?namespace_id=ns_abc123&detail=stats" \
  -H "Authorization: Bearer $API_KEY" \
  -o namespace-with-stats.yaml
```

Each stats read has a 2 second limit. A field or cluster whose read fails or times out shows `unavailable` with the reason, never an empty list.

### Pre-deployment Validation

```bash theme={null}
# Validate without applying
curl -X POST https://api.mixpeek.com/v1/manifest/validate \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'   # only if the manifest declares parameters

# Check what would change
curl -X POST https://api.mixpeek.com/v1/manifest/diff \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"
```

### CI/CD Integration

```yaml theme={null}
# GitHub Actions example
- name: Deploy Mixpeek Resources
  run: |
    curl -X POST https://api.mixpeek.com/v1/manifest/apply \
      -H "Authorization: Bearer ${{ secrets.MIXPEEK_API_KEY }}" \
      -F "manifest_file=@mixpeek.yaml"
```

## References

* [Apply Manifest](https://docs.mixpeek.com/docs/api-reference/manifest/apply-manifest.md)
* [Validate Manifest](https://docs.mixpeek.com/docs/api-reference/manifest/validate-manifest.md)
* [Export Manifest](https://docs.mixpeek.com/docs/api-reference/manifest/export-manifest.md)
* [Diff Manifest](https://docs.mixpeek.com/docs/api-reference/manifest/diff-manifest.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.