Bulk-submit a whole bucket as N auto-chunked, auto-queued batches

One call that streams the bucket’s objects (paginated, de-duplicated), chunks them server-side into chunk_size-object batches, and submits each (accept-and-queue). Replaces the client-side chunk loop + cursor pacing — no 409 cursor-overlap, no 5k count-timeout, no manual 429 handling. Poll the returned batch ids / batch_group_id for QUEUED -> PROCESSING.

POST /v1/buckets/{bucket_identifier}/batches/bulk-submit

Bulk-submit a whole bucket as N auto-chunked, auto-queued batches

cURL

curl --request POST \
  --url https://api.mixpeek.com/v1/buckets/{bucket_identifier}/batches/bulk-submit \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --header 'X-Namespace: <api-key>' \
  --data '
{
  "collection_ids": [
    "<string>"
  ],
  "filters": {
    "AND": [
      {
        "field": "name",
        "operator": "eq",
        "value": "John"
      },
      {
        "field": "age",
        "operator": "gte",
        "value": 30
      }
    ],
    "OR": [
      {
        "field": "status",
        "operator": "eq",
        "value": "active"
      },
      {
        "field": "role",
        "operator": "eq",
        "value": "admin"
      }
    ],
    "NOT": [
      {
        "field": "department",
        "operator": "eq",
        "value": "HR"
      },
      {
        "field": "location",
        "operator": "eq",
        "value": "remote"
      }
    ],
    "case_sensitive": true
  },
  "chunk_size": 1000,
  "dedup_strategy": "skip",
  "max_objects": 2
}
'

200

{
  "batch_group_id": "<string>",
  "total_objects": 123,
  "total_batches": 123,
  "submitted": 123,
  "failed": 123,
  "chunk_size": 123,
  "batches": [
    {
      "batch_id": "<string>",
      "status": "<string>",
      "object_count": 123,
      "queue_position": 123,
      "error": "<string>",
      "estimate": {
        "batch_id": "<string>",
        "collection_ids": [
          "<string>"
        ],
        "dedup_strategy": "skip",
        "object_count": 0,
        "already_extracted_count": 0,
        "estimated_credits": 0,
        "estimated_usd": 123,
        "estimated_repay_credits": 0,
        "features": [
          "<string>"
        ],
        "extractors": [
          "<string>"
        ],
        "content_summary": {},
        "documents_to_remove": 123,
        "warnings": [
          "<string>"
        ]
      }
    }
  ]
}

Authorizations

Authorization · string · header · required

Mixpeek API key, sent as Authorization: Bearer mxp_sk_.... Create one in Studio under Settings → API Keys, or with an admin key via POST /v1/organizations/users/{user_email}/api-keys. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403.

X-Namespace · string · header · required

Namespace id (ns_...), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked x-mixpeek-namespace-scoped.

Path Parameters

bucket_identifier · string · required

The bucket to ingest.

Body

application/json

Keys this body does not declare are refused with 422 extra_forbidden naming the accepted keys (since 2026-11-02).

Submit a WHOLE bucket as N auto-chunked, auto-queued batches in one call.

The server streams the bucket's objects (paginated), de-duplicates them, chunks them into chunk_size-object batches, and submits each — so the client makes ONE call instead of a paced chunk-loop. This kills the client-side cursor-overlap (409 storm), the 5k object-count timeout, and the no-whole-bucket frustration at once (phase 4). Each batch is accepted and QUEUED (accept-and-queue admission); poll the returned batch_group_id / batch ids for QUEUED -> PROCESSING.

collection_ids · string[] | null

Collections to process. Omit to auto-discover all collections sourced from this bucket (plus downstream dependencies).

filters · LogicalOperator · object | null

Optional filter to scope which objects are submitted. Omit to submit EVERY object in the bucket (the whole-bucket case).

Show child attributes

chunk_size · integer · default: 1000

Objects per batch. Clamped down to your tier's max_batch_size. Default 1000 — a good balance of parallelism and per-batch overhead. Sizing model: each batch runs as its own job whose workers scale at ~1 CPU worker per 500 objects up to a per-job worker ceiling, so 16k-20k objects saturates one job's parallelism — for large corpora (100k+ objects) prefer chunk_size 20000. Larger chunks don't run faster; they only raise the cost of a mid-run failure (progress is still resumable per-object via the processing ledger). Much smaller chunks pay one cluster cold-start each and queue behind your tier's concurrent-batch limit. Full guide: docs.mixpeek.com/operations/batch-ingestion-at-scale.

Required range: 1 <= x <= 50000

dedup_strategy · enum<string> · default: skip

How already-processed objects are handled (skip/replace/force).

Available options: skip, replace, force

max_objects · integer | null

Optional cap on total objects streamed (safety bound for very large buckets). Omit to submit the whole bucket.

Required range: x >= 1

Response

200

application/json

Successful Response

Result of a whole-bucket bulk submit.

batch_group_id · string · required

Correlates all batches created by this call.

total_objects · integer · required

Distinct objects streamed (post-dedup).

total_batches · integer · required

Batches created.

submitted · integer · required

Batches accepted/queued successfully.

failed · integer · required

Batches that failed admission/submit.

chunk_size · integer · required

Effective per-batch object count used.

batches · BulkSubmittedBatch · object[] · required

Per-batch results with status + queue position.

Show child attributes