> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Bulk-submit a whole bucket as N auto-chunked, auto-queued batches

> One call that streams the bucket's objects (paginated, de-duplicated), chunks them server-side into `chunk_size`-object batches, and submits each (accept-and-queue). Replaces the client-side chunk loop + cursor pacing — no 409 cursor-overlap, no 5k count-timeout, no manual 429 handling. Poll the returned batch ids / `batch_group_id` for QUEUED -> PROCESSING.



## OpenAPI

````yaml post /v1/buckets/{bucket_identifier}/batches/bulk-submit
openapi: 3.1.0
info:
  title: Mixpeek API
  description: >-
    This is the Mixpeek API, providing access to various endpoints for data
    processing and retrieval.
  termsOfService: https://mixpeek.com/terms
  contact:
    name: Mixpeek Support
    url: https://mixpeek.com/contact
    email: info@mixpeek.com
  version: '0.82'
servers:
  - url: https://api.mixpeek.com
    description: Production
security:
  - BearerAuth: []
paths:
  /v1/buckets/{bucket_identifier}/batches/bulk-submit:
    post:
      tags:
        - Bucket Batches
      summary: Bulk-submit a whole bucket as N auto-chunked, auto-queued batches
      description: >-
        One call that streams the bucket's objects (paginated, de-duplicated),
        chunks them server-side into `chunk_size`-object batches, and submits
        each (accept-and-queue). Replaces the client-side chunk loop + cursor
        pacing — no 409 cursor-overlap, no 5k count-timeout, no manual 429
        handling. Poll the returned batch ids / `batch_group_id` for QUEUED ->
        PROCESSING.
      operationId: >-
        bulk_submit_batches_v1_buckets__bucket_identifier__batches_bulk_submit_post
      parameters:
        - name: bucket_identifier
          in: path
          required: true
          schema:
            type: string
            description: The bucket to ingest.
            title: Bucket Identifier
          description: The bucket to ingest.
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/BulkSubmitBatchesRequest'
        description: >-
          Keys this body does not declare are refused with 422 extra_forbidden
          naming the accepted keys (since 2026-11-02).
        x-unknown-keys:
          until: '2026-11-02'
          then: 422 extra_forbidden
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/BulkSubmitBatchesResponse'
        '400':
          description: Bad Request
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Unauthorized
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '403':
          description: Forbidden
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Not Found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Unprocessable Entity
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '429':
          description: Too Many Requests
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '500':
          description: Internal Server Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '503':
          description: Service Unavailable
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
      security:
        - BearerAuth: []
          NamespaceHeader: []
components:
  schemas:
    BulkSubmitBatchesRequest:
      properties:
        collection_ids:
          anyOf:
            - items:
                type: string
              type: array
            - type: 'null'
          title: Collection Ids
          description: >-
            Collections to process. Omit to auto-discover all collections
            sourced from this bucket (plus downstream dependencies).
        filters:
          anyOf:
            - $ref: '#/components/schemas/LogicalOperator-Input'
            - type: 'null'
          description: >-
            Optional filter to scope which objects are submitted. Omit to submit
            EVERY object in the bucket (the whole-bucket case).
        chunk_size:
          type: integer
          maximum: 50000
          minimum: 1
          title: Chunk Size
          description: >-
            Objects per batch. Clamped down to your tier's max_batch_size.
            Default 1000 — a good balance of parallelism and per-batch overhead.
            Sizing model: each batch runs as its own job whose workers scale at
            ~1 CPU worker per 500 objects up to a per-job worker ceiling, so
            16k-20k objects saturates one job's parallelism — for large corpora
            (100k+ objects) prefer chunk_size 20000. Larger chunks don't run
            faster; they only raise the cost of a mid-run failure (progress is
            still resumable per-object via the processing ledger). Much smaller
            chunks pay one cluster cold-start each and queue behind your tier's
            concurrent-batch limit. Full guide:
            docs.mixpeek.com/operations/batch-ingestion-at-scale.
          default: 1000
        dedup_strategy:
          $ref: '#/components/schemas/DedupStrategy'
          description: How already-processed objects are handled (skip/replace/force).
          default: skip
        max_objects:
          anyOf:
            - type: integer
              minimum: 1
            - type: 'null'
          title: Max Objects
          description: >-
            Optional cap on total objects streamed (safety bound for very large
            buckets). Omit to submit the whole bucket.
      type: object
      title: BulkSubmitBatchesRequest
      description: >-
        Submit a WHOLE bucket as N auto-chunked, auto-queued batches in one
        call.


        The server streams the bucket's objects (paginated), de-duplicates them,

        chunks them into ``chunk_size``-object batches, and submits each — so
        the

        client makes ONE call instead of a paced chunk-loop. This kills the

        client-side cursor-overlap (409 storm), the 5k object-count timeout, and
        the

        no-whole-bucket frustration at once (phase 4). Each batch is

        accepted and QUEUED (accept-and-queue admission); poll the returned

        ``batch_group_id`` / batch ids for QUEUED -> PROCESSING.
    BulkSubmitBatchesResponse:
      properties:
        batch_group_id:
          type: string
          title: Batch Group Id
          description: Correlates all batches created by this call.
        total_objects:
          type: integer
          title: Total Objects
          description: Distinct objects streamed (post-dedup).
        total_batches:
          type: integer
          title: Total Batches
          description: Batches created.
        submitted:
          type: integer
          title: Submitted
          description: Batches accepted/queued successfully.
        failed:
          type: integer
          title: Failed
          description: Batches that failed admission/submit.
        chunk_size:
          type: integer
          title: Chunk Size
          description: Effective per-batch object count used.
        batches:
          items:
            $ref: '#/components/schemas/BulkSubmittedBatch'
          type: array
          title: Batches
          description: Per-batch results with status + queue position.
      type: object
      required:
        - batch_group_id
        - total_objects
        - total_batches
        - submitted
        - failed
        - chunk_size
        - batches
      title: BulkSubmitBatchesResponse
      description: Result of a whole-bucket bulk submit.
    ErrorResponse:
      properties:
        success:
          type: boolean
          title: Success
          description: Always false for error responses
          default: false
        status:
          type: integer
          title: Status
          description: HTTP status code for this error
        error:
          $ref: '#/components/schemas/ErrorDetail'
          description: Error details payload
      type: object
      required:
        - status
        - error
      title: ErrorResponse
      description: Error response model.
      examples:
        - error:
            details:
              id: ns_123
              resource: namespace
            message: Namespace not found
            type: NotFoundError
          status: 404
          success: false
    LogicalOperator-Input:
      properties:
        AND:
          anyOf:
            - items:
                anyOf:
                  - $ref: '#/components/schemas/LogicalOperator-Input'
                  - $ref: '#/components/schemas/FilterCondition'
              type: array
            - type: 'null'
          title: And
          description: Logical AND operation - all conditions must be true
          example:
            - field: name
              operator: eq
              value: John
            - field: age
              operator: gte
              value: 30
        OR:
          anyOf:
            - items:
                anyOf:
                  - $ref: '#/components/schemas/LogicalOperator-Input'
                  - $ref: '#/components/schemas/FilterCondition'
              type: array
            - type: 'null'
          title: Or
          description: Logical OR operation - at least one condition must be true
          example:
            - field: status
              operator: eq
              value: active
            - field: role
              operator: eq
              value: admin
        NOT:
          anyOf:
            - items:
                anyOf:
                  - $ref: '#/components/schemas/LogicalOperator-Input'
                  - $ref: '#/components/schemas/FilterCondition'
              type: array
            - type: 'null'
          title: Not
          description: Logical NOT operation - all conditions must be false
          example:
            - field: department
              operator: eq
              value: HR
            - field: location
              operator: eq
              value: remote
        case_sensitive:
          anyOf:
            - type: boolean
            - type: 'null'
          title: Case Sensitive
          description: Whether to perform case-sensitive matching
          default: false
          example: true
      additionalProperties: true
      type: object
      title: LogicalOperator
      description: >-
        Represents a logical operation (AND, OR, NOT) on filter conditions.


        Allows nesting with a defined depth limit.


        Also supports shorthand syntax where field names can be passed directly

        as key-value pairs for equality filtering (e.g., {"metadata.title":
        "value"}).
    DedupStrategy:
      type: string
      enum:
        - skip
        - replace
        - force
      title: DedupStrategy
      description: |-
        Controls how duplicate objects are handled during batch processing.

        Dedup is scoped to (bucket_id, collection_id): an object is considered
        a duplicate if the target collection already has documents produced from
        the same source object in any prior batch.
    BulkSubmittedBatch:
      properties:
        batch_id:
          type: string
          title: Batch Id
        status:
          type: string
          title: Status
        object_count:
          type: integer
          title: Object Count
        queue_position:
          anyOf:
            - type: integer
            - type: 'null'
          title: Queue Position
        error:
          anyOf:
            - type: string
            - type: 'null'
          title: Error
          x-mixpeek-redacted: driver_text
        estimate:
          anyOf:
            - $ref: '#/components/schemas/BatchCostEstimate'
            - type: 'null'
          description: >-
            OPTIONAL. What this batch was estimated to cost when it was
            submitted: the same calculation submit bills by, read from the batch
            before it ran. Absent when the estimate could not be computed in
            time.
      type: object
      required:
        - batch_id
        - status
        - object_count
      title: BulkSubmittedBatch
      description: One batch in a bulk-submit group.
    ErrorDetail:
      properties:
        message:
          type: string
          title: Message
          description: Human-readable error message
        type:
          type: string
          title: Type
          description: Stable error type identifier (machine-readable)
        code:
          anyOf:
            - type: string
            - type: 'null'
          title: Code
          description: >-
            Fine-grained error code for programmatic handling (e.g.,
            namespace_name_taken, feature_extractor_not_found). Present only
            when consumers may need to branch on a specific error condition.
        details:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Details
          description: >-
            Optional structured details to help debugging (validation errors,
            IDs, etc.)
      type: object
      required:
        - message
        - type
      title: ErrorDetail
      description: Error detail model.
    FilterCondition:
      properties:
        field:
          type: string
          title: Field
          description: Field name to filter on
        operator:
          $ref: '#/components/schemas/FilterOperator'
          description: Comparison operator
          default: eq
        value:
          anyOf:
            - $ref: '#/components/schemas/DynamicValue'
            - {}
          title: Value
          description: Value to compare against
      type: object
      required:
        - field
        - value
      title: FilterCondition
      description: |-
        Represents a single filter condition.

        Attributes:
            field: The field to filter on
            operator: The comparison operator
            value: The value to compare against
    BatchCostEstimate:
      properties:
        batch_id:
          type: string
          title: Batch Id
        collection_ids:
          items:
            type: string
          type: array
          title: Collection Ids
        dedup_strategy:
          type: string
          title: Dedup Strategy
          description: skip | replace | force — controls re-extraction
          default: skip
        object_count:
          type: integer
          title: Object Count
          description: Objects this batch will process
          default: 0
        already_extracted_count:
          type: integer
          title: Already Extracted Count
          description: >-
            Objects in this batch already extracted before (re-paid under
            force/replace)
          default: 0
        estimated_credits:
          type: integer
          title: Estimated Credits
          description: Credits submitting this batch will consume (same calc as submit)
          default: 0
        estimated_usd:
          anyOf:
            - type: number
            - type: 'null'
          title: Estimated Usd
          description: >-
            Dollar cost of this batch under live v2 pricing (file type × amount
            × search-by features). Credits are milli-dollars of this value.
        estimated_repay_credits:
          type: integer
          title: Estimated Repay Credits
          description: >-
            ~credits re-paying already-extracted objects (force/replace only;
            linear approx)
          default: 0
        features:
          items:
            type: string
          type: array
          title: Features
          description: >-
            What this batch searches by (dimension 3 of pricing): feature keys
            billed per file type; custom plugins as 'custom:<name>'.
        extractors:
          items:
            type: string
          type: array
          title: Extractors
          description: >-
            Internal implementation of `features` — prefer rendering `features`;
            extractor names are the deeper disclosure layer.
        content_summary:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Content Summary
          description: >-
            Dimensions 1+2 of pricing: file types and how much of each (counts,
            video minutes, pages, tokens…)
        documents_to_remove:
          anyOf:
            - type: integer
            - type: 'null'
          title: Documents To Remove
          description: >-
            OPTIONAL. For dedup_strategy replace: the documents the batch's
            objects have now in its collections. A replace removes each of these
            that its re-run does not overwrite in place, after the new documents
            are written, so this is the most it removes. Absent for skip and
            force.
        warnings:
          items:
            type: string
          type: array
          title: Warnings
      type: object
      required:
        - batch_id
      title: BatchCostEstimate
      description: Dry-run estimate of what submitting a DRAFT batch will cost + re-pay.
    FilterOperator:
      description: >-
        Supported filter operators across database implementations.


        `text` is a case-insensitive substring match on the MVS vector store, in

        the in-memory evaluator and in the collections list filter; it does not

        tokenize or rank. On the Mongo query translator it becomes a Mongo
        `$text`

        search, which needs a collection text index and searches every indexed

        field. For whole-word matching on one field use `words`: every word of
        the

        value must appear as a whole word, compared on its English stem.
      enum:
        - eq
        - ne
        - gt
        - lt
        - gte
        - lte
        - in
        - nin
        - contains
        - starts_with
        - ends_with
        - regex
        - exists
        - is_null
        - text
        - phrase
        - words
        - geo_radius
        - geo_bounding_box
        - geo_polygon
      title: FilterOperator
      type: string
    DynamicValue:
      description: A value that should be dynamically resolved from the query request.
      properties:
        type:
          type: string
          const: dynamic
          title: Type
          default: dynamic
        field:
          type: string
          title: Field
          description: >-
            The dot-notation path to the value in the runtime query request,
            e.g., 'inputs.user_id'
          examples:
            - inputs.query_text
            - filters.AND[0].value
      required:
        - field
      title: DynamicValue
      type: object
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
      description: >-
        Mixpeek API key, sent as `Authorization: Bearer mxp_sk_...`. Create one
        in Studio under Settings → API Keys, or with an admin key via `POST
        /v1/organizations/users/{user_email}/api-keys`. A missing, malformed,
        unknown, revoked, inactive or expired key returns 401; a key without the
        permission, scope or plan an operation needs returns 403.
    NamespaceHeader:
      type: apiKey
      in: header
      name: X-Namespace
      description: >-
        Namespace id (`ns_...`), not the namespace name. This scopes the request
        rather than authenticating it, and it is required on every operation
        marked `x-mixpeek-namespace-scoped`.

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.