Submit Batch for Processing

Submit a draft batch for asynchronous processing. The batch must be in ‘DRAFT’ status and contain objects.

POST /v1/buckets/{bucket_identifier}/batches/{batch_id}/submit

Submit Batch for Processing

cURL

curl --request POST \
  --url https://api.mixpeek.com/v1/buckets/{bucket_identifier}/batches/{batch_id}/submit \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --header 'X-Namespace: <api-key>' \
  --data '
{
  "description": "Submit batch with processing history (default)",
  "include_processing_history": true
}
'

200

{
  "additional_data": {
    "batch_id": "btch_xyz789",
    "bucket_id": "bkt_products",
    "collection_ids": [
      "col_tier0",
      "col_tier1",
      "col_tier2"
    ],
    "current_tier": 1,
    "job_id": "ray_job_123",
    "namespace_id": "ns_abc123",
    "object_count": 10000,
    "sample_object_ids": [
      "obj_001",
      "obj_002",
      "obj_003",
      "obj_004",
      "obj_005"
    ],
    "total_tiers": 3
  },
  "description": "Multi-tier batch processing task in progress (tier 1 of 3) with 10k objects",
  "inputs": [
    "batch_xyz789"
  ],
  "status": "IN_PROGRESS",
  "task_id": "2d322a05-3178-4eca-aac6-b82b0a0313aa",
  "task_type": "api_buckets_batches_process"
}

Authorizations

Authorization · string · header · required

Mixpeek API key, sent as Authorization: Bearer mxp_sk_.... Create one in Studio under Settings → API Keys, or with an admin key via POST /v1/organizations/users/{user_email}/api-keys. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403.

X-Namespace · string · header · required

Namespace id (ns_...), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked x-mixpeek-namespace-scoped.

Path Parameters

bucket_identifier · string · required

The unique identifier of the bucket.

batch_id · string · required

The unique identifier of the batch.

Query Parameters

force · boolean · default: false

Bypass the concurrent batch dedup check. Use when you intentionally want to reprocess the same bucket+collection combination while another batch is in flight.

Body

application/json

Keys this body does not declare are refused with 422 extra_forbidden naming the accepted keys (since 2026-11-02).

Request model for submitting a batch for processing.

This model allows configuration of processing behavior for the batch, such as whether to track processing history in document metadata.

Use Cases: - Submit batch with full audit trail (include_processing_history=True) - Submit batch without processing history for cleaner metadata (include_processing_history=False) - Default behavior includes processing history for debugging and lineage tracking

Requirements: - include_processing_history: OPTIONAL, defaults to True

include_processing_history · boolean · default: true

OPTIONAL (defaults to True). Controls whether processing operations are tracked in document internal_metadata.processing_history. When True: Each enrichment operation (taxonomy application, clustering, etc.) adds an audit trail entry. When False: Documents are enriched without processing history tracking, resulting in cleaner metadata. Use True for: Debugging, audit requirements, lineage tracking, understanding document transformations. Use False for: Production workloads where metadata size matters, simplified document structure. Processing history entries include: operation type, timestamp, and IDs of applied resources (taxonomies, clusters, etc.).

Examples:

true

false

collection_ids · string[] | null

DEPRECATED submit-time compatibility field. Collection scope is resolved when the batch is created via CreateBatchRequest.collection_ids. If provided at submit time, it must match the batch's tier-0 scope; otherwise submission fails closed instead of silently processing unexpected collections.

Example:

["col_text_only"]

webhook_url · string | null

OPTIONAL. URL to receive an HTTP POST notification when the batch reaches a terminal state (COMPLETED, FAILED, or CANCELED). The payload includes batch_id, status, failure_reason, documents_written, progress, and completed_at. The webhook is fire-and-forget: delivery failures are logged but do not affect batch processing. Must be an HTTPS or HTTP URL reachable from the server.

Example:

"https://example.com/webhooks/batch-complete"

Response

200

application/json

Successful Response

Task response model returned by the API.

Extends TaskModel with additional convenience fields for API responses. This is the model returned when you GET /v1/tasks/{task_id}.

Additional Fields: error_message: Convenience field that surfaces errors from additional_data for easier error handling in client code.

Inheritance: Inherits all fields and documentation from TaskModel, including: - task_id: Unique identifier - task_type: Operation type - status: Current status - inputs: Input parameters - outputs: Output results - additional_data: Metadata and context

Storage Architecture: Same as TaskModel - stored in Redis (24hr TTL) with MongoDB fallback.

Usage: This model is automatically returned by task API endpoints. You don't need to construct it manually - just call GET /v1/tasks/{task_id}.

Error Handling: Check the error_message field for a user-friendly error string, or additional_data['error'] for the full error details.

Example Response: { "task_id": "task_abc123", "task_type": "api_buckets_batches_process", "status": "FAILED", "inputs": ["batch_xyz"], "outputs": null, "additional_data": { "error": "Failed to process batch: Object not found", "batch_id": "batch_xyz" }, "error_message": "Failed to process batch: Object not found" }

task_id · string · required

Unique identifier for the task. REQUIRED. Used to poll task status via GET /v1/tasks/{task_id}. This ID is also stored on parent resources (batches, clusters, etc.) for cross-referencing. Format: UUID v4 or custom string identifier.

Examples:

"task_abc123def456"

"550e8400-e29b-41d4-a716-446655440000"

task_type · enum<string> · required

Type of operation this task represents. REQUIRED. Identifies the specific async operation being performed. Used for filtering and categorizing tasks. Common types: api_buckets_batches_process, engine_cluster_build, api_taxonomies_execute. See TaskType enum for complete list of supported operations.

Available options: api_namespaces_create, api_namespaces_delete, api_namespaces_snapshot_create, api_namespaces_snapshot_restore, api_namespaces_migrations_run, api_buckets_objects_create, api_buckets_delete, api_buckets_batches_process, api_buckets_batches_submit, api_buckets_uploads_create, api_buckets_uploads_confirm, api_buckets_uploads_batch_confirm, api_collections_documents_create, api_collections_documents_bulk_update, api_collections_documents_bulk_delete, api_collections_reembed, api_collections_extraction_artifacts, api_taxonomies_create, api_taxonomies_execute, api_taxonomies_materialize, api_evaluations_run, api_evaluations_dataset_create, api_retrievers_publish, api_manifest_apply, api_retrievers_execute, api_collections_export, api_collections_trigger, engine_feature_extractor_run, engine_inference_run, engine_object_processing, engine_cluster_build, thumbnail, video_segment, audio_segment, converted_video, materialize, plugin_custom, model_custom

Examples:

"api_buckets_batches_process"

"engine_cluster_build"

"api_taxonomies_execute"

status · enum<string> · required

Current status of the task. REQUIRED. Indicates the current state of the async operation. Terminal statuses (COMPLETED, FAILED, CANCELED) indicate the task has finished and will not change. Active statuses (PENDING, IN_PROGRESS, PROCESSING) indicate the task is still running and should be polled. Use this field to determine when to stop polling.

Available options: PENDING, QUEUED, IN_PROGRESS, PROCESSING, COMPLETED, COMPLETED_WITH_ERRORS, FAILED, CANCELED, INTERRUPTED, UNKNOWN, SKIPPED, DRAFT, ACTIVE, ARCHIVED, SUSPENDED, DEACTIVATED

Examples:

"PENDING"

"PROCESSING"

"COMPLETED"

"FAILED"

inputs · (string | object)[] | null

Input parameters or data used to start the task. OPTIONAL. May include IDs, configuration objects, or file references. Useful for debugging and understanding what data the task processed. Format: List of strings (IDs) or objects (configuration). Example: ['batch_id_123'] or [{'bucket_id': 'bkt_abc', 'config': {...}}]

Example:

["batch_xyz789"]

outputs · (string | object)[] | null

Output results produced by the task. OPTIONAL. Populated when task completes successfully. May include processed file IDs, result metrics, or status summaries. Check this field after task reaches COMPLETED status to get results. Format: List of strings (output IDs) or objects (result data).

Example:

["document_123", "document_456"]

additional_data · Additional Data · object | null

Additional metadata and context for the task. OPTIONAL. Contains job IDs, error details, progress info, and other task-specific metadata.

Common fields (all task types): - 'error': Error message if task failed - 'job_id': Ray job ID for engine tasks - 'from_mongodb': True if retrieved from MongoDB fallback (not Redis)

Batch-specific fields (task_type=api_buckets_batches_process): - 'batch_id': Batch identifier (REQUIRED) - 'bucket_id': Source bucket identifier (REQUIRED) - 'namespace_id': Namespace identifier (REQUIRED) - 'current_tier': Currently processing tier number, 0-indexed (OPTIONAL, None if not started) - 'total_tiers': Total number of tiers in the batch pipeline (REQUIRED) - 'collection_ids': Array of ALL collection IDs across all tiers (REQUIRED) - 'object_count': Number of objects being processed (REQUIRED) - 'sample_object_ids': First 5 object IDs for debugging/display (OPTIONAL)

Performance Note: Full object_ids array is NOT stored in task metadata to avoid bloating task documents (batches with 10k+ objects would add 200KB+ per task). For full object list, query the batch directly via GET /v1/buckets/{bucket_id}/batches/{batch_id}.

Note: For detailed per-tier status, use GET /v1/buckets/{bucket_id}/batches/{batch_id} to access the tier_tasks[] array which contains individual tier statuses, collection_ids, and timestamps for each tier.

Example:

{
  "batch_id": "btch_xyz789",
  "bucket_id": "bkt_products",
  "collection_ids": ["col_tier0", "col_tier1", "col_tier2"],
  "current_tier": 1,
  "job_id": "ray_job_123",
  "namespace_id": "ns_abc123",
  "object_count": 10000,
  "sample_object_ids": [
    "obj_001",
    "obj_002",
    "obj_003",
    "obj_004",
    "obj_005"
  ],
  "total_tiers": 3
}

error · string | null

Flattened error message for convenient error handling. OPTIONAL. Automatically populated from additional_data['error'] when the task has FAILED status. This is a convenience field - the full error details are always available in additional_data['error']. Use this field for displaying errors to users or logging. Will be None if task has not failed or if no error details are available. Serialized as 'error' in API responses for backward compatibility.

Example:

"Failed to process batch: Object not found"

cancellation · CancellationOutcome · object | null

Dispatch outcome of a cancel request, present on responses from the batch cancel endpoints. Absent on every other task response, and on a cancel that returned before any dispatch ran. Absent and all-zero are different answers: all-zero means the cancel dispatched to nothing.

Show child attributes

queue_position · integer | null

1-based position in the Ray processing waitlist. None if the batch was dispatched immediately (no queue). Position 1 means this batch will be processed next.

estimated_wait_minutes · number | null

Estimated minutes until this batch starts processing, based on queue position and average batch duration. None if the batch was dispatched immediately.