Get Batch by ID
Retrieve a single batch by its ID without requiring the bucket_id. Returns full batch details including real-time progress (objects_processed, total_objects, items_per_second, eta_seconds) for in-flight batches.
Authorizations
Mixpeek API key, sent as Authorization: Bearer mxp_sk_.... Create one in Studio under Settings → API Keys, or with an admin key via POST /v1/organizations/users/{user_email}/api-keys. A missing header returns 403; an invalid or revoked key returns 401.
Path Parameters
The unique identifier of the batch.
Response
Successful Response
Model representing a batch of objects for processing through collections.
A batch groups bucket objects together for processing through one or more collections. Batches support multi-tier processing where collections are processed in dependency order (e.g., bucket → chunks → frames → scenes). Each tier has independent task tracking.
Use Cases: - Process multiple objects through collections in a single batch - Track progress of multi-tier decomposition pipelines - Monitor and retry individual processing tiers - Query batch status and tier-specific task information
Lifecycle: 1. Created in DRAFT status with object_ids 2. Submitted for processing → status changes to PENDING 3. Each tier processes sequentially (tier 0 → tier 1 → ... → tier N) 4. Batch completes when all tiers finish (status=COMPLETED) or any tier fails (status=FAILED)
Multi-Tier Processing: - Tier 0: Bucket objects → Collections (bucket as source) - Tier N (N > 0): Collection documents → Collections (upstream collection as source) - Each tier gets independent task tracking via tier_tasks array - Processing proceeds tier-by-tier with automatic chaining
Requirements: - batch_id: OPTIONAL (auto-generated if not provided) - bucket_id: REQUIRED - status: OPTIONAL (defaults to DRAFT) - object_ids: REQUIRED for processing (must have at least 1 object) - collection_ids: OPTIONAL (discovered via DAG resolution) - tier_tasks: OPTIONAL (populated during processing) - current_tier: OPTIONAL (set during processing) - total_tiers: OPTIONAL (defaults to 1, set during DAG resolution) - dag_tiers: OPTIONAL (populated during DAG resolution)
REQUIRED. Unique identifier of the bucket containing the objects to process. Must be a valid bucket ID that exists in the system. All object_ids must belong to this bucket. Format: Bucket ID as defined when bucket was created.
"bkt_videos"
"bkt_documents_q4"
OPTIONAL (auto-generated if not provided). Unique identifier for this batch. Format: 'btch_' prefix followed by 12-character secure token. Generated using generate_secure_token() from shared.utilities.helpers. Used to query batch status and track processing across tiers. Immutable after creation.
"btch_abc123xyz789"
"btch_video_batch_01"
Namespace this batch belongs to. Stored at creation time.
OPTIONAL (defaults to DRAFT). Current processing status of the batch. Lifecycle: DRAFT → PENDING → IN_PROGRESS → COMPLETED/FAILED. DRAFT: Batch created but not yet submitted. PENDING: Batch submitted and queued for processing. IN_PROGRESS: Batch currently processing (one or more tiers active). COMPLETED: All tiers successfully completed. FAILED: One or more tiers failed. Aggregated from tier_tasks statuses during multi-tier processing.
PENDING, QUEUED, IN_PROGRESS, PROCESSING, COMPLETED, COMPLETED_WITH_ERRORS, FAILED, CANCELED, INTERRUPTED, UNKNOWN, SKIPPED, DRAFT, ACTIVE, ARCHIVED, SUSPENDED, DEACTIVATED "DRAFT"
"PENDING"
"IN_PROGRESS"
"COMPLETED"
"FAILED"
Populated on GET while the batch is PENDING/PROCESSING: how many earlier active batches are ahead in this namespace's queue and how long this batch has been waiting, so an in-flight status is never opaque.
List of object IDs to include in this batch. All objects must exist in the specified bucket_id. These objects are the source data for tier 0 processing. Collection-sourced batches may have empty object_ids. Objects are processed in parallel within each tier.
Controls how objects already processed in prior batches are handled. Scoped to (bucket, collection). 'skip': Don't reprocess objects that already have documents. 'replace': Delete existing documents and reprocess. 'force': Process regardless, allowing duplicates.
skip, replace, force key_id of the API key that submitted this batch, resolved server-side at creation. None for async/system-created batches with no request actor context. Audit only.
Display prefix of the submitting API key. Audit only.
Whether the submitting key was server-verified internal AT SUBMIT TIME. None = no actor context (unattributable); never used for billing decisions (those read the live key marker).
Per-collection dedup decisions. Keys are collection_ids; values contain dedup_strategy, total_input, skipped, processed, and skipped_object_ids (up to 1000). Written at TWO stages, deep-merged per collection: (1) the API at manifest build, when smart-skip enforcement excludes already-complete objects before any engine submission (fields prefixed manifest_*), and (2) the Engine after its resume filter runs on whatever residue was submitted.
OPTIONAL. List of all collection IDs involved in this batch's processing. Automatically populated during DAG resolution from dag_tiers. Includes collections from all tiers (flattened view of dag_tiers). Used for quick lookups without traversing tier structure. Format: List of collection IDs across all tiers.
OPTIONAL. Legacy error message field for backward compatibility. None if batch succeeded or is still processing. Contains human-readable error description from first failed tier. DEPRECATED: Use tier_tasks[].errors for detailed error information. For multi-tier batches, typically contains the error from the first failed tier. Check tier_tasks array for tier-specific error details and error_summary for aggregation.
"Failed to process batch: Object not found"
OPTIONAL. Human-readable explanation of why the batch failed. None if batch succeeded, is still processing, or is in DRAFT/PENDING state. Populated automatically when a batch transitions to FAILED status. Provides a concise, actionable summary of the root cause. Common reasons include: Ray job failure (spot preemption, OOM, code errors), 0 documents written (processing completed but produced no output), processing stall (no activity detected for extended period), or task exception (submission/validation failures). Use this field for user-facing error displays and alerting.
"Ray job failed: ImportError: No module named 'google.genai'"
OPTIONAL. Aggregated summary of errors across ALL tiers in the batch. Maps error_type (category) to total count of affected DOCUMENTS across all tiers. None has THREE causes, not two: the batch succeeded, it is still processing, OR a tier FAILED AT TIER LEVEL and wrote zero documents, so no individual document carries an error and this histogram is legitimately empty. So None does NOT mean 'no errors', and None alongside COMPLETED_WITH_ERRORS is NOT a contradiction — read failure_reason and tier_tasks[].error for tier-level diagnosis. Provides quick batch-wide overview of error distribution. Example: {'dependency': 15, 'authentication': 25, 'validation': 5} means across all tiers, 15 documents failed with dependency errors, 25 with auth errors, 5 with validation errors. Automatically aggregated from tier_tasks[].error_summary. Used for batch health dashboard and error trend analysis.
null
OPTIONAL. Machine-readable classification of the batch failure. None if batch succeeded or is still processing. Auto-derived from failure_reason if not explicitly set, so existing writers that only populate failure_reason still get a category on read. Categories: timeout (stall/no-progress), infrastructure (OOM, workers died, spot preemption), orphaned (RayJob CRD gone), pipeline (genuine extractor/pipeline failure), validation (submission/schema errors), unknown (uncategorized). Use this instead of parsing failure_reason.
timeout, infrastructure, orphaned, pipeline, validation, unknown null
OPTIONAL. List of per-object failure records from batch processing. Populated when individual objects fail while others succeed. Each record includes the object_id, error message, error classification (transient/permanent/resource), and timestamp. When this list is non-empty and some objects succeeded, batch status is COMPLETED_WITH_ERRORS. Enables targeted resubmission of only failed objects.
OPTIONAL. Count of objects that failed during batch processing. Shorthand for len(failed_objects). Stored separately for efficient queries and sorting without loading full failed_objects array.
x >= 00
3
15
BACKE-3079: objects that produced NEITHER a document NOR a failure entry — the per-tier balance audits' summed lost (submitted - processed - failed - skipped), rolled up at completion so the gap is STATED on the record instead of inferred by subtraction. None when every submitted object is accounted for; per-tier detail lives at tier_tasks[].audit.
x >= 038
OPTIONAL (defaults to BUCKET). Type of batch. BUCKET: Standard batch processing bucket objects through collections. COLLECTION: Reserved for future collection-only batch processing. Currently only BUCKET type is supported.
BUCKET, COLLECTION "BUCKET"
OPTIONAL. S3 key where the batch manifest is stored. Contains metadata and row data (Parquet) for Engine processing. For tier 0, points to bucket object manifest. For tier N+, points to collection document manifest. Format: S3 path (e.g., 'namespace_id/internal_id/manifests/tier_0.parquet'). Generated during batch submission.
"ns_abc/org_123/manifests/tier_0.parquet"
OPTIONAL. Primary task ID for the batch (typically tier 0 task). Used for backward compatibility with single-tier batch tracking. For multi-tier batches, prefer querying tier_tasks array for granular tracking. Format: Task ID as generated for tier 0.
"task_tier0_abc123"
OPTIONAL. List of object IDs that were successfully validated and loaded into the batch. Subset of object_ids that passed validation. Used to track which objects are ready for processing. None if batch hasn't been validated yet.
OPTIONAL. Internal engine/job metadata for system use. May contain: job_id (provider-specific), engine_version, processing hints, last_health_check. last_health_check: Most recent health check results with health_status, enriched_documents, vector_populated_count, stall_duration_seconds, recommendations, missing_features. Populated asynchronously (non-blocking, best-effort). Used for troubleshooting batch processing issues via API. NOTE: In MongoDB, this is stored under '_internal.processing' path.
OPTIONAL. User-defined metadata for the batch. Has typed fields (campaign_id, source, tags, notes) and also accepts arbitrary extra keys. Persisted with the batch and returned in API responses. Not used by the system for processing logic.
OPTIONAL. List of tier task tracking information for multi-tier processing. Each element represents one tier in the processing pipeline. Empty array for simple single-tier batches. Populated during batch submission with tier 0 info, then appended as tiers progress. Each TierTaskInfo contains: tier_num, task_id, status, collection_ids, timestamps. Used for granular monitoring: 'Show me status of tier 2' or 'Retry tier 1'. Array index typically matches tier_num (tier_tasks[0] = tier 0, tier_tasks[1] = tier 1, etc.).
OPTIONAL. Zero-based index of the currently processing tier. None if batch hasn't started processing (status=DRAFT or PENDING). Updated as batch progresses through tiers. Used to show processing progress: 'Processing tier 2 of 5'. Set to last tier number when batch completes. Example: If processing tier 1 (frames), current_tier=1.
x >= 00
OPTIONAL (defaults to 1). Total number of tiers in the collection DAG. Minimum 1 (tier 0 only = bucket → collection). Set during DAG resolution when batch is submitted. Equals len(dag_tiers) if dag_tiers is populated. Used to calculate progress: current_tier / total_tiers. Example: 5-tier pipeline (bucket → chunks → frames → scenes → summaries) has total_tiers=5.
x >= 11
3
5
OPTIONAL. Complete DAG tier structure for this batch. List of tiers, where each tier is a list of collection IDs to process at that stage. Tier 0 = bucket-sourced collections. Tier N (N > 0) = collection-sourced collections. Collections within same tier have no dependencies (can run in parallel). Collections in tier N+1 depend on collections in tier N. Populated during DAG resolution at batch submission. Used for tier-by-tier processing orchestration. Example: [['col_chunks'], ['col_frames', 'col_objects'], ['col_scenes']] = 3 tiers where frames and objects run in parallel at tier 1.
OPTIONAL (auto-set on creation). ISO 8601 timestamp when batch was created. Set using current_time() from shared.utilities.helpers. Immutable after creation. Used for batch age tracking and cleanup of old batches.
"2025-11-03T10:00:00Z"
OPTIONAL. Live progress snapshot updated approximately every 10 seconds while the batch is IN_PROGRESS. Written by the Ray ProgressActor inside the engine job. None when status is DRAFT or PENDING (job not started), or after COMPLETED/FAILED. Use this to show real-time progress bars: processed/total objects, percent complete, throughput (items_per_second), and estimated time remaining (eta_seconds).
null
OPTIONAL. Read-time aggregate of documents_written from tier_tasks and extractor_jobs. None means the completion callback has not reported write accounting yet.
null
OPTIONAL. Read-time aggregate of pages NOT indexed across extractor jobs — multi-page inputs (PDFs) capped by max_document_pages or dropped by per-page failures. Only present when > 0: a COMPLETED batch with pages_dropped > 0 indexed its inputs PARTIALLY. See pages_dropped_reasons; raise the collection's max_document_pages to index more pages.
null
OPTIONAL. Read-time aggregate of dropped-page counts by reason: max_document_pages_cap (input exceeded the collection's max_document_pages) or page_processing_failure (per-page extract/embed errors).
null
OPTIONAL. Read-time aggregate of video segments NOT indexed across extractor jobs — videos capped by max_video_segments. Only present when > 0: a COMPLETED batch with segments_dropped > 0 indexed its videos PARTIALLY. See segments_dropped_reasons; raise max_video_segments to cover more of each video.
null
OPTIONAL. Read-time aggregate of dropped-segment counts by reason: config_cap_max_video_segments (video exceeded the collection's max_video_segments).
null
Read-time diagnostics explaining terminal status, error indicators, and document-write accounting.
ALWAYS PRESENT (BACKE-2960). True only after the server CONFIRMED this terminal batch's just-written documents are actually queryable via the filtered document get path: a confirmation, never a prediction, timestamp, or elapsed time. A COMPLETED batch with documents_resolvable=false is still finalizing: its documents may 404 on direct GET even though they are durable (Studio renders status==COMPLETED && documents_resolvable==false as a 'Finalizing, indexing your results' state and keeps polling). Batches created before the feature shipped emit true (historical, long since resolvable); post-ship batches with no stored value emit false (fail toward not-ready). If the bounded post-completion confirm cannot succeed, documents_resolvable_error is set and this flag stays false.
Terminal failure signal for the bounded resolvability confirm (BACKE-2960/BACKE-2987). Set when the post-completion confirm loop could not confirm this batch's documents were queryable within its bound, the case where documents may NEVER become resolvable. When set, documents_resolvable stays false and will not flip: stop polling and surface a 'taking longer than expected' state instead of a spinner. Null while the confirm is pending or after it succeeded.
OPTIONAL. Computed health status for actively processing batches. Only populated when status is PROCESSING or IN_PROGRESS. Values: 'healthy' (recent activity detected), 'stalled' (no activity for 5+ minutes), 'unknown' (no heartbeat data yet). Computed from tier_tasks[].last_activity_at and updated_at. Use this to detect stuck batches before the internal stall detector kills them.
"healthy"
OPTIONAL. Dollar cost of this batch, derived at read time from the credits recorded against it in usage_records (credits_consumed * $0.001/credit). Lets callers answer 'what did this batch cost?' from the batch response without GCP billing labels. None when cost could not be looked up (e.g. the usage store was unavailable); 0 credits/USD when no usage has been recorded for the batch yet.
OPTIONAL. Timestamp of the most recent activity across all tier tasks. Aggregated from tier_tasks[].last_activity_at — the latest heartbeat from any tier. Updated approximately every 10 seconds by the BatchJobPoller while processing. A stale value (minutes old) while status is PROCESSING indicates the batch may be stalled. None for batches that have not started processing or have no heartbeat data.
OPTIONAL (defaults to 0). Number of times this batch has been auto-retried due to transient infrastructure failures (spot node preemption, OOM, actor death). Incremented each time the batch is automatically requeued after a retryable failure. User-facing: lets users see that retries happened transparently.
x >= 00
1
2
3
OPTIONAL (defaults to 3). Maximum number of automatic retries for transient failures. When retry_count reaches max_retries, the batch stays in FAILED state. Only transient/infrastructure failures trigger retries — validation and data errors do not.
x >= 03
5
OPTIONAL. ISO 8601 timestamp of the most recent auto-retry attempt. None if the batch has never been retried. Used to calculate exponential backoff for subsequent retries.
null
OPTIONAL. Human-readable reason for the most recent auto-retry. None if the batch has never been retried. Describes the transient failure that triggered the retry (e.g., 'Spot node preempted', 'Ray actor died', 'OOM killed').
null
OPTIONAL. URL to receive an HTTP POST notification when the batch reaches a terminal state (COMPLETED, FAILED, or CANCELED). Set at submit time via SubmitBatchRequest. The webhook is fire-and-forget: delivery failures are logged but never affect batch processing.
"https://example.com/webhooks/batch-complete"
OPTIONAL (auto-updated). ISO 8601 timestamp when batch was last modified. Updated using current_time() whenever batch status or tier_tasks change. Used to track batch activity and identify stale batches.
"2025-11-03T10:30:00Z"
COMPUTED. Human-readable description of the current batch state. Examples: 'Processing 724/50,000 objects (1.4%)', 'Queued — 2 batches ahead', 'Completed in 5m 23s', 'Loading model (stage 1/3)'. Computed on read, not stored in the database.
COMPUTED. Estimated completion timestamp based on current throughput. Derived from progress.eta_seconds + now. None if throughput data is unavailable. Computed on read, not stored in the database.