This endpoint partially updates a batch (PATCH operation).
Only provided fields will be updated. At minimum, metadata can always be updated.
Immutable fields like batch_id and timestamps cannot be modified.
Mixpeek API key, sent as Authorization: Bearer mxp_sk_.... Create one in Studio under Settings → API Keys, or with an admin key via POST /v1/organizations/users/{user_email}/api-keys. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403.
Namespace id (ns_...), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked x-mixpeek-namespace-scoped.
User-defined metadata for the batch. Has typed fields (campaign_id, source, tags, notes) and also accepts arbitrary extra keys.
Show child attributes
Response
Successful Response
Model representing a batch of objects for processing through collections.
A batch groups bucket objects together for processing through one or more collections.
Batches support multi-tier processing where collections are processed in dependency order
(e.g., bucket → chunks → frames → scenes). Each tier has independent task tracking.
Use Cases:
- Process multiple objects through collections in a single batch
- Track progress of multi-tier decomposition pipelines
- Monitor and retry individual processing tiers
- Query batch status and tier-specific task information
Lifecycle:
1. Created in DRAFT status with object_ids
2. Submitted for processing → status changes to PENDING
3. Each tier processes sequentially (tier 0 → tier 1 → ... → tier N)
4. Batch completes when all tiers finish (status=COMPLETED) or any tier fails (status=FAILED)
Multi-Tier Processing:
- Tier 0: Bucket objects → Collections (bucket as source)
- Tier N (N > 0): Collection documents → Collections (upstream collection as source)
- Each tier gets independent task tracking via tier_tasks array
- Processing proceeds tier-by-tier with automatic chaining
Requirements:
- batch_id: OPTIONAL (auto-generated if not provided)
- bucket_id: REQUIRED
- status: OPTIONAL (defaults to DRAFT)
- object_ids: REQUIRED for processing (must have at least 1 object)
- collection_ids: OPTIONAL (discovered via DAG resolution)
- tier_tasks: OPTIONAL (populated during processing)
- current_tier: OPTIONAL (set during processing)
- total_tiers: OPTIONAL (defaults to 1, set during DAG resolution)
- dag_tiers: OPTIONAL (populated during DAG resolution)
REQUIRED. Unique identifier of the bucket containing the objects to process. Must be a valid bucket ID that exists in the system. All object_ids must belong to this bucket. Format: Bucket ID as defined when bucket was created.
Most times Mixpeek resubmits a batch on its own after node loss. Once node_death_resubmit_count reaches this number, Mixpeek stops resubmitting and the batch needs a manual resubmit.
Most times Mixpeek resubmits a batch on its own after an engine deploy interrupted it. Once deploy_resubmit_count reaches this number, Mixpeek stops resubmitting and the batch needs a manual resubmit.
Where the batch is in its life, one word: 'accepted' (created, not submitted), 'queued' (submitted, waiting for a processing slot), 'admitted' (has a slot, the extraction job has not started), 'extracting' (the extraction job is running), 'indexed' (finished, documents written), 'failed' or 'canceled'. status alone reads PENDING for both queued and admitted batches; this field tells them apart. lifecycle_reason says why the batch is where it is.
Why the batch is in its current lifecycle state, in a sentence: the queue reason while queued, the failure reason once failed. Null when there is nothing to add.
Example:
"Waiting for a processing slot: 2 earlier batches ahead."
OPTIONAL (auto-generated if not provided). Unique identifier for this batch. Format: 'btch_' prefix followed by 12-character secure token. Generated using generate_secure_token() from shared.utilities.helpers. Used to query batch status and track processing across tiers. Immutable after creation.
OPTIONAL (defaults to DRAFT). Current processing status of the batch. Lifecycle: DRAFT → PENDING → IN_PROGRESS → COMPLETED/FAILED. DRAFT: Batch created but not yet submitted. PENDING: Batch submitted and queued for processing. IN_PROGRESS: Batch currently processing (one or more tiers active). COMPLETED: All tiers successfully completed. FAILED: One or more tiers failed. Aggregated from tier_tasks statuses during multi-tier processing.
The batch's wait for a processing slot. While the batch is PENDING/PROCESSING, a GET adds how many earlier active batches are ahead in this namespace's queue (batches_ahead) and its age (waiting_seconds). Once a batch that queued is admitted, the record keeps the wait for good: waited_seconds (total seconds spent queued, across tiers), reason (why it waited) and admitted_at; queued_at is set while it is waiting.
List of object IDs to include in this batch. All objects must exist in the specified bucket_id. These objects are the source data for tier 0 processing. Collection-sourced batches may have empty object_ids. Objects are processed in parallel within each tier.
Controls how objects already processed in prior batches are handled. Scoped to (bucket, collection). 'skip': Don't reprocess objects that already have documents. 'replace': Delete existing documents and reprocess. 'force': Process regardless, allowing duplicates.
The dedup strategy the batch actually ran with, when it differs from dedup_strategy. Set when the organization's spend policy blocks re-extraction, so a requested 'replace' or 'force' runs as 'skip'; progress.status_warnings carries the reason. Read-only, null when the requested strategy ran.
key_id of the API key that submitted this batch, resolved server-side at creation. None for async/system-created batches with no request actor context. Audit only.
Whether the submitting key was server-verified internal AT SUBMIT TIME. None = no actor context (unattributable); never used for billing decisions (those read the live key marker).
Per-collection dedup decisions. Keys are collection_ids; values contain dedup_strategy, total_input, skipped, processed, and skipped_object_ids (up to 1000). Written at TWO stages, deep-merged per collection: (1) the API at manifest build, when smart-skip enforcement excludes already-complete objects before any engine submission (fields prefixed manifest_*), and (2) the Engine after its resume filter runs on whatever residue was submitted.
OPTIONAL. List of all collection IDs involved in this batch's processing. Automatically populated during DAG resolution from dag_tiers. Includes collections from all tiers (flattened view of dag_tiers). Used for quick lookups without traversing tier structure. Format: List of collection IDs across all tiers.
OPTIONAL. Legacy error message field for backward compatibility. None if batch succeeded or is still processing. Contains human-readable error description from first failed tier. DEPRECATED: Use tier_tasks[].errors for detailed error information. For multi-tier batches, typically contains the error from the first failed tier. Check tier_tasks array for tier-specific error details and error_summary for aggregation.
OPTIONAL. Human-readable explanation of why the batch failed. None if batch succeeded, is still processing, or is in DRAFT/PENDING state. Populated automatically when a batch transitions to FAILED status. Provides a concise, actionable summary of the root cause. Common reasons include: Ray job failure (spot preemption, OOM, code errors), 0 documents written (processing completed but produced no output), processing stall (no activity detected for extended period), or task exception (submission/validation failures). Use this field for user-facing error displays and alerting.
Example:
"Ray job failed: ImportError: No module named 'google.genai'"
OPTIONAL. Aggregated summary of errors across ALL tiers in the batch. Maps error_type (category) to total count of affected DOCUMENTS across all tiers. None has THREE causes, not two: the batch succeeded, it is still processing, OR a tier FAILED AT TIER LEVEL and wrote zero documents, so no individual document carries an error and this histogram is legitimately empty. So None does NOT mean 'no errors', and None alongside COMPLETED_WITH_ERRORS is NOT a contradiction — read failure_reason and tier_tasks[].error for tier-level diagnosis. Provides quick batch-wide overview of error distribution. Example: {'dependency': 15, 'authentication': 25, 'validation': 5} means across all tiers, 15 documents failed with dependency errors, 25 with auth errors, 5 with validation errors. Automatically aggregated from tier_tasks[].error_summary. Used for batch health dashboard and error trend analysis.
OPTIONAL. Machine-readable classification of the batch failure. None if batch succeeded or is still processing. Auto-derived from failure_reason if not explicitly set, so existing writers that only populate failure_reason still get a category on read. Categories: timeout (stall/no-progress), infrastructure (OOM, workers died, spot preemption), orphaned (RayJob CRD gone), pipeline (genuine extractor/pipeline failure), validation (submission/schema errors), unknown (uncategorized). Use this instead of parsing failure_reason.
OPTIONAL. List of per-object failure records from batch processing. Populated when individual objects fail while others succeed. Each record includes the object_id, error message, error classification (transient/permanent/resource), and timestamp. When this list is non-empty and some objects succeeded, batch status is COMPLETED_WITH_ERRORS. Enables targeted resubmission of only failed objects.
OPTIONAL. Count of objects that failed during batch processing and wrote no documents (an object whose documents were written but one feature failed is in partial_objects instead). Shorthand for len(failed_objects). Stored separately for efficient queries and sorting without loading full failed_objects array.
Objects that wrote documents while one of their features failed, for example a transcription error on a video whose scenes, descriptions and embeddings were written. Each entry carries the object_id, the error and missing_feature, the feature the object has no documents for, or null when the error names none. An object with no document in one of the batch's collections also carries missing_collection_ids. These objects are searchable through the documents they wrote and are not counted in failed_object_count; they count in progress.errors.
Objects that produced NEITHER a document NOR a failure entry — the per-tier balance audits' summed lost (submitted - processed - failed - skipped), rolled up at completion so the gap is STATED on the record instead of inferred by subtraction. None when every submitted object is accounted for; per-tier detail lives at tier_tasks[].audit.
OPTIONAL (defaults to BUCKET). Type of batch. BUCKET: Standard batch processing bucket objects through collections. COLLECTION: Reserved for future collection-only batch processing. Currently only BUCKET type is supported.
OPTIONAL. S3 key where the batch manifest is stored. Contains metadata and row data (Parquet) for Engine processing. For tier 0, points to bucket object manifest. For tier N+, points to collection document manifest. Format: S3 path (e.g., 'namespace_id/internal_id/manifests/tier_0.parquet'). Generated during batch submission.
OPTIONAL. Primary task ID for the batch (typically tier 0 task). Used for backward compatibility with single-tier batch tracking. For multi-tier batches, prefer querying tier_tasks array for granular tracking. Format: Task ID as generated for tier 0.
OPTIONAL. List of object IDs that were successfully validated and loaded into the batch. Subset of object_ids that passed validation. Used to track which objects are ready for processing. None if batch hasn't been validated yet.
OPTIONAL. Internal engine/job metadata for system use. May contain: job_id (provider-specific), engine_version, processing hints, last_health_check. last_health_check: Most recent health check results with health_status, enriched_documents, vector_populated_count, stall_duration_seconds, recommendations, missing_features. Populated asynchronously (non-blocking, best-effort). Used for troubleshooting batch processing issues via API. NOTE: In MongoDB, this is stored under '_internal.processing' path.
OPTIONAL. User-defined metadata for the batch. Has typed fields (campaign_id, source, tags, notes) and also accepts arbitrary extra keys. Persisted with the batch and returned in API responses. Not used by the system for processing logic.
OPTIONAL. List of tier task tracking information for multi-tier processing. Each element represents one tier in the processing pipeline. Empty array for simple single-tier batches. Populated during batch submission with tier 0 info, then appended as tiers progress. Each TierTaskInfo contains: tier_num, task_id, status, collection_ids, timestamps. Used for granular monitoring: 'Show me status of tier 2' or 'Retry tier 1'. Array index typically matches tier_num (tier_tasks[0] = tier 0, tier_tasks[1] = tier 1, etc.).
OPTIONAL. Zero-based index of the currently processing tier. None if batch hasn't started processing (status=DRAFT or PENDING). Updated as batch progresses through tiers. Used to show processing progress: 'Processing tier 2 of 5'. Set to last tier number when batch completes. Example: If processing tier 1 (frames), current_tier=1.
OPTIONAL (defaults to 1). Total number of tiers in the collection DAG. Minimum 1 (tier 0 only = bucket → collection). Set during DAG resolution when batch is submitted. Equals len(dag_tiers) if dag_tiers is populated. Used to calculate progress: current_tier / total_tiers. Example: 5-tier pipeline (bucket → chunks → frames → scenes → summaries) has total_tiers=5.
OPTIONAL. Complete DAG tier structure for this batch. List of tiers, where each tier is a list of collection IDs to process at that stage. Tier 0 = bucket-sourced collections. Tier N (N > 0) = collection-sourced collections. Collections within same tier have no dependencies (can run in parallel). Collections in tier N+1 depend on collections in tier N. Populated during DAG resolution at batch submission. Used for tier-by-tier processing orchestration. Example: [['col_chunks'], ['col_frames', 'col_objects'], ['col_scenes']] = 3 tiers where frames and objects run in parallel at tier 1.
OPTIONAL (auto-set on creation). ISO 8601 timestamp when batch was created. Set using current_time() from shared.utilities.helpers. Immutable after creation. Used for batch age tracking and cleanup of old batches.
OPTIONAL. Live progress snapshot updated approximately every 10 seconds while the batch is IN_PROGRESS. Written by the Ray ProgressActor inside the engine job. None when status is DRAFT or PENDING (job not started), or after COMPLETED/FAILED. Use this to show real-time progress bars: processed/total objects, percent complete, throughput (items_per_second), and estimated time remaining (eta_seconds).
OPTIONAL. Read-time aggregate of documents_written from tier_tasks and extractor_jobs. None means the completion callback has not reported write accounting yet.
OPTIONAL. For dedup_strategy replace: documents this batch wrote for the objects it replaced, summed over its tiers. Absent when the batch is not a replace.
OPTIONAL. For dedup_strategy replace: the replaced objects' existing documents removed after the new ones were written, summed over its tiers. A failed or cancelled batch removes none. Absent when the batch is not a replace.
OPTIONAL. Read-time aggregate of pages NOT indexed across extractor jobs — multi-page inputs (PDFs) capped by max_document_pages or dropped by per-page failures. Only present when > 0: a COMPLETED batch with pages_dropped > 0 indexed its inputs PARTIALLY. See pages_dropped_reasons; raise the collection's max_document_pages to index more pages.
OPTIONAL. Read-time aggregate of dropped-page counts by reason: max_document_pages_cap (input exceeded the collection's max_document_pages) or page_processing_failure (per-page extract/embed errors).
OPTIONAL. Read-time aggregate of video segments NOT indexed across extractor jobs. Only present when > 0: a COMPLETED batch with segments_dropped > 0 indexed its videos PARTIALLY. segments_dropped_reasons says why: for config_cap_max_video_segments, raise max_video_segments to cover more of each video; segment_embedding_failure segments were not written because their embedding failed, which raising max_video_segments does not recover.
OPTIONAL. Read-time aggregate of dropped-segment counts by reason: config_cap_max_video_segments (video exceeded the collection's max_video_segments) or segment_embedding_failure (a segment's embedding failed, so the segment was not written).
ALWAYS PRESENT. True only after the server CONFIRMED this terminal batch's just-written documents are actually queryable via the filtered document get path: a confirmation, never a prediction, timestamp, or elapsed time. A COMPLETED batch with documents_resolvable=false is still finalizing: its documents may 404 on direct GET even though they are durable (Studio renders status==COMPLETED && documents_resolvable==false as a 'Finalizing, indexing your results' state and keeps polling). Batches created before the feature shipped emit true (historical, long since resolvable); post-ship batches with no stored value emit false (fail toward not-ready). If the bounded post-completion confirm cannot succeed, documents_resolvable_error is set and this flag stays false.
Terminal failure signal for the bounded resolvability confirm. Set when the post-completion confirm loop could not confirm this batch's documents were queryable within its bound, the case where documents may NEVER become resolvable. When set, documents_resolvable stays false and will not flip: stop polling and surface a 'taking longer than expected' state instead of a spinner. Null while the confirm is pending or after it succeeded.
OPTIONAL. Computed health status for actively processing batches. Only populated when status is PROCESSING or IN_PROGRESS. Values: 'healthy' (recent activity detected), 'stalled' (no activity for 5+ minutes), 'unknown' (no heartbeat data yet). Computed from tier_tasks[].last_activity_at and updated_at. Use this to detect stuck batches before the internal stall detector kills them.
OPTIONAL. Dollar cost of this batch, derived at read time from the credits recorded against it in usage_records (credits_consumed * $0.001/credit). Lets callers answer 'what did this batch cost?' from the batch response without GCP billing labels. None when cost could not be looked up (e.g. the usage store was unavailable); 0 credits/USD when no usage has been recorded for the batch yet.
OPTIONAL. Timestamp of the most recent activity across all tier tasks. Aggregated from tier_tasks[].last_activity_at — the latest heartbeat from any tier. Updated approximately every 10 seconds by the BatchJobPoller while processing. A stale value (minutes old) while status is PROCESSING indicates the batch may be stalled. None for batches that have not started processing or have no heartbeat data.
OPTIONAL (defaults to 0). Number of times this batch has been auto-retried due to transient infrastructure failures (spot node preemption, OOM, actor death). Incremented each time the batch is automatically requeued after a retryable failure. User-facing: lets users see that retries happened transparently.
OPTIONAL (defaults to 3). Maximum number of automatic retries for transient failures. When retry_count reaches max_retries, the batch stays in FAILED state. Only transient/infrastructure failures trigger retries — validation and data errors do not.
OPTIONAL. ISO 8601 timestamp of the most recent auto-retry attempt. None if the batch has never been retried. Used to calculate exponential backoff for subsequent retries.
OPTIONAL. Human-readable reason for the most recent auto-retry. None if the batch has never been retried. Describes the transient failure that triggered the retry (e.g., 'Spot node preempted', 'Ray actor died', 'OOM killed').
OPTIONAL (defaults to 0). Number of times Mixpeek resubmitted this batch on its own after its job was lost to node loss (a spot preemption, a node that died, or the job reaped for its resources). Each resubmit resumes from the objects already finished, so completed work is not re-extracted. Compare with max_node_death_resubmits to see how many automatic resubmits remain.
OPTIONAL (defaults to 0). Number of times Mixpeek resubmitted this batch on its own after an engine deploy interrupted its job. Compare with max_deploy_resubmits to see how many automatic resubmits remain.
OPTIONAL. Set the first time this batch's Ray submission hit a quota-exhaustion or engine-deploy-lock gate (K8s admission 429/503), and cleared once a submission succeeds or the block's requeue window expires. Distinct from retry_count/last_retry_at above, which track the transient-infrastructure-failure retry mechanism (worker died, OOM, spot preemption) rather than this pre-submission gate. Used to bound the requeue loop to a fixed wall-clock window instead of retrying indefinitely.
OPTIONAL. URL to receive an HTTP POST notification when the batch reaches a terminal state (COMPLETED, FAILED, or CANCELED). Set at submit time via SubmitBatchRequest. The webhook is fire-and-forget: delivery failures are logged but never affect batch processing.
OPTIONAL (auto-updated). ISO 8601 timestamp when batch was last modified. Updated using current_time() whenever batch status or tier_tasks change. Used to track batch activity and identify stale batches.
COMPUTED. Human-readable description of the current batch state. Examples: 'Processing 724/50,000 objects (1.4%)', 'Queued — 2 batches ahead', 'Completed in 5m 23s', 'Loading model (stage 1/3)'. Computed on read, not stored in the database.
COMPUTED. Estimated completion timestamp based on current throughput. Derived from progress.eta_seconds + now. None if throughput data is unavailable. Computed on read, not stored in the database.
Was this page helpful?
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Mixpeek","colors":{"primary":"#FC5185","light":"#FFA0BD","dark":"#364F6B"},"logo":{"light":"https://mintcdn.com/mixpeek/QQyAGObvvz8PjpE0/logo/light.png?fit=max\u0026auto=format\u0026n=QQyAGObvvz8PjpE0\u0026q=85\u0026s=ac5ab550180057404fe833886513775f","dark":"https://mintcdn.com/mixpeek/QQyAGObvvz8PjpE0/logo/dark.png?fit=max\u0026auto=format\u0026n=QQyAGObvvz8PjpE0\u0026q=85\u0026s=e18979c495dbd68be4fc44473098773e"},"favicon":"https://mintcdn.com/mixpeek/QQyAGObvvz8PjpE0/favicon.png?fit=max\u0026auto=format\u0026n=QQyAGObvvz8PjpE0\u0026q=85\u0026s=8be5d2f6dc7e4cb44127a0493d57cde1","api":{"openapi":"api-reference/openapi.json","mdx":{"auth":{"method":"bearer","name":"Authorization"},"server":["https://api.mixpeek.com"]}},"background":{"image":"https://mintcdn.com/mixpeek/QQyAGObvvz8PjpE0/images/background.png?fit=max\u0026auto=format\u0026n=QQyAGObvvz8PjpE0\u0026q=85\u0026s=a68d0aa9f5eba448bcf67f5f410a6c90","color":{"dark":"#121212"}},"navbar":{"links":[{"href":"https://mixpeek.com/contact","label":"Talk to Engineers"}],"primary":{"type":"button","label":"Use Mixpeek","href":"https://studio.mixpeek.com/?utm_source=docs\u0026utm_medium=docs_navbar\u0026utm_campaign=mvs_onramp"}},"footer":{"socials":{"website":"https://mixpeek.com/","github":"https://github.com/mixpeek"}},"integrations":{"posthog":{"apiKey":"phc_x1G3IrW97qXoodTPWbyBK8dbZ3H7x5k5aZPCXmRvuWw","apiHost":"https://us.i.posthog.com"}},"contextual":{"options":["view","claude","chatgpt","cursor","vscode"],"display":"header"},"styling":{"latex":true},"navigation":{"global":{"anchors":[{"anchor":"Homepage","icon":"house","href":"https://mixpeek.com"},{"anchor":"Engineering Blog","icon":"newspaper","href":"https://mixpeek.com/blog"},{"anchor":"API Swagger","icon":"file-code","href":"https://api.mixpeek.com/docs"}]},"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"mixpeek","hasAudioNarration":false,"actualSubdomain":"mixpeek","customDomains":["docs.mixpeek.com"],"gitSource":{"type":"github","owner":"mixpeek","repo":"mixpeek","deployBranch":"main","contentDirectory":"docs","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true,"edits":false,"issues":false},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6aeff52f552e9ea17aa8eeb5","clientVersion":"0.0.4087","searchFilterCounts":[{"language":"default","tag":"API Reference","count":536},{"language":"default","tag":"Documentation","count":195},{"language":"default","tag":"Advanced","count":140},{"language":"default","tag":"Administer","count":77},{"language":"default","tag":"Buckets","count":70},{"language":"default","tag":"Organization","count":62},{"language":"default","tag":"Retrievers","count":61},{"language":"default","tag":"Integrate \u0026 operate","count":59},{"language":"default","tag":"Build retrievers","count":55},{"language":"default","tag":"Apps","count":53}]},"assistantConfig":{"deflection":{"enabled":true,"email":"ethan@mixpeek.com"}},"basePath":"/docs","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"comment-mode","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[],"page":{"pagePath":"api-reference/bucket-batches/partially-update-batch","slug":"/api-reference/bucket-batches/partially-update-batch","pageMetadata":{"openapi":"patch /v1/buckets/{bucket_identifier}/batches/{batch_id}","title":"Partially Update Batch","description":"This endpoint partially updates a batch (PATCH operation).\n Only provided fields will be updated. At minimum, metadata can always be updated.\n Immutable fields like batch_id and timestamps cannot be modified.","href":"/api-reference/bucket-batches/partially-update-batch"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"This endpoint partially updates a batch (PATCH operation).\\nOnly provided fields will be updated. At minimum, metadata can always be updated.\\nImmutable fields like batch_id and timestamps cannot be modified.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){return _jsx(_Fragment,{})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"openapi":"patch /v1/buckets/{bucket_identifier}/batches/{batch_id}","title":"Partially Update Batch","description":"This endpoint partially updates a batch (PATCH operation).\n Only provided fields will be updated. At minimum, metadata can always be updated.\n Immutable fields like batch_id and timestamps cannot be modified.","href":"/api-reference/bucket-batches/partially-update-batch"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[],"codeExamples":{}},"jsLinks":["/docs/_mintlify/custom/mixpeek/fd427c04b3bf84bb/custom.js","/docs/_mintlify/custom/mixpeek/e878da59c725dc0d/scripts/test_first_touch.js"],"openApiReferenceData":{"operation":{"tags":["Bucket Batches"],"summary":"Partially Update Batch","description":"This endpoint partially updates a batch (PATCH operation).\n Only provided fields will be updated. At minimum, metadata can always be updated.\n Immutable fields like batch_id and timestamps cannot be modified.","operationId":"patch_batch_v1_buckets__bucket_identifier__batches__batch_id__patch","parameters":["748503ce-b1d9-4715-b395-9629ddccf4cf","6ec619ce-0cf6-4cc8-a6e6-a72c00c238d0"],"requestBody":"33d01e24-30b4-43df-bebd-42bb4ac6179a","responses":{"200":"3cb63586-64bd-49d4-b1ba-fa4d77d6f7a0","400":"77626a0a-930d-4d53-9c2c-f349dc2b08f0","401":"94a84887-1a34-44f9-b7be-df47da91d6a2","403":"0ceb9ddc-4a80-4cb8-a366-366b1f0116a1","404":"d0a731f3-948c-42fd-9abf-e12600566965","422":"6a62aeb5-4cc7-408b-b2d9-e3191b75113f","429":"ef7ab2f0-842c-49aa-b34f-1162ba929db3","500":"d51fa5be-7e9d-4afe-9569-b200abfcb0a0","503":"b6bec9b4-6471-4d9d-8b75-0014e936673d"},"x-mixpeek-plan":{"minimum_tier":null,"requires_selected_plan":false,"permissions":["write"]},"x-mixpeek-namespace-scoped":true,"security":["ef19c76f-e833-4f86-97f8-9ef3a310e317"],"method":"patch","type":"path","servers":["a3ca4d02-9041-4306-8eae-afdf7d554be4"],"path":"/v1/buckets/{bucket_identifier}/batches/{batch_id}","baseUrlOptions":["https://api.mixpeek.com"],"title":"Partially Update Batch","codeSamples":[],"requiredOnlyExamples":false,"requestExampleType":"generated","responseExampleType":"generated"},"dependencies":{"parameters":{"748503ce-b1d9-4715-b395-9629ddccf4cf":{"name":"bucket_identifier","in":"path","required":true,"schema":{"type":"string","description":"The unique identifier of the bucket.","title":"Bucket Identifier","isRequired":true,"uniqueKey":"","typeLabel":"string"},"description":"The unique identifier of the bucket."},"6ec619ce-0cf6-4cc8-a6e6-a72c00c238d0":{"name":"batch_id","in":"path","required":true,"schema":{"type":"string","description":"The unique identifier of the batch.","title":"Batch Id","isRequired":true,"uniqueKey":"","typeLabel":"string"},"description":"The unique identifier of the batch."}},"requestBody":{"required":true,"content":{"application/json":{"schema":{"properties":{"metadata":{"oneOf":[{"type":"object","title":"BatchMetadata","description":"User-defined metadata for the batch. Has typed fields (campaign_id, source, tags, notes) and also accepts arbitrary extra keys.","properties":{"campaign_id":{"oneOf":[{"type":"string","title":"Campaign Id","description":"Identifier linking this batch to a marketing or processing campaign.","example":"Q4_2025","uniqueKey":"metadata.oneOf.0.campaign_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Campaign Id","description":"Identifier linking this batch to a marketing or processing campaign.","example":"Q4_2025","uniqueKey":"metadata.oneOf.0.campaign_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.oneOf.0.campaign_id","typeLabel":"string | null"},"source":{"oneOf":[{"type":"string","title":"Source","description":"Origin of the batch data (e.g. an S3 prefix, partner name, or pipeline stage).","example":"s3://raw-uploads/2026-05/","uniqueKey":"metadata.oneOf.0.source.oneOf.0","typeLabel":"string"},{"type":"null","title":"Source","description":"Origin of the batch data (e.g. an S3 prefix, partner name, or pipeline stage).","example":"s3://raw-uploads/2026-05/","uniqueKey":"metadata.oneOf.0.source.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.oneOf.0.source","typeLabel":"string | null"},"tags":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"metadata.oneOf.0.tags.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Tags","description":"Free-form tags for filtering and grouping batches.","example":["video","high-priority"],"uniqueKey":"metadata.oneOf.0.tags.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Tags","description":"Free-form tags for filtering and grouping batches.","example":["video","high-priority"],"uniqueKey":"metadata.oneOf.0.tags.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.oneOf.0.tags","typeLabel":"string[] | null"},"notes":{"oneOf":[{"type":"string","title":"Notes","description":"Free-form notes about this batch (intent, context, special handling).","example":"Re-run after Whisper quota fix","uniqueKey":"metadata.oneOf.0.notes.oneOf.0","typeLabel":"string"},{"type":"null","title":"Notes","description":"Free-form notes about this batch (intent, context, special handling).","example":"Re-run after Whisper quota fix","uniqueKey":"metadata.oneOf.0.notes.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.oneOf.0.notes","typeLabel":"string | null"}},"additionalProperties":true,"uniqueKey":"metadata.oneOf.0","typeLabel":"BatchMetadata · object"},{"type":"null","description":"User-defined metadata for the batch. Has typed fields (campaign_id, source, tags, notes) and also accepts arbitrary extra keys.","uniqueKey":"metadata.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata","typeLabel":"BatchMetadata · object | null"}},"type":"object","title":"PatchBatchRequest","description":"Request model for partially updating a batch (PATCH operation).","uniqueKey":"","typeLabel":"PatchBatchRequest · object"},"examples":{"Example":{"title":"Example","value":{"metadata":{"campaign_id":"Q4_2025","source":"s3://raw-uploads/2026-05/","tags":["video","high-priority"],"notes":"Re-run after Whisper quota fix"}}}},"exampleType":"generated"}},"description":"Keys this body does not declare are refused with 422 extra_forbidden naming the accepted keys (since 2026-11-02).","x-unknown-keys":{"until":"2026-11-02","then":"422 extra_forbidden"},"exampleType":"generated"},"responses":{"3cb63586-64bd-49d4-b1ba-fa4d77d6f7a0":{"description":"Successful Response","content":{"application/json":{"schema":{"properties":{"bucket_id":{"type":"string","title":"Bucket Id","description":"REQUIRED. Unique identifier of the bucket containing the objects to process. Must be a valid bucket ID that exists in the system. All object_ids must belong to this bucket. Format: Bucket ID as defined when bucket was created.","examples":["bkt_videos","bkt_documents_q4"],"isRequired":true,"uniqueKey":"bucket_id","typeLabel":"string"},"max_node_death_resubmits":{"type":"integer","title":"Max Node Death Resubmits","description":"Most times Mixpeek resubmits a batch on its own after node loss. Once node_death_resubmit_count reaches this number, Mixpeek stops resubmitting and the batch needs a manual resubmit.","readOnly":true,"examples":[3],"isRequired":true,"uniqueKey":"max_node_death_resubmits","typeLabel":"integer"},"max_deploy_resubmits":{"type":"integer","title":"Max Deploy Resubmits","description":"Most times Mixpeek resubmits a batch on its own after an engine deploy interrupted it. Once deploy_resubmit_count reaches this number, Mixpeek stops resubmitting and the batch needs a manual resubmit.","readOnly":true,"examples":[2],"isRequired":true,"uniqueKey":"max_deploy_resubmits","typeLabel":"integer"},"lifecycle":{"oneOf":[{"type":"string","title":"Lifecycle","description":"Where the batch is in its life, one word: 'accepted' (created, not submitted), 'queued' (submitted, waiting for a processing slot), 'admitted' (has a slot, the extraction job has not started), 'extracting' (the extraction job is running), 'indexed' (finished, documents written), 'failed' or 'canceled'. status alone reads PENDING for both queued and admitted batches; this field tells them apart. lifecycle_reason says why the batch is where it is.","example":"queued","isRequired":true,"uniqueKey":"lifecycle.oneOf.0","typeLabel":"string"},{"type":"null","title":"Lifecycle","description":"Where the batch is in its life, one word: 'accepted' (created, not submitted), 'queued' (submitted, waiting for a processing slot), 'admitted' (has a slot, the extraction job has not started), 'extracting' (the extraction job is running), 'indexed' (finished, documents written), 'failed' or 'canceled'. status alone reads PENDING for both queued and admitted batches; this field tells them apart. lifecycle_reason says why the batch is where it is.","example":"queued","isRequired":true,"uniqueKey":"lifecycle.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"lifecycle","typeLabel":"string | null"},"lifecycle_reason":{"oneOf":[{"type":"string","title":"Lifecycle Reason","description":"Why the batch is in its current lifecycle state, in a sentence: the queue reason while queued, the failure reason once failed. Null when there is nothing to add.","example":"Waiting for a processing slot: 2 earlier batches ahead.","isRequired":true,"uniqueKey":"lifecycle_reason.oneOf.0","typeLabel":"string"},{"type":"null","title":"Lifecycle Reason","description":"Why the batch is in its current lifecycle state, in a sentence: the queue reason while queued, the failure reason once failed. Null when there is nothing to add.","example":"Waiting for a processing slot: 2 earlier batches ahead.","isRequired":true,"uniqueKey":"lifecycle_reason.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"lifecycle_reason","typeLabel":"string | null"},"batch_id":{"type":"string","title":"Batch Id","description":"OPTIONAL (auto-generated if not provided). Unique identifier for this batch. Format: 'btch_' prefix followed by 12-character secure token. Generated using generate_secure_token() from shared.utilities.helpers. Used to query batch status and track processing across tiers. Immutable after creation.","examples":["btch_abc123xyz789","btch_video_batch_01"],"uniqueKey":"batch_id","typeLabel":"string"},"namespace_id":{"oneOf":[{"type":"string","title":"Namespace Id","description":"Namespace this batch belongs to. Stored at creation time.","uniqueKey":"namespace_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Namespace Id","description":"Namespace this batch belongs to. Stored at creation time.","uniqueKey":"namespace_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"namespace_id","typeLabel":"string | null"},"batch_name":{"oneOf":[{"type":"string","title":"Batch Name","description":"OPTIONAL. Human-readable name given when the batch was created. Not unique; batch_id identifies the batch.","maxLength":256,"example":"january_products","uniqueKey":"batch_name.oneOf.0","typeLabel":"string"},{"type":"null","title":"Batch Name","description":"OPTIONAL. Human-readable name given when the batch was created. Not unique; batch_id identifies the batch.","example":"january_products","uniqueKey":"batch_name.oneOf.1","typeLabel":"null"}],"uniqueKey":"batch_name","typeLabel":"string | null"},"status":{"type":"string","enum":["PENDING","QUEUED","IN_PROGRESS","PROCESSING","COMPLETED","COMPLETED_WITH_ERRORS","FAILED","CANCELED","INTERRUPTED","UNKNOWN","SKIPPED","DRAFT","ACTIVE","ARCHIVED","SUSPENDED","DEACTIVATED"],"title":"TaskStatusEnum","description":"OPTIONAL (defaults to DRAFT). Current processing status of the batch. Lifecycle: DRAFT → PENDING → IN_PROGRESS → COMPLETED/FAILED. DRAFT: Batch created but not yet submitted. PENDING: Batch submitted and queued for processing. IN_PROGRESS: Batch currently processing (one or more tiers active). COMPLETED: All tiers successfully completed. FAILED: One or more tiers failed. Aggregated from tier_tasks statuses during multi-tier processing.","default":"DRAFT","examples":["DRAFT","PENDING","IN_PROGRESS","COMPLETED","FAILED"],"uniqueKey":"status","typeLabel":"enum\u003cstring>"},"queue_context":{"oneOf":[{"type":"object","title":"Queue Context","description":"The batch's wait for a processing slot. While the batch is PENDING/PROCESSING, a GET adds how many earlier active batches are ahead in this namespace's queue (batches_ahead) and its age (waiting_seconds). Once a batch that queued is admitted, the record keeps the wait for good: waited_seconds (total seconds spent queued, across tiers), reason (why it waited) and admitted_at; queued_at is set while it is waiting.","additionalProperties":true,"uniqueKey":"queue_context.oneOf.0","typeLabel":"Queue Context · object"},{"type":"null","title":"Queue Context","description":"The batch's wait for a processing slot. While the batch is PENDING/PROCESSING, a GET adds how many earlier active batches are ahead in this namespace's queue (batches_ahead) and its age (waiting_seconds). Once a batch that queued is admitted, the record keeps the wait for good: waited_seconds (total seconds spent queued, across tiers), reason (why it waited) and admitted_at; queued_at is set while it is waiting.","uniqueKey":"queue_context.oneOf.1","typeLabel":"null"}],"uniqueKey":"queue_context","typeLabel":"Queue Context · object | null"},"object_ids":{"items":{"type":"string","uniqueKey":"object_ids.items.[INDEX]","typeLabel":"string"},"type":"array","minItems":0,"title":"Object Ids","description":"List of object IDs to include in this batch. All objects must exist in the specified bucket_id. These objects are the source data for tier 0 processing. Collection-sourced batches may have empty object_ids. Objects are processed in parallel within each tier.","examples":[["obj_video_001","obj_video_002"],["obj_doc_123"]],"uniqueKey":"object_ids","typeLabel":"string[]"},"dedup_strategy":{"type":"string","enum":["skip","replace","force"],"title":"DedupStrategy","description":"Controls how objects already processed in prior batches are handled. Scoped to (bucket, collection). 'skip': Don't reprocess objects that already have documents. 'replace': Delete existing documents and reprocess. 'force': Process regardless, allowing duplicates.","default":"skip","uniqueKey":"dedup_strategy","typeLabel":"enum\u003cstring>"},"effective_dedup_strategy":{"oneOf":[{"type":"string","title":"DedupStrategy","description":"The dedup strategy the batch actually ran with, when it differs from dedup_strategy. Set when the organization's spend policy blocks re-extraction, so a requested 'replace' or 'force' runs as 'skip'; progress.status_warnings carries the reason. Read-only, null when the requested strategy ran.","enum":["skip","replace","force"],"uniqueKey":"effective_dedup_strategy.oneOf.0","typeLabel":"enum\u003cstring>"},{"type":"null","description":"The dedup strategy the batch actually ran with, when it differs from dedup_strategy. Set when the organization's spend policy blocks re-extraction, so a requested 'replace' or 'force' runs as 'skip'; progress.status_warnings carries the reason. Read-only, null when the requested strategy ran.","uniqueKey":"effective_dedup_strategy.oneOf.1","typeLabel":"null"}],"uniqueKey":"effective_dedup_strategy","typeLabel":"enum\u003cstring> | null"},"submitted_by_key_id":{"oneOf":[{"type":"string","title":"Submitted By Key Id","description":"key_id of the API key that submitted this batch, resolved server-side at creation. None for async/system-created batches with no request actor context. Audit only.","uniqueKey":"submitted_by_key_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Submitted By Key Id","description":"key_id of the API key that submitted this batch, resolved server-side at creation. None for async/system-created batches with no request actor context. Audit only.","uniqueKey":"submitted_by_key_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"submitted_by_key_id","typeLabel":"string | null"},"submitted_by_key_prefix":{"oneOf":[{"type":"string","title":"Submitted By Key Prefix","description":"Display prefix of the submitting API key. Audit only.","uniqueKey":"submitted_by_key_prefix.oneOf.0","typeLabel":"string"},{"type":"null","title":"Submitted By Key Prefix","description":"Display prefix of the submitting API key. Audit only.","uniqueKey":"submitted_by_key_prefix.oneOf.1","typeLabel":"null"}],"uniqueKey":"submitted_by_key_prefix","typeLabel":"string | null"},"submitted_by_is_internal":{"oneOf":[{"type":"boolean","title":"Submitted By Is Internal","description":"Whether the submitting key was server-verified internal AT SUBMIT TIME. None = no actor context (unattributable); never used for billing decisions (those read the live key marker).","uniqueKey":"submitted_by_is_internal.oneOf.0","typeLabel":"boolean"},{"type":"null","title":"Submitted By Is Internal","description":"Whether the submitting key was server-verified internal AT SUBMIT TIME. None = no actor context (unattributable); never used for billing decisions (those read the live key marker).","uniqueKey":"submitted_by_is_internal.oneOf.1","typeLabel":"null"}],"uniqueKey":"submitted_by_is_internal","typeLabel":"boolean | null"},"dedup_audit":{"oneOf":[{"type":"object","title":"Dedup Audit","description":"Per-collection dedup decisions. Keys are collection_ids; values contain dedup_strategy, total_input, skipped, processed, and skipped_object_ids (up to 1000). Written at TWO stages, deep-merged per collection: (1) the API at manifest build, when smart-skip enforcement excludes already-complete objects before any engine submission (fields prefixed manifest_*), and (2) the Engine after its resume filter runs on whatever residue was submitted.","additionalProperties":true,"uniqueKey":"dedup_audit.oneOf.0","typeLabel":"Dedup Audit · object"},{"type":"null","title":"Dedup Audit","description":"Per-collection dedup decisions. Keys are collection_ids; values contain dedup_strategy, total_input, skipped, processed, and skipped_object_ids (up to 1000). Written at TWO stages, deep-merged per collection: (1) the API at manifest build, when smart-skip enforcement excludes already-complete objects before any engine submission (fields prefixed manifest_*), and (2) the Engine after its resume filter runs on whatever residue was submitted.","uniqueKey":"dedup_audit.oneOf.1","typeLabel":"null"}],"uniqueKey":"dedup_audit","typeLabel":"Dedup Audit · object | null"},"collection_ids":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"collection_ids.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Collection Ids","description":"OPTIONAL. List of all collection IDs involved in this batch's processing. Automatically populated during DAG resolution from dag_tiers. Includes collections from all tiers (flattened view of dag_tiers). Used for quick lookups without traversing tier structure. Format: List of collection IDs across all tiers.","example":["col_chunks"],"uniqueKey":"collection_ids.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Collection Ids","description":"OPTIONAL. List of all collection IDs involved in this batch's processing. Automatically populated during DAG resolution from dag_tiers. Includes collections from all tiers (flattened view of dag_tiers). Used for quick lookups without traversing tier structure. Format: List of collection IDs across all tiers.","example":["col_chunks"],"uniqueKey":"collection_ids.oneOf.1","typeLabel":"null"}],"uniqueKey":"collection_ids","typeLabel":"string[] | null"},"error":{"oneOf":[{"type":"string","title":"Error","description":"OPTIONAL. Legacy error message field for backward compatibility. None if batch succeeded or is still processing. Contains human-readable error description from first failed tier. DEPRECATED: Use tier_tasks[].errors for detailed error information. For multi-tier batches, typically contains the error from the first failed tier. Check tier_tasks array for tier-specific error details and error_summary for aggregation.","x-mixpeek-redacted":"driver_text","example":"Failed to process batch: Object not found","uniqueKey":"error.oneOf.0","typeLabel":"string"},{"type":"null","title":"Error","description":"OPTIONAL. Legacy error message field for backward compatibility. None if batch succeeded or is still processing. Contains human-readable error description from first failed tier. DEPRECATED: Use tier_tasks[].errors for detailed error information. For multi-tier batches, typically contains the error from the first failed tier. Check tier_tasks array for tier-specific error details and error_summary for aggregation.","x-mixpeek-redacted":"driver_text","example":"Failed to process batch: Object not found","uniqueKey":"error.oneOf.1","typeLabel":"null"}],"uniqueKey":"error","typeLabel":"string | null"},"failure_reason":{"oneOf":[{"type":"string","title":"Failure Reason","description":"OPTIONAL. Human-readable explanation of why the batch failed. None if batch succeeded, is still processing, or is in DRAFT/PENDING state. Populated automatically when a batch transitions to FAILED status. Provides a concise, actionable summary of the root cause. Common reasons include: Ray job failure (spot preemption, OOM, code errors), 0 documents written (processing completed but produced no output), processing stall (no activity detected for extended period), or task exception (submission/validation failures). Use this field for user-facing error displays and alerting.","x-mixpeek-redacted":"driver_text","example":"Ray job failed: ImportError: No module named 'google.genai'","uniqueKey":"failure_reason.oneOf.0","typeLabel":"string"},{"type":"null","title":"Failure Reason","description":"OPTIONAL. Human-readable explanation of why the batch failed. None if batch succeeded, is still processing, or is in DRAFT/PENDING state. Populated automatically when a batch transitions to FAILED status. Provides a concise, actionable summary of the root cause. Common reasons include: Ray job failure (spot preemption, OOM, code errors), 0 documents written (processing completed but produced no output), processing stall (no activity detected for extended period), or task exception (submission/validation failures). Use this field for user-facing error displays and alerting.","x-mixpeek-redacted":"driver_text","example":"Ray job failed: ImportError: No module named 'google.genai'","uniqueKey":"failure_reason.oneOf.1","typeLabel":"null"}],"uniqueKey":"failure_reason","typeLabel":"string | null"},"error_summary":{"oneOf":[{"type":"object","title":"Error Summary","description":"OPTIONAL. Aggregated summary of errors across ALL tiers in the batch. Maps error_type (category) to total count of affected DOCUMENTS across all tiers. None has THREE causes, not two: the batch succeeded, it is still processing, OR a tier FAILED AT TIER LEVEL and wrote zero documents, so no individual document carries an error and this histogram is legitimately empty. So None does NOT mean 'no errors', and None alongside COMPLETED_WITH_ERRORS is NOT a contradiction — read `failure_reason` and `tier_tasks[].error` for tier-level diagnosis. Provides quick batch-wide overview of error distribution. Example: {'dependency': 15, 'authentication': 25, 'validation': 5} means across all tiers, 15 documents failed with dependency errors, 25 with auth errors, 5 with validation errors. Automatically aggregated from tier_tasks[].error_summary. Used for batch health dashboard and error trend analysis.","example":null,"additionalProperties":{"type":"integer","uniqueKey":"error_summary.oneOf.0.additionalProperties","typeLabel":"integer"},"uniqueKey":"error_summary.oneOf.0","typeLabel":"Error Summary · object"},{"type":"null","title":"Error Summary","description":"OPTIONAL. Aggregated summary of errors across ALL tiers in the batch. Maps error_type (category) to total count of affected DOCUMENTS across all tiers. None has THREE causes, not two: the batch succeeded, it is still processing, OR a tier FAILED AT TIER LEVEL and wrote zero documents, so no individual document carries an error and this histogram is legitimately empty. So None does NOT mean 'no errors', and None alongside COMPLETED_WITH_ERRORS is NOT a contradiction — read `failure_reason` and `tier_tasks[].error` for tier-level diagnosis. Provides quick batch-wide overview of error distribution. Example: {'dependency': 15, 'authentication': 25, 'validation': 5} means across all tiers, 15 documents failed with dependency errors, 25 with auth errors, 5 with validation errors. Automatically aggregated from tier_tasks[].error_summary. Used for batch health dashboard and error trend analysis.","example":null,"uniqueKey":"error_summary.oneOf.1","typeLabel":"null"}],"uniqueKey":"error_summary","typeLabel":"Error Summary · object | null"},"failure_category":{"oneOf":[{"type":"string","title":"FailureCategory","description":"OPTIONAL. Machine-readable classification of the batch failure. None if batch succeeded or is still processing. Auto-derived from `failure_reason` if not explicitly set, so existing writers that only populate `failure_reason` still get a category on read. Categories: timeout (stall/no-progress), infrastructure (OOM, workers died, spot preemption), orphaned (RayJob CRD gone), pipeline (genuine extractor/pipeline failure), validation (submission/schema errors), unknown (uncategorized). Use this instead of parsing failure_reason.","enum":["timeout","infrastructure","orphaned","pipeline","mvs_durability","validation","unknown"],"example":null,"uniqueKey":"failure_category.oneOf.0","typeLabel":"enum\u003cstring>"},{"type":"null","description":"OPTIONAL. Machine-readable classification of the batch failure. None if batch succeeded or is still processing. Auto-derived from `failure_reason` if not explicitly set, so existing writers that only populate `failure_reason` still get a category on read. Categories: timeout (stall/no-progress), infrastructure (OOM, workers died, spot preemption), orphaned (RayJob CRD gone), pipeline (genuine extractor/pipeline failure), validation (submission/schema errors), unknown (uncategorized). Use this instead of parsing failure_reason.","example":null,"uniqueKey":"failure_category.oneOf.1","typeLabel":"null"}],"uniqueKey":"failure_category","typeLabel":"enum\u003cstring> | null"},"failed_objects":{"items":{"properties":{"object_id":{"type":"string","title":"Object Id","description":"ID of the object that failed processing.","examples":["obj_video_001","obj_doc_123"],"isRequired":true,"uniqueKey":"failed_objects.items.[INDEX].object_id","typeLabel":"string"},"error":{"type":"string","title":"Error","description":"Human-readable error message describing what went wrong.","examples":["Invalid video format: unsupported codec","Connection timeout to embedding service","CUDA out of memory"],"x-mixpeek-redacted":"driver_text","isRequired":true,"uniqueKey":"failed_objects.items.[INDEX].error","typeLabel":"string"},"error_type":{"type":"string","enum":["transient","permanent","resource"],"title":"Error Type","description":"Classification of the error for retry decisions. transient: network, timeout, temporary service issues (worth retrying). permanent: bad data, unsupported format (will never succeed). resource: GPU OOM, quota exceeded (may succeed with different resources).","examples":["transient","permanent","resource"],"isRequired":true,"uniqueKey":"failed_objects.items.[INDEX].error_type","typeLabel":"enum\u003cstring>"},"timestamp":{"type":"string","title":"Timestamp","description":"ISO 8601 timestamp when the error occurred.","examples":["2025-11-29T10:15:30Z"],"isRequired":true,"uniqueKey":"failed_objects.items.[INDEX].timestamp","typeLabel":"string"}},"type":"object","required":["object_id","error","error_type","timestamp"],"title":"FailedObjectRecord","description":"Record of a single object that failed during batch processing.\n\nStored on the batch document in MongoDB to provide per-object error\nvisibility without requiring a separate query to the failed_documents\ncollection.\n\nUsed to:\n- Show users exactly which objects failed and why\n- Classify errors for retry decisions (transient vs permanent vs resource)\n- Enable selective resubmission of failed objects","uniqueKey":"failed_objects.items.[INDEX]","typeLabel":"FailedObjectRecord · object"},"type":"array","title":"Failed Objects","description":"OPTIONAL. List of per-object failure records from batch processing. Populated when individual objects fail while others succeed. Each record includes the object_id, error message, error classification (transient/permanent/resource), and timestamp. When this list is non-empty and some objects succeeded, batch status is COMPLETED_WITH_ERRORS. Enables targeted resubmission of only failed objects.","examples":[[],[{"error":"Unsupported codec: VP9","error_type":"permanent","object_id":"obj_video_001","timestamp":"2025-11-29T10:15:30Z"},{"error":"Connection timeout to embedding service","error_type":"transient","object_id":"obj_doc_123","timestamp":"2025-11-29T10:15:35Z"}]],"uniqueKey":"failed_objects","typeLabel":"FailedObjectRecord · object[]"},"failed_object_count":{"type":"integer","minimum":0,"title":"Failed Object Count","description":"OPTIONAL. Count of objects that failed during batch processing and wrote no documents (an object whose documents were written but one feature failed is in partial_objects instead). Shorthand for len(failed_objects). Stored separately for efficient queries and sorting without loading full failed_objects array.","default":0,"examples":[0,3,15],"uniqueKey":"failed_object_count","typeLabel":"integer"},"partial_objects":{"oneOf":[{"type":"array","items":{"additionalProperties":true,"type":"object","uniqueKey":"partial_objects.oneOf.0.items.[INDEX]","typeLabel":"object"},"title":"Partial Objects","description":"Objects that wrote documents while one of their features failed, for example a transcription error on a video whose scenes, descriptions and embeddings were written. Each entry carries the object_id, the error and missing_feature, the feature the object has no documents for, or null when the error names none. An object with no document in one of the batch's collections also carries missing_collection_ids. These objects are searchable through the documents they wrote and are not counted in failed_object_count; they count in progress.errors.","uniqueKey":"partial_objects.oneOf.0","typeLabel":"Partial Objects · object[]"},{"type":"null","title":"Partial Objects","description":"Objects that wrote documents while one of their features failed, for example a transcription error on a video whose scenes, descriptions and embeddings were written. Each entry carries the object_id, the error and missing_feature, the feature the object has no documents for, or null when the error names none. An object with no document in one of the batch's collections also carries missing_collection_ids. These objects are searchable through the documents they wrote and are not counted in failed_object_count; they count in progress.errors.","uniqueKey":"partial_objects.oneOf.1","typeLabel":"null"}],"uniqueKey":"partial_objects","typeLabel":"Partial Objects · object[] | null"},"partial_object_count":{"oneOf":[{"type":"integer","title":"Partial Object Count","description":"Number of objects in partial_objects.","minimum":0,"uniqueKey":"partial_object_count.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Partial Object Count","description":"Number of objects in partial_objects.","uniqueKey":"partial_object_count.oneOf.1","typeLabel":"null"}],"uniqueKey":"partial_object_count","typeLabel":"integer | null"},"unaccounted_object_count":{"oneOf":[{"type":"integer","title":"Unaccounted Object Count","description":"Objects that produced NEITHER a document NOR a failure entry — the per-tier balance audits' summed `lost` (submitted - processed - failed - skipped), rolled up at completion so the gap is STATED on the record instead of inferred by subtraction. None when every submitted object is accounted for; per-tier detail lives at tier_tasks[].audit.","minimum":0,"example":38,"uniqueKey":"unaccounted_object_count.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Unaccounted Object Count","description":"Objects that produced NEITHER a document NOR a failure entry — the per-tier balance audits' summed `lost` (submitted - processed - failed - skipped), rolled up at completion so the gap is STATED on the record instead of inferred by subtraction. None when every submitted object is accounted for; per-tier detail lives at tier_tasks[].audit.","example":38,"uniqueKey":"unaccounted_object_count.oneOf.1","typeLabel":"null"}],"uniqueKey":"unaccounted_object_count","typeLabel":"integer | null"},"type":{"type":"string","enum":["BUCKET","COLLECTION"],"title":"BatchType","description":"OPTIONAL (defaults to BUCKET). Type of batch. BUCKET: Standard batch processing bucket objects through collections. COLLECTION: Reserved for future collection-only batch processing. Currently only BUCKET type is supported.","default":"BUCKET","examples":["BUCKET"],"uniqueKey":"type","typeLabel":"enum\u003cstring>"},"manifest_key":{"oneOf":[{"type":"string","title":"Manifest Key","description":"OPTIONAL. S3 key where the batch manifest is stored. Contains metadata and row data (Parquet) for Engine processing. For tier 0, points to bucket object manifest. For tier N+, points to collection document manifest. Format: S3 path (e.g., 'namespace_id/internal_id/manifests/tier_0.parquet'). Generated during batch submission.","example":"ns_abc/org_123/manifests/tier_0.parquet","uniqueKey":"manifest_key.oneOf.0","typeLabel":"string"},{"type":"null","title":"Manifest Key","description":"OPTIONAL. S3 key where the batch manifest is stored. Contains metadata and row data (Parquet) for Engine processing. For tier 0, points to bucket object manifest. For tier N+, points to collection document manifest. Format: S3 path (e.g., 'namespace_id/internal_id/manifests/tier_0.parquet'). Generated during batch submission.","example":"ns_abc/org_123/manifests/tier_0.parquet","uniqueKey":"manifest_key.oneOf.1","typeLabel":"null"}],"uniqueKey":"manifest_key","typeLabel":"string | null"},"task_id":{"oneOf":[{"type":"string","title":"Task Id","description":"OPTIONAL. Primary task ID for the batch (typically tier 0 task). Used for backward compatibility with single-tier batch tracking. For multi-tier batches, prefer querying tier_tasks array for granular tracking. Format: Task ID as generated for tier 0.","example":"task_tier0_abc123","uniqueKey":"task_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Task Id","description":"OPTIONAL. Primary task ID for the batch (typically tier 0 task). Used for backward compatibility with single-tier batch tracking. For multi-tier batches, prefer querying tier_tasks array for granular tracking. Format: Task ID as generated for tier 0.","example":"task_tier0_abc123","uniqueKey":"task_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"task_id","typeLabel":"string | null"},"loaded_object_ids":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"loaded_object_ids.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Loaded Object Ids","description":"OPTIONAL. List of object IDs that were successfully validated and loaded into the batch. Subset of object_ids that passed validation. Used to track which objects are ready for processing. None if batch hasn't been validated yet.","example":["obj_video_001","obj_video_002"],"uniqueKey":"loaded_object_ids.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Loaded Object Ids","description":"OPTIONAL. List of object IDs that were successfully validated and loaded into the batch. Subset of object_ids that passed validation. Used to track which objects are ready for processing. None if batch hasn't been validated yet.","example":["obj_video_001","obj_video_002"],"uniqueKey":"loaded_object_ids.oneOf.1","typeLabel":"null"}],"uniqueKey":"loaded_object_ids","typeLabel":"string[] | null"},"internal_metadata":{"oneOf":[{"type":"object","title":"Internal Metadata","description":"OPTIONAL. Internal engine/job metadata for system use. May contain: job_id (provider-specific), engine_version, processing hints, last_health_check. last_health_check: Most recent health check results with health_status, enriched_documents, vector_populated_count, stall_duration_seconds, recommendations, missing_features. Populated asynchronously (non-blocking, best-effort). Used for troubleshooting batch processing issues via API. NOTE: In MongoDB, this is stored under '_internal.processing' path.","example":{"include_history":true,"last_health_check":{"enriched_documents":98,"health_status":"HEALTHY","missing_features":["text_embedding"],"processed_documents":100,"recommendations":[],"stall_duration_seconds":0,"timestamp":"2025-11-06T10:05:00Z","total_documents":100,"vector_populated_count":98}},"additionalProperties":true,"uniqueKey":"internal_metadata.oneOf.0","typeLabel":"Internal Metadata · object"},{"type":"null","title":"Internal Metadata","description":"OPTIONAL. Internal engine/job metadata for system use. May contain: job_id (provider-specific), engine_version, processing hints, last_health_check. last_health_check: Most recent health check results with health_status, enriched_documents, vector_populated_count, stall_duration_seconds, recommendations, missing_features. Populated asynchronously (non-blocking, best-effort). Used for troubleshooting batch processing issues via API. NOTE: In MongoDB, this is stored under '_internal.processing' path.","example":{"include_history":true,"last_health_check":{"enriched_documents":98,"health_status":"HEALTHY","missing_features":["text_embedding"],"processed_documents":100,"recommendations":[],"stall_duration_seconds":0,"timestamp":"2025-11-06T10:05:00Z","total_documents":100,"vector_populated_count":98}},"uniqueKey":"internal_metadata.oneOf.1","typeLabel":"null"}],"uniqueKey":"internal_metadata","typeLabel":"Internal Metadata · object | null"},"metadata":{"properties":{"campaign_id":{"oneOf":[{"type":"string","title":"Campaign Id","description":"Identifier linking this batch to a marketing or processing campaign.","example":"Q4_2025","uniqueKey":"metadata.campaign_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Campaign Id","description":"Identifier linking this batch to a marketing or processing campaign.","example":"Q4_2025","uniqueKey":"metadata.campaign_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.campaign_id","typeLabel":"string | null"},"source":{"oneOf":[{"type":"string","title":"Source","description":"Origin of the batch data (e.g. an S3 prefix, partner name, or pipeline stage).","example":"s3://raw-uploads/2026-05/","uniqueKey":"metadata.source.oneOf.0","typeLabel":"string"},{"type":"null","title":"Source","description":"Origin of the batch data (e.g. an S3 prefix, partner name, or pipeline stage).","example":"s3://raw-uploads/2026-05/","uniqueKey":"metadata.source.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.source","typeLabel":"string | null"},"tags":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"metadata.tags.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Tags","description":"Free-form tags for filtering and grouping batches.","example":["video","high-priority"],"uniqueKey":"metadata.tags.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Tags","description":"Free-form tags for filtering and grouping batches.","example":["video","high-priority"],"uniqueKey":"metadata.tags.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.tags","typeLabel":"string[] | null"},"notes":{"oneOf":[{"type":"string","title":"Notes","description":"Free-form notes about this batch (intent, context, special handling).","example":"Re-run after Whisper quota fix","uniqueKey":"metadata.notes.oneOf.0","typeLabel":"string"},{"type":"null","title":"Notes","description":"Free-form notes about this batch (intent, context, special handling).","example":"Re-run after Whisper quota fix","uniqueKey":"metadata.notes.oneOf.1","typeLabel":"null"}],"uniqueKey":"metadata.notes","typeLabel":"string | null"}},"additionalProperties":true,"type":"object","title":"BatchMetadata","description":"OPTIONAL. User-defined metadata for the batch. Has typed fields (campaign_id, source, tags, notes) and also accepts arbitrary extra keys. Persisted with the batch and returned in API responses. Not used by the system for processing logic.","examples":[{"campaign_id":"Q4_2025","tags":["video","high-priority"]},{"notes":"Nightly uploads ingest","source":"s3://raw-uploads/2026-05/"}],"uniqueKey":"metadata","typeLabel":"BatchMetadata · object"},"tier_tasks":{"items":{"properties":{"tier_num":{"type":"integer","minimum":0,"title":"Tier Num","description":"REQUIRED. Zero-based tier number indicating the processing stage. Tier 0 = initial bucket-to-collection processing (bucket objects as source). Tier N (N > 0) = collection-to-collection processing (upstream documents as source). Used to determine processing order and identify which stage a task represents. Example: In a 5-tier pipeline (bucket → chunks → frames → scenes → summaries), chunks=tier 0, frames=tier 1, scenes=tier 2, summaries=tier 3.","examples":[0,1,2],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].tier_num","typeLabel":"integer"},"source_type":{"type":"string","title":"Source Type","description":"REQUIRED. Type of data source for this tier's processing. 'bucket': Tier 0 processing where source is bucket objects from the objects table. 'collection': Tier N+ processing where source is documents from upstream collection(s). Determines how the API prepares the input dataset manifest for the Engine. Bucket sources query the objects table and include file blobs. Collection sources query the documents table and include processed features.","examples":["bucket","collection"],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].source_type","typeLabel":"string"},"scoped_sources":{"oneOf":[{"type":"integer","title":"Scoped Sources","description":"OPTIONAL. How many source collections this tier read for the batch's own objects only, summed over its extractor groups. 0 means every source was read in full. Set on collection-sourced tiers of a bucket batch.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].scoped_sources.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Scoped Sources","description":"OPTIONAL. How many source collections this tier read for the batch's own objects only, summed over its extractor groups. 0 means every source was read in full. Set on collection-sourced tiers of a bucket batch.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].scoped_sources.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].scoped_sources","typeLabel":"integer | null"},"skipped_collection_reasons":{"oneOf":[{"type":"object","title":"Skipped Collection Reasons","description":"OPTIONAL. Why a collection of this tier submitted no work, keyed by collection id. 'scoped_read_empty' means none of the batch's objects has documents in that collection's sources. Empty when no collection was skipped for a recorded reason.","example":{},"additionalProperties":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].skipped_collection_reasons.oneOf.0.additionalProperties","typeLabel":"string"},"uniqueKey":"tier_tasks.items.[INDEX].skipped_collection_reasons.oneOf.0","typeLabel":"Skipped Collection Reasons · object"},{"type":"null","title":"Skipped Collection Reasons","description":"OPTIONAL. Why a collection of this tier submitted no work, keyed by collection id. 'scoped_read_empty' means none of the batch's objects has documents in that collection's sources. Empty when no collection was skipped for a recorded reason.","example":{},"uniqueKey":"tier_tasks.items.[INDEX].skipped_collection_reasons.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].skipped_collection_reasons","typeLabel":"Skipped Collection Reasons · object | null"},"replace_cleanup_skipped_collections":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].replace_cleanup_skipped_collections.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Replace Cleanup Skipped Collections","description":"OPTIONAL. For dedup_strategy replace: collections of this tier whose old documents were left in place because their extractor job did not end COMPLETED or COMPLETED_WITH_ERRORS (for example it failed or was still pending).","example":[],"uniqueKey":"tier_tasks.items.[INDEX].replace_cleanup_skipped_collections.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Replace Cleanup Skipped Collections","description":"OPTIONAL. For dedup_strategy replace: collections of this tier whose old documents were left in place because their extractor job did not end COMPLETED or COMPLETED_WITH_ERRORS (for example it failed or was still pending).","example":[],"uniqueKey":"tier_tasks.items.[INDEX].replace_cleanup_skipped_collections.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].replace_cleanup_skipped_collections","typeLabel":"string[] | null"},"task_id":{"oneOf":[{"type":"string","title":"Task Id","description":"OPTIONAL. Unique task identifier for this tier's processing task. None if tier has not yet started (status=PENDING). Assigned when tier processing begins (status=IN_PROGRESS). Used to query task status via GET /v1/tasks/{task_id}. Format: 'task_' prefix followed by secure token. Generated using generate_secure_token() from shared.utilities.helpers.","example":"task_tier0_abc123","uniqueKey":"tier_tasks.items.[INDEX].task_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Task Id","description":"OPTIONAL. Unique task identifier for this tier's processing task. None if tier has not yet started (status=PENDING). Assigned when tier processing begins (status=IN_PROGRESS). Used to query task status via GET /v1/tasks/{task_id}. Format: 'task_' prefix followed by secure token. Generated using generate_secure_token() from shared.utilities.helpers.","example":"task_tier0_abc123","uniqueKey":"tier_tasks.items.[INDEX].task_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].task_id","typeLabel":"string | null"},"status":{"type":"string","enum":["PENDING","QUEUED","IN_PROGRESS","PROCESSING","COMPLETED","COMPLETED_WITH_ERRORS","FAILED","CANCELED","INTERRUPTED","UNKNOWN","SKIPPED","DRAFT","ACTIVE","ARCHIVED","SUSPENDED","DEACTIVATED"],"title":"TaskStatusEnum","description":"REQUIRED. Current processing status of this tier's task. Lifecycle: PENDING → IN_PROGRESS → COMPLETED/FAILED. PENDING: Tier scheduled but not yet started. IN_PROGRESS: Tier currently processing documents. COMPLETED: Tier successfully processed all documents. FAILED: Tier encountered an error and stopped processing. Used to determine overall batch status and whether to proceed to next tier.","default":"PENDING","examples":["PENDING","IN_PROGRESS","COMPLETED","FAILED"],"uniqueKey":"tier_tasks.items.[INDEX].status","typeLabel":"enum\u003cstring>"},"collection_ids":{"items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].collection_ids.items.[INDEX]","typeLabel":"string"},"type":"array","minItems":1,"title":"Collection Ids","description":"REQUIRED. List of collection IDs being processed in this tier. Flattened from extractor_jobs for convenience. Each tier can process one or more collections in parallel. Collections in the same tier have no dependencies on each other. Format: Collection IDs as defined when collections were created. Minimum 1 collection per tier. Example: Tier 1 might process ['col_frames_30fps', 'col_frames_60fps'] in parallel.","examples":[["col_chunks"],["col_frames","col_scenes"],["col_summaries"]],"uniqueKey":"tier_tasks.items.[INDEX].collection_ids","typeLabel":"string[]"},"extractor_jobs":{"items":{"properties":{"extractor_type":{"type":"string","title":"Extractor Type","description":"Feature extractor type (e.g., 'image_extractor', 'face_identity_extractor')","examples":["image_extractor","face_identity_extractor","video_extractor"],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].extractor_type","typeLabel":"string"},"rows_duplicate_id":{"oneOf":[{"type":"integer","title":"Rows Duplicate Id","description":"OPTIONAL. Set on a retriever_transform job. Rows not written because their document id repeated an earlier row from the same source document, in a pipeline with no group_by stage. This usually means the id key is missing the field that tells those rows apart, such as the position an unwind stage writes; add it to the transform's unique_id_field. The job's warnings name the fix when this is above 0.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_duplicate_id.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Rows Duplicate Id","description":"OPTIONAL. Set on a retriever_transform job. Rows not written because their document id repeated an earlier row from the same source document, in a pipeline with no group_by stage. This usually means the id key is missing the field that tells those rows apart, such as the position an unwind stage writes; add it to the transform's unique_id_field. The job's warnings name the fix when this is above 0.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_duplicate_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_duplicate_id","typeLabel":"integer | null"},"rows_collapsed":{"oneOf":[{"type":"integer","title":"Rows Collapsed","description":"OPTIONAL. Set on a retriever_transform job. Rows not written because their document id repeated an earlier row from a different source document under the transform's unique_id_field, or any repeated id after a group_by stage. The first row with an id is the one written.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_collapsed.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Rows Collapsed","description":"OPTIONAL. Set on a retriever_transform job. Rows not written because their document id repeated an earlier row from a different source document under the transform's unique_id_field, or any repeated id after a group_by stage. The first row with an id is the one written.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_collapsed.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_collapsed","typeLabel":"integer | null"},"first_duplicate_key":{"oneOf":[{"type":"string","title":"First Duplicate Key","description":"OPTIONAL. The identity key of the first row skipped for a repeated id, counted in rows_duplicate_id or rows_collapsed.","example":"document_id=doc_abc123","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].first_duplicate_key.oneOf.0","typeLabel":"string"},{"type":"null","title":"First Duplicate Key","description":"OPTIONAL. The identity key of the first row skipped for a repeated id, counted in rows_duplicate_id or rows_collapsed.","example":"document_id=doc_abc123","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].first_duplicate_key.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].first_duplicate_key","typeLabel":"string | null"},"rows_skipped_by_stages":{"oneOf":[{"type":"integer","title":"Rows Skipped By Stages","description":"OPTIONAL. Set on a retriever_transform job. Rows the pipeline's json_transform stages dropped because the template failed on them and the stage has fail_on_error set to false. first_stage_error says why.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_skipped_by_stages.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Rows Skipped By Stages","description":"OPTIONAL. Set on a retriever_transform job. Rows the pipeline's json_transform stages dropped because the template failed on them and the stage has fail_on_error set to false. first_stage_error says why.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_skipped_by_stages.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].rows_skipped_by_stages","typeLabel":"integer | null"},"first_stage_error":{"oneOf":[{"type":"string","title":"First Stage Error","description":"OPTIONAL. The first error recorded by the earliest pipeline stage, in pipeline order, that recorded one during a retriever_transform build.","x-mixpeek-redacted":"driver_text","example":"Document doc_abc123 transformation failed: unsupported operand type(s)","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].first_stage_error.oneOf.0","typeLabel":"string"},{"type":"null","title":"First Stage Error","description":"OPTIONAL. The first error recorded by the earliest pipeline stage, in pipeline order, that recorded one during a retriever_transform build.","x-mixpeek-redacted":"driver_text","example":"Document doc_abc123 transformation failed: unsupported operand type(s)","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].first_stage_error.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].first_stage_error","typeLabel":"string | null"},"collection_ids":{"items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].collection_ids.items.[INDEX]","typeLabel":"string"},"type":"array","minItems":1,"title":"Collection Ids","description":"Collections processed by this extractor job","examples":[["col_abc123"],["col_def456","col_ghi789"]],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].collection_ids","typeLabel":"string[]"},"extractor_id":{"oneOf":[{"type":"string","title":"Extractor Id","description":"Concrete extractor identifier used for this job, e.g. 'universal_extractor_v1'.","example":"universal_extractor_v1","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].extractor_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Extractor Id","description":"Concrete extractor identifier used for this job, e.g. 'universal_extractor_v1'.","example":"universal_extractor_v1","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].extractor_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].extractor_id","typeLabel":"string | null"},"ray_job_id":{"oneOf":[{"type":"string","title":"Ray Job Id","description":"Ray job ID for this extractor job","example":"raysubmit_abc123","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].ray_job_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Ray Job Id","description":"Ray job ID for this extractor job","example":"raysubmit_abc123","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].ray_job_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].ray_job_id","typeLabel":"string | null"},"celery_task_id":{"oneOf":[{"type":"string","title":"Task Id","description":"Background task ID that submitted this processing job","example":"celery_task_abc123","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].celery_task_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Task Id","description":"Background task ID that submitted this processing job","example":"celery_task_abc123","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].celery_task_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].celery_task_id","typeLabel":"string | null"},"callback_job_id":{"oneOf":[{"type":"string","title":"Callback Job Id","description":"Job ID expected in the tier completion callback for this extractor job.","example":"celery_fast_path_btch_abc123_0_universal_extractor_v1","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].callback_job_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Callback Job Id","description":"Job ID expected in the tier completion callback for this extractor job.","example":"celery_fast_path_btch_abc123_0_universal_extractor_v1","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].callback_job_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].callback_job_id","typeLabel":"string | null"},"execution_mode":{"oneOf":[{"type":"string","title":"Execution Mode","description":"Execution backend used for this extractor job.","example":"celery_fast_path_universal","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].execution_mode.oneOf.0","typeLabel":"string"},{"type":"null","title":"Execution Mode","description":"Execution backend used for this extractor job.","example":"celery_fast_path_universal","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].execution_mode.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].execution_mode","typeLabel":"string | null"},"status":{"type":"string","enum":["PENDING","QUEUED","IN_PROGRESS","PROCESSING","COMPLETED","COMPLETED_WITH_ERRORS","FAILED","CANCELED","INTERRUPTED","UNKNOWN","SKIPPED","DRAFT","ACTIVE","ARCHIVED","SUSPENDED","DEACTIVATED"],"title":"TaskStatusEnum","description":"Current status of this extractor job","default":"PENDING","examples":["PENDING","IN_PROGRESS","COMPLETED","FAILED"],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].status","typeLabel":"enum\u003cstring>"},"started_at":{"oneOf":[{"type":"string","title":"Started At","description":"When this extractor job started processing","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].started_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Started At","description":"When this extractor job started processing","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].started_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].started_at","typeLabel":"string\u003cdate-time> | null"},"completed_at":{"oneOf":[{"type":"string","title":"Completed At","description":"When this extractor job finished processing","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].completed_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Completed At","description":"When this extractor job finished processing","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].completed_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].completed_at","typeLabel":"string\u003cdate-time> | null"},"duration_ms":{"oneOf":[{"type":"number","title":"Duration Ms","description":"Processing duration in milliseconds","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].duration_ms.oneOf.0","typeLabel":"number"},{"type":"null","title":"Duration Ms","description":"Processing duration in milliseconds","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].duration_ms.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].duration_ms","typeLabel":"number | null"},"documents_written":{"oneOf":[{"type":"integer","title":"Documents Written","description":"Number of documents written by this extractor job","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].documents_written.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Written","description":"Number of documents written by this extractor job","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].documents_written.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].documents_written","typeLabel":"integer | null"},"warnings":{"items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].warnings.items.[INDEX]","typeLabel":"string"},"type":"array","title":"Warnings","description":"Caller-visible warnings this job produced (e.g. a retriever_transform stage's degradation warnings like document_enrich's 'direct join matched 0 of N', or feature_search fusion drops). Persisted here by the inline transform path so BatchModel.compute_observability can lift them to progress.status_warnings; without this declared field the persisted warnings are dropped on read (extra='ignore') and a COMPLETED[_WITH_ERRORS] batch surfaces an empty warnings list, the exact silent-forgiveness class this closes.","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].warnings","typeLabel":"string[]"},"pages_dropped":{"oneOf":[{"type":"integer","title":"Pages Dropped","description":"OPTIONAL. Pages of multi-page inputs (PDFs) that were NOT indexed by this job — capped by max_document_pages or dropped by per-page failures. Only present when > 0. A COMPLETED job with pages_dropped > 0 indexed the input PARTIALLY; see pages_dropped_reasons and raise max_document_pages to index more.","example":10,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Pages Dropped","description":"OPTIONAL. Pages of multi-page inputs (PDFs) that were NOT indexed by this job — capped by max_document_pages or dropped by per-page failures. Only present when > 0. A COMPLETED job with pages_dropped > 0 indexed the input PARTIALLY; see pages_dropped_reasons and raise max_document_pages to index more.","example":10,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped","typeLabel":"integer | null"},"pages_dropped_reasons":{"oneOf":[{"type":"object","title":"Pages Dropped Reasons","description":"OPTIONAL. Dropped-page counts by reason: max_document_pages_cap (input exceeded the collection's max_document_pages) or page_processing_failure (per-page extract/embed errors).","example":{"max_document_pages_cap":10},"additionalProperties":{"type":"integer","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped_reasons.oneOf.0.additionalProperties","typeLabel":"integer"},"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped_reasons.oneOf.0","typeLabel":"Pages Dropped Reasons · object"},{"type":"null","title":"Pages Dropped Reasons","description":"OPTIONAL. Dropped-page counts by reason: max_document_pages_cap (input exceeded the collection's max_document_pages) or page_processing_failure (per-page extract/embed errors).","example":{"max_document_pages_cap":10},"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped_reasons.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].pages_dropped_reasons","typeLabel":"Pages Dropped Reasons · object | null"},"segments_dropped":{"oneOf":[{"type":"integer","title":"Segments Dropped","description":"OPTIONAL. Video segments that were NOT indexed by this job. Only present when > 0. A COMPLETED job with segments_dropped > 0 indexed the video PARTIALLY. segments_dropped_reasons says why: config_cap_max_video_segments means the video had more segments than the collection's max_video_segments, so raise that setting to cover more of the video; segment_embedding_failure means a segment's embedding failed and it was not written, which raising max_video_segments does not recover.","example":5,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Segments Dropped","description":"OPTIONAL. Video segments that were NOT indexed by this job. Only present when > 0. A COMPLETED job with segments_dropped > 0 indexed the video PARTIALLY. segments_dropped_reasons says why: config_cap_max_video_segments means the video had more segments than the collection's max_video_segments, so raise that setting to cover more of the video; segment_embedding_failure means a segment's embedding failed and it was not written, which raising max_video_segments does not recover.","example":5,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped","typeLabel":"integer | null"},"segments_dropped_reasons":{"oneOf":[{"type":"object","title":"Segments Dropped Reasons","description":"OPTIONAL. Dropped-segment counts by reason: config_cap_max_video_segments (video exceeded the collection's max_video_segments) or segment_embedding_failure (a segment's embedding failed, so the segment was not written).","example":{"config_cap_max_video_segments":5},"additionalProperties":{"type":"integer","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped_reasons.oneOf.0.additionalProperties","typeLabel":"integer"},"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped_reasons.oneOf.0","typeLabel":"Segments Dropped Reasons · object"},{"type":"null","title":"Segments Dropped Reasons","description":"OPTIONAL. Dropped-segment counts by reason: config_cap_max_video_segments (video exceeded the collection's max_video_segments) or segment_embedding_failure (a segment's embedding failed, so the segment was not written).","example":{"config_cap_max_video_segments":5},"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped_reasons.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].segments_dropped_reasons","typeLabel":"Segments Dropped Reasons · object | null"},"documents_skipped":{"oneOf":[{"type":"integer","title":"Documents Skipped","description":"OPTIONAL. Number of input rows skipped (not failed) — typically metadata-only objects with no embeddable content, content-flag filtered rows, or null-text chunks. Skipped rows count toward the tier-completion invariant (processed + failed + skipped == submitted) so a tier with 100 inputs and 100 skips still ends in a valid terminal state.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].documents_skipped.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Skipped","description":"OPTIONAL. Number of input rows skipped (not failed) — typically metadata-only objects with no embeddable content, content-flag filtered rows, or null-text chunks. Skipped rows count toward the tier-completion invariant (processed + failed + skipped == submitted) so a tier with 100 inputs and 100 skips still ends in a valid terminal state.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].documents_skipped.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].documents_skipped","typeLabel":"integer | null"},"errors":{"items":{"properties":{"error_type":{"type":"string","enum":["dependency","authentication","validation","runtime","network","resource"],"title":"ErrorCategory","description":"REQUIRED. Category of error that occurred. Used for filtering, retry logic, and error analytics. DEPENDENCY: Missing packages/modules (e.g., google-genai not installed). AUTHENTICATION: Invalid credentials or expired tokens. VALIDATION: Schema mismatches or invalid input data. RUNTIME: Code exceptions or processing failures. NETWORK: Connectivity issues or timeouts. RESOURCE: Out of memory or disk space.","examples":["dependency","authentication","validation"],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].error_type","typeLabel":"enum\u003cstring>"},"message":{"type":"string","title":"Message","description":"REQUIRED. Human-readable error message. Concise description of what went wrong. Should be actionable and help users understand the issue.","examples":["Missing required package: google-genai","Invalid API key for Vertex AI","Schema validation failed: missing required field 'products'"],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].message","typeLabel":"string"},"component":{"oneOf":[{"type":"string","title":"Component","description":"OPTIONAL. Component or service where the error occurred. Helps identify which part of the system failed. Examples: service class names, module names, or feature names.","example":"VertexMultimodalService","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].component.oneOf.0","typeLabel":"string"},{"type":"null","title":"Component","description":"OPTIONAL. Component or service where the error occurred. Helps identify which part of the system failed. Examples: service class names, module names, or feature names.","example":"VertexMultimodalService","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].component.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].component","typeLabel":"string | null"},"stage":{"oneOf":[{"type":"string","title":"Stage","description":"OPTIONAL. Processing stage where the error occurred. Identifies which pipeline stage failed. Examples: pipeline stage names from collection configuration.","example":"gemini_extraction","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].stage.oneOf.0","typeLabel":"string"},{"type":"null","title":"Stage","description":"OPTIONAL. Processing stage where the error occurred. Identifies which pipeline stage failed. Examples: pipeline stage names from collection configuration.","example":"gemini_extraction","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].stage.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].stage","typeLabel":"string | null"},"traceback":{"oneOf":[{"type":"string","title":"Traceback","description":"OPTIONAL. Full Python traceback for debugging. Includes stack trace for code-level troubleshooting. Should be truncated if too long (e.g., max 2000 chars).","x-mixpeek-redacted":"driver_text","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].traceback.oneOf.0","typeLabel":"string"},{"type":"null","title":"Traceback","description":"OPTIONAL. Full Python traceback for debugging. Includes stack trace for code-level troubleshooting. Should be truncated if too long (e.g., max 2000 chars).","x-mixpeek-redacted":"driver_text","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].traceback.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].traceback","typeLabel":"string | null"},"timestamp":{"type":"string","format":"date-time","title":"Timestamp","description":"REQUIRED. ISO 8601 timestamp when the error occurred. Used for chronological error tracking and debugging.","examples":["2025-11-29T10:15:30Z"],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].timestamp","typeLabel":"string\u003cdate-time>"},"affected_document_ids":{"items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].affected_document_ids.items.[INDEX]","typeLabel":"string"},"type":"array","title":"Affected Document Ids","description":"OPTIONAL. List of document IDs affected by this error. For object-level errors: contains single document ID. For batch-level aggregation: contains all affected document IDs. Used to identify scope of impact.","examples":[["doc_123"],["doc_123","doc_456","doc_789"]],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].affected_document_ids","typeLabel":"string[]"},"affected_count":{"type":"integer","minimum":1,"title":"Affected Count","description":"REQUIRED. Number of documents affected by this error. For object-level: typically 1. For batch-level aggregation: total count of affected documents. Used for error impact analysis.","default":1,"examples":[1,10,50],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].affected_count","typeLabel":"integer"},"recovery_suggestion":{"oneOf":[{"type":"string","title":"Recovery Suggestion","description":"OPTIONAL. Actionable suggestion for resolving the error. Helps users quickly fix common issues. Examples: install missing package, check credentials, update schema.","example":"Install google-genai package: pip install google-genai","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].recovery_suggestion.oneOf.0","typeLabel":"string"},{"type":"null","title":"Recovery Suggestion","description":"OPTIONAL. Actionable suggestion for resolving the error. Helps users quickly fix common issues. Examples: install missing package, check credentials, update schema.","example":"Install google-genai package: pip install google-genai","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].recovery_suggestion.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].recovery_suggestion","typeLabel":"string | null"},"metadata":{"additionalProperties":true,"type":"object","title":"Metadata","description":"OPTIONAL. Additional error context and metadata. Free-form dictionary for error-specific details. Examples: retry_count, last_retry_at, error_code, http_status.","examples":[{"last_retry_at":"2025-11-29T10:10:00Z","retry_count":2},{"api_endpoint":"https://vertex.googleapis.com","http_status":401}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX].metadata","typeLabel":"Metadata · object"}},"type":"object","required":["error_type","message"],"title":"BatchErrorDetail","description":"Detailed error information for batch processing failures.\n\nProvides structured error tracking at both object and batch levels.\nEnables better debugging, retry logic, and error analytics.\n\nUse Cases:\n - Track specific errors that occurred during processing\n - Identify error patterns across multiple documents\n - Provide actionable recovery suggestions\n - Enable intelligent retry logic based on error type\n\nObject-level tracking: Attached to individual document processing failures\nBatch-level tracking: Aggregated summaries in batch metadata","examples":[{"affected_count":1,"affected_document_ids":["doc_123"],"component":"VertexMultimodalService","description":"Dependency error: Missing package","error_type":"dependency","message":"Cannot import 'genai' from 'google' package","metadata":{"import_path":"google.genai","package_name":"google-genai"},"recovery_suggestion":"Install google-genai package: pip install google-genai","stage":"gemini_extraction","timestamp":"2025-11-29T10:15:30Z","traceback":"ImportError: cannot import name 'genai' from 'google'..."},{"affected_count":2,"affected_document_ids":["doc_456","doc_789"],"component":"VertexMultimodalService","description":"Authentication error: Invalid credentials","error_type":"authentication","message":"Invalid grant: Bad Request","metadata":{"http_status":401,"retry_count":3},"recovery_suggestion":"Verify your Vertex AI service account credentials","stage":"gemini_extraction","timestamp":"2025-11-29T10:20:00Z"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors.items.[INDEX]","typeLabel":"BatchErrorDetail · object"},"type":"array","title":"Errors","description":"Detailed errors from this extractor job","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].errors","typeLabel":"BatchErrorDetail · object[]"},"error":{"oneOf":[{"type":"string","title":"Error","description":"OPTIONAL. Simple error message string for quick debugging. Set when the Ray job fails with error details from JobStatusMonitor. For detailed error information, see errors array.","x-mixpeek-redacted":"driver_text","example":"Ray job FAILED","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].error.oneOf.0","typeLabel":"string"},{"type":"null","title":"Error","description":"OPTIONAL. Simple error message string for quick debugging. Set when the Ray job fails with error details from JobStatusMonitor. For detailed error information, see errors array.","x-mixpeek-redacted":"driver_text","example":"Ray job FAILED","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].error.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].error","typeLabel":"string | null"},"last_activity_at":{"oneOf":[{"type":"string","title":"Last Activity At","description":"OPTIONAL. Timestamp of the last BatchJobPoller heartbeat confirming this extractor job was non-terminal (RUNNING or PENDING). Updated approximately every 10 seconds while IN_PROGRESS. A stale value (minutes old) may indicate a stuck or lost job.","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].last_activity_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Last Activity At","description":"OPTIONAL. Timestamp of the last BatchJobPoller heartbeat confirming this extractor job was non-terminal (RUNNING or PENDING). Updated approximately every 10 seconds while IN_PROGRESS. A stale value (minutes old) may indicate a stuck or lost job.","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].last_activity_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].last_activity_at","typeLabel":"string\u003cdate-time> | null"},"ray_job_status":{"oneOf":[{"type":"string","title":"Ray Job Status","description":"OPTIONAL. Last observed Ray job status for this extractor job as of last_activity_at. Values: 'RUNNING', 'PENDING', 'SUCCEEDED', 'FAILED'. Use with last_activity_at to assess whether the job is actively running.","example":"RUNNING","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].ray_job_status.oneOf.0","typeLabel":"string"},{"type":"null","title":"Ray Job Status","description":"OPTIONAL. Last observed Ray job status for this extractor job as of last_activity_at. Values: 'RUNNING', 'PENDING', 'SUCCEEDED', 'FAILED'. Use with last_activity_at to assess whether the job is actively running.","example":"RUNNING","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].ray_job_status.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].ray_job_status","typeLabel":"string | null"},"submission_params":{"oneOf":[{"type":"object","title":"SubmissionParams","description":"Parameters used for Ray/GKE job submission. Persisted at submit time for debugging.","properties":{"entrypoint":{"oneOf":[{"type":"string","title":"Entrypoint","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.entrypoint.oneOf.0","typeLabel":"string"},{"type":"null","title":"Entrypoint","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.entrypoint.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.entrypoint","typeLabel":"string | null"},"deployment_mode":{"oneOf":[{"type":"string","title":"Deployment Mode","description":"'ray' (local/dev) or 'gke' (production).","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.deployment_mode.oneOf.0","typeLabel":"string"},{"type":"null","title":"Deployment Mode","description":"'ray' (local/dev) or 'gke' (production).","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.deployment_mode.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.deployment_mode","typeLabel":"string | null"},"requires_gpu":{"oneOf":[{"type":"boolean","title":"Requires Gpu","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.requires_gpu.oneOf.0","typeLabel":"boolean"},{"type":"null","title":"Requires Gpu","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.requires_gpu.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.requires_gpu","typeLabel":"boolean | null"},"num_cpus":{"oneOf":[{"type":"number","title":"Num Cpus","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.num_cpus.oneOf.0","typeLabel":"number"},{"type":"null","title":"Num Cpus","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.num_cpus.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.num_cpus","typeLabel":"number | null"},"num_gpus":{"oneOf":[{"type":"number","title":"Num Gpus","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.num_gpus.oneOf.0","typeLabel":"number"},{"type":"null","title":"Num Gpus","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.num_gpus.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.num_gpus","typeLabel":"number | null"},"memory_bytes":{"oneOf":[{"type":"integer","title":"Memory Bytes","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.memory_bytes.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Memory Bytes","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.memory_bytes.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.memory_bytes","typeLabel":"integer | null"},"priority":{"oneOf":[{"type":"integer","title":"Priority","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.priority.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Priority","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.priority.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.priority","typeLabel":"integer | null"},"plugin_archives":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_archives.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Plugin Archives","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_archives.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Plugin Archives","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_archives.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_archives","typeLabel":"string[] | null"},"plugin_dependencies":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_dependencies.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Plugin Dependencies","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_dependencies.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Plugin Dependencies","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_dependencies.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.plugin_dependencies","typeLabel":"string[] | null"},"image_uri":{"oneOf":[{"type":"string","title":"Image Uri","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.image_uri.oneOf.0","typeLabel":"string"},{"type":"null","title":"Image Uri","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.image_uri.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.image_uri","typeLabel":"string | null"},"extractor_name":{"oneOf":[{"type":"string","title":"Extractor Name","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.extractor_name.oneOf.0","typeLabel":"string"},{"type":"null","title":"Extractor Name","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.extractor_name.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.extractor_name","typeLabel":"string | null"},"extractor_version":{"oneOf":[{"type":"string","title":"Extractor Version","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.extractor_version.oneOf.0","typeLabel":"string"},{"type":"null","title":"Extractor Version","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.extractor_version.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.extractor_version","typeLabel":"string | null"},"env_vars_keys":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.env_vars_keys.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Env Vars Keys","description":"Environment variable keys (values omitted for security).","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.env_vars_keys.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Env Vars Keys","description":"Environment variable keys (values omitted for security).","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.env_vars_keys.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.env_vars_keys","typeLabel":"string[] | null"},"manifest_key":{"oneOf":[{"type":"string","title":"Manifest Key","description":"S3 key of the batch manifest (metadata.json) used for this job. The manifest survives Ray cluster rebuilds, so the poller can resubmit an in-cluster (raysubmit_*) job killed by a redeploy.","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.manifest_key.oneOf.0","typeLabel":"string"},{"type":"null","title":"Manifest Key","description":"S3 key of the batch manifest (metadata.json) used for this job. The manifest survives Ray cluster rebuilds, so the poller can resubmit an in-cluster (raysubmit_*) job killed by a redeploy.","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.manifest_key.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.manifest_key","typeLabel":"string | null"},"submitted_at":{"oneOf":[{"type":"string","title":"Submitted At","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.submitted_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Submitted At","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.submitted_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0.submitted_at","typeLabel":"string\u003cdate-time> | null"}},"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.0","typeLabel":"SubmissionParams · object"},{"type":"null","description":"Parameters used for Ray/GKE job submission. Persisted at submit time for debugging.","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX].submission_params","typeLabel":"SubmissionParams · object | null"}},"type":"object","required":["extractor_type"],"title":"ExtractorJobInfo","description":"Tracking information for a single feature extractor job within a tier.\n\nEach tier can have multiple extractor jobs running in parallel, one per unique\nfeature_extractor_type. This allows different extractors to have independent:\n- Resource requirements (GPUs, CPU, memory)\n- Status tracking and retry logic\n- Ray job IDs and background work IDs\n- Completion times and performance metrics\n\nExample:\n Tier 1 with collections using different extractors:\n - col_A: image_extractor\n - col_B: face_identity_extractor\n - col_C: image_extractor\n\n Creates 2 ExtractorJobInfo instances:\n 1. extractor_type=\"image_extractor\", collection_ids=[\"col_A\", \"col_C\"]\n 2. extractor_type=\"face_identity_extractor\", collection_ids=[\"col_B\"]\n\nLifecycle:\n 1. Created with status=PENDING when tier is scheduled\n 2. Updated to IN_PROGRESS when Ray job starts\n 3. Finalized to COMPLETED/FAILED when Ray job completes","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs.items.[INDEX]","typeLabel":"ExtractorJobInfo · object"},"type":"array","title":"Extractor Jobs","description":"List of extractor jobs for this tier (one per unique feature_extractor_type). NEW as of 2025-12-31: Tiers now support multiple Ray jobs. Empty list for backwards compatibility with old batches. Tier completes when ALL extractor_jobs reach COMPLETED status.","uniqueKey":"tier_tasks.items.[INDEX].extractor_jobs","typeLabel":"ExtractorJobInfo · object[]"},"source_collection_ids":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].source_collection_ids.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Source Collection Ids","description":"OPTIONAL for tier 0 (must be None). REQUIRED for tier N+ (N > 0). List of upstream collection IDs that provide documents as input to this tier. Typically contains collection IDs from the previous tier (tier_num - 1). Used by the API to query documents from these collections for processing. These upstream documents are converted to a Parquet manifest for the Engine. Example: If tier 1 processes 'col_frames' and sources from tier 0's 'col_chunks', then source_collection_ids=['col_chunks'].","example":["col_chunks"],"uniqueKey":"tier_tasks.items.[INDEX].source_collection_ids.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Source Collection Ids","description":"OPTIONAL for tier 0 (must be None). REQUIRED for tier N+ (N > 0). List of upstream collection IDs that provide documents as input to this tier. Typically contains collection IDs from the previous tier (tier_num - 1). Used by the API to query documents from these collections for processing. These upstream documents are converted to a Parquet manifest for the Engine. Example: If tier 1 processes 'col_frames' and sources from tier 0's 'col_chunks', then source_collection_ids=['col_chunks'].","example":["col_chunks"],"uniqueKey":"tier_tasks.items.[INDEX].source_collection_ids.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].source_collection_ids","typeLabel":"string[] | null"},"parent_task_id":{"oneOf":[{"type":"string","title":"Parent Task Id","description":"OPTIONAL. Task ID of the previous tier (tier_num - 1) that processed before this tier. Used to link tiers together for audit trail and lineage tracking. None for tier 0 (no parent). Enables queries like 'show all tiers that processed after tier 0' or 'trace back through all parent tiers to find the original batch'. Format: Same as task_id (e.g., 'task_tier0_abc123').","example":"task_tier0_abc123","uniqueKey":"tier_tasks.items.[INDEX].parent_task_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Parent Task Id","description":"OPTIONAL. Task ID of the previous tier (tier_num - 1) that processed before this tier. Used to link tiers together for audit trail and lineage tracking. None for tier 0 (no parent). Enables queries like 'show all tiers that processed after tier 0' or 'trace back through all parent tiers to find the original batch'. Format: Same as task_id (e.g., 'task_tier0_abc123').","example":"task_tier0_abc123","uniqueKey":"tier_tasks.items.[INDEX].parent_task_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].parent_task_id","typeLabel":"string | null"},"started_at":{"oneOf":[{"type":"string","title":"Started At","description":"OPTIONAL. ISO 8601 timestamp when this tier began processing. None if tier has not yet started (status=PENDING). Set using current_time() from shared.utilities.helpers when tier starts. Used to calculate tier processing duration and identify long-running tiers. Example: '2025-11-03T10:00:00Z'.","format":"date-time","example":"2025-11-03T10:00:00Z","uniqueKey":"tier_tasks.items.[INDEX].started_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Started At","description":"OPTIONAL. ISO 8601 timestamp when this tier began processing. None if tier has not yet started (status=PENDING). Set using current_time() from shared.utilities.helpers when tier starts. Used to calculate tier processing duration and identify long-running tiers. Example: '2025-11-03T10:00:00Z'.","example":"2025-11-03T10:00:00Z","uniqueKey":"tier_tasks.items.[INDEX].started_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].started_at","typeLabel":"string\u003cdate-time> | null"},"completed_at":{"oneOf":[{"type":"string","title":"Completed At","description":"OPTIONAL. ISO 8601 timestamp when this tier finished processing (success or failure). None if tier has not yet completed (status=PENDING or IN_PROGRESS). Set using current_time() from shared.utilities.helpers when tier completes. Used to calculate tier processing duration (completed_at - started_at). Set for both COMPLETED and FAILED statuses. Example: '2025-11-03T10:05:00Z'.","format":"date-time","example":"2025-11-03T10:05:00Z","uniqueKey":"tier_tasks.items.[INDEX].completed_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Completed At","description":"OPTIONAL. ISO 8601 timestamp when this tier finished processing (success or failure). None if tier has not yet completed (status=PENDING or IN_PROGRESS). Set using current_time() from shared.utilities.helpers when tier completes. Used to calculate tier processing duration (completed_at - started_at). Set for both COMPLETED and FAILED statuses. Example: '2025-11-03T10:05:00Z'.","example":"2025-11-03T10:05:00Z","uniqueKey":"tier_tasks.items.[INDEX].completed_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].completed_at","typeLabel":"string\u003cdate-time> | null"},"duration_ms":{"oneOf":[{"type":"number","title":"Duration Ms","description":"OPTIONAL. Processing duration in milliseconds for this tier. Calculated as (completed_at - started_at) when tier completes. None if tier has not yet completed or if started_at was not set. Provides a pre-computed duration for easy querying without timestamp math. Set for both COMPLETED and FAILED statuses.","example":300000,"uniqueKey":"tier_tasks.items.[INDEX].duration_ms.oneOf.0","typeLabel":"number"},{"type":"null","title":"Duration Ms","description":"OPTIONAL. Processing duration in milliseconds for this tier. Calculated as (completed_at - started_at) when tier completes. None if tier has not yet completed or if started_at was not set. Provides a pre-computed duration for easy querying without timestamp math. Set for both COMPLETED and FAILED statuses.","example":300000,"uniqueKey":"tier_tasks.items.[INDEX].duration_ms.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].duration_ms","typeLabel":"number | null"},"errors":{"items":{"properties":{"error_type":{"type":"string","enum":["dependency","authentication","validation","runtime","network","resource"],"title":"ErrorCategory","description":"REQUIRED. Category of error that occurred. Used for filtering, retry logic, and error analytics. DEPENDENCY: Missing packages/modules (e.g., google-genai not installed). AUTHENTICATION: Invalid credentials or expired tokens. VALIDATION: Schema mismatches or invalid input data. RUNTIME: Code exceptions or processing failures. NETWORK: Connectivity issues or timeouts. RESOURCE: Out of memory or disk space.","examples":["dependency","authentication","validation"],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].error_type","typeLabel":"enum\u003cstring>"},"message":{"type":"string","title":"Message","description":"REQUIRED. Human-readable error message. Concise description of what went wrong. Should be actionable and help users understand the issue.","examples":["Missing required package: google-genai","Invalid API key for Vertex AI","Schema validation failed: missing required field 'products'"],"isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].message","typeLabel":"string"},"component":{"oneOf":[{"type":"string","title":"Component","description":"OPTIONAL. Component or service where the error occurred. Helps identify which part of the system failed. Examples: service class names, module names, or feature names.","example":"VertexMultimodalService","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].component.oneOf.0","typeLabel":"string"},{"type":"null","title":"Component","description":"OPTIONAL. Component or service where the error occurred. Helps identify which part of the system failed. Examples: service class names, module names, or feature names.","example":"VertexMultimodalService","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].component.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].component","typeLabel":"string | null"},"stage":{"oneOf":[{"type":"string","title":"Stage","description":"OPTIONAL. Processing stage where the error occurred. Identifies which pipeline stage failed. Examples: pipeline stage names from collection configuration.","example":"gemini_extraction","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].stage.oneOf.0","typeLabel":"string"},{"type":"null","title":"Stage","description":"OPTIONAL. Processing stage where the error occurred. Identifies which pipeline stage failed. Examples: pipeline stage names from collection configuration.","example":"gemini_extraction","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].stage.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].stage","typeLabel":"string | null"},"traceback":{"oneOf":[{"type":"string","title":"Traceback","description":"OPTIONAL. Full Python traceback for debugging. Includes stack trace for code-level troubleshooting. Should be truncated if too long (e.g., max 2000 chars).","x-mixpeek-redacted":"driver_text","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].traceback.oneOf.0","typeLabel":"string"},{"type":"null","title":"Traceback","description":"OPTIONAL. Full Python traceback for debugging. Includes stack trace for code-level troubleshooting. Should be truncated if too long (e.g., max 2000 chars).","x-mixpeek-redacted":"driver_text","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].traceback.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].traceback","typeLabel":"string | null"},"timestamp":{"type":"string","format":"date-time","title":"Timestamp","description":"REQUIRED. ISO 8601 timestamp when the error occurred. Used for chronological error tracking and debugging.","examples":["2025-11-29T10:15:30Z"],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].timestamp","typeLabel":"string\u003cdate-time>"},"affected_document_ids":{"items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].affected_document_ids.items.[INDEX]","typeLabel":"string"},"type":"array","title":"Affected Document Ids","description":"OPTIONAL. List of document IDs affected by this error. For object-level errors: contains single document ID. For batch-level aggregation: contains all affected document IDs. Used to identify scope of impact.","examples":[["doc_123"],["doc_123","doc_456","doc_789"]],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].affected_document_ids","typeLabel":"string[]"},"affected_count":{"type":"integer","minimum":1,"title":"Affected Count","description":"REQUIRED. Number of documents affected by this error. For object-level: typically 1. For batch-level aggregation: total count of affected documents. Used for error impact analysis.","default":1,"examples":[1,10,50],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].affected_count","typeLabel":"integer"},"recovery_suggestion":{"oneOf":[{"type":"string","title":"Recovery Suggestion","description":"OPTIONAL. Actionable suggestion for resolving the error. Helps users quickly fix common issues. Examples: install missing package, check credentials, update schema.","example":"Install google-genai package: pip install google-genai","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].recovery_suggestion.oneOf.0","typeLabel":"string"},{"type":"null","title":"Recovery Suggestion","description":"OPTIONAL. Actionable suggestion for resolving the error. Helps users quickly fix common issues. Examples: install missing package, check credentials, update schema.","example":"Install google-genai package: pip install google-genai","uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].recovery_suggestion.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].recovery_suggestion","typeLabel":"string | null"},"metadata":{"additionalProperties":true,"type":"object","title":"Metadata","description":"OPTIONAL. Additional error context and metadata. Free-form dictionary for error-specific details. Examples: retry_count, last_retry_at, error_code, http_status.","examples":[{"last_retry_at":"2025-11-29T10:10:00Z","retry_count":2},{"api_endpoint":"https://vertex.googleapis.com","http_status":401}],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX].metadata","typeLabel":"Metadata · object"}},"type":"object","required":["error_type","message"],"title":"BatchErrorDetail","description":"Detailed error information for batch processing failures.\n\nProvides structured error tracking at both object and batch levels.\nEnables better debugging, retry logic, and error analytics.\n\nUse Cases:\n - Track specific errors that occurred during processing\n - Identify error patterns across multiple documents\n - Provide actionable recovery suggestions\n - Enable intelligent retry logic based on error type\n\nObject-level tracking: Attached to individual document processing failures\nBatch-level tracking: Aggregated summaries in batch metadata","examples":[{"affected_count":1,"affected_document_ids":["doc_123"],"component":"VertexMultimodalService","description":"Dependency error: Missing package","error_type":"dependency","message":"Cannot import 'genai' from 'google' package","metadata":{"import_path":"google.genai","package_name":"google-genai"},"recovery_suggestion":"Install google-genai package: pip install google-genai","stage":"gemini_extraction","timestamp":"2025-11-29T10:15:30Z","traceback":"ImportError: cannot import name 'genai' from 'google'..."},{"affected_count":2,"affected_document_ids":["doc_456","doc_789"],"component":"VertexMultimodalService","description":"Authentication error: Invalid credentials","error_type":"authentication","message":"Invalid grant: Bad Request","metadata":{"http_status":401,"retry_count":3},"recovery_suggestion":"Verify your Vertex AI service account credentials","stage":"gemini_extraction","timestamp":"2025-11-29T10:20:00Z"}],"uniqueKey":"tier_tasks.items.[INDEX].errors.items.[INDEX]","typeLabel":"BatchErrorDetail · object"},"type":"array","title":"Errors","description":"OPTIONAL. List of detailed errors that occurred during tier processing. Empty list if tier succeeded or has not yet completed. Each error includes: error_type, message, component, stage, traceback, timestamp. Multiple errors may occur if different documents fail with different issues. Used for detailed error analysis, debugging, and intelligent retry logic. Example: Multiple documents failing with different errors (dependency vs auth). For backward compatibility, check if list is empty for success/in-progress status.","examples":[[],[{"affected_count":5,"component":"VertexMultimodalService","error_type":"dependency","message":"Missing required package: google-genai","recovery_suggestion":"Install google-genai package","stage":"gemini_extraction"}],[{"affected_count":10,"component":"VertexMultimodalService","error_type":"authentication","message":"Invalid Vertex AI credentials"},{"affected_count":3,"component":"GeminiExtractor","error_type":"validation","message":"Schema validation failed"}]],"uniqueKey":"tier_tasks.items.[INDEX].errors","typeLabel":"BatchErrorDetail · object[]"},"error_summary":{"oneOf":[{"type":"object","title":"Error Summary","description":"OPTIONAL. Aggregated summary of errors by error type. Maps error_type (category) to count of affected DOCUMENTS. None has THREE causes, not two: the tier succeeded, the tier has not yet completed, OR the tier FAILED AT TIER LEVEL — it wrote zero documents (infrastructure failure, job crash, no resolvable inputs), so no individual document carries an error and this histogram is legitimately empty. So None does NOT mean 'no errors', and None alongside COMPLETED_WITH_ERRORS is NOT a contradiction. For a tier-level failure the diagnosis is in `failure_reason` and `tier_tasks[].error`, which carry the Ray job id and where to look. Provides quick overview of error distribution without parsing full error list. Example: {'dependency': 5, 'authentication': 10, 'validation': 3} means 5 documents failed with dependency errors, 10 with auth errors, 3 with validation. Automatically generated from errors list for convenience. Used for batch health monitoring and error trend analysis.","example":null,"additionalProperties":{"type":"integer","uniqueKey":"tier_tasks.items.[INDEX].error_summary.oneOf.0.additionalProperties","typeLabel":"integer"},"uniqueKey":"tier_tasks.items.[INDEX].error_summary.oneOf.0","typeLabel":"Error Summary · object"},{"type":"null","title":"Error Summary","description":"OPTIONAL. Aggregated summary of errors by error type. Maps error_type (category) to count of affected DOCUMENTS. None has THREE causes, not two: the tier succeeded, the tier has not yet completed, OR the tier FAILED AT TIER LEVEL — it wrote zero documents (infrastructure failure, job crash, no resolvable inputs), so no individual document carries an error and this histogram is legitimately empty. So None does NOT mean 'no errors', and None alongside COMPLETED_WITH_ERRORS is NOT a contradiction. For a tier-level failure the diagnosis is in `failure_reason` and `tier_tasks[].error`, which carry the Ray job id and where to look. Provides quick overview of error distribution without parsing full error list. Example: {'dependency': 5, 'authentication': 10, 'validation': 3} means 5 documents failed with dependency errors, 10 with auth errors, 3 with validation. Automatically generated from errors list for convenience. Used for batch health monitoring and error trend analysis.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].error_summary.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].error_summary","typeLabel":"Error Summary · object | null"},"performance":{"oneOf":[{"type":"object","title":"Performance","description":"OPTIONAL. Performance metrics summary for this tier's execution. Automatically populated after tier completion by collecting data from the analytics store. Contains: total_time_ms (total execution time), avg_latency_ms (average operation latency), bottlenecks (list of slowest operations), stage_count (number of profiled stages). Used for troubleshooting performance issues and identifying bottlenecks. None if tier has not completed or performance data collection failed. Populated asynchronously (non-blocking, best-effort).","example":{"avg_latency_ms":234.56,"bottlenecks":[{"avg_time_ms":113.58,"execution_count":50,"max_time_ms":234.56,"stage_name":"gcs_batch_upload_all_segments","total_time_ms":5678.9},{"avg_time_ms":69.14,"execution_count":50,"max_time_ms":123.45,"stage_name":"pipeline_run","total_time_ms":3456.78}],"stage_count":5,"timestamp":"2025-11-06T10:05:00Z","total_time_ms":12345.67},"additionalProperties":true,"uniqueKey":"tier_tasks.items.[INDEX].performance.oneOf.0","typeLabel":"Performance · object"},{"type":"null","title":"Performance","description":"OPTIONAL. Performance metrics summary for this tier's execution. Automatically populated after tier completion by collecting data from the analytics store. Contains: total_time_ms (total execution time), avg_latency_ms (average operation latency), bottlenecks (list of slowest operations), stage_count (number of profiled stages). Used for troubleshooting performance issues and identifying bottlenecks. None if tier has not completed or performance data collection failed. Populated asynchronously (non-blocking, best-effort).","example":{"avg_latency_ms":234.56,"bottlenecks":[{"avg_time_ms":113.58,"execution_count":50,"max_time_ms":234.56,"stage_name":"gcs_batch_upload_all_segments","total_time_ms":5678.9},{"avg_time_ms":69.14,"execution_count":50,"max_time_ms":123.45,"stage_name":"pipeline_run","total_time_ms":3456.78}],"stage_count":5,"timestamp":"2025-11-06T10:05:00Z","total_time_ms":12345.67},"uniqueKey":"tier_tasks.items.[INDEX].performance.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].performance","typeLabel":"Performance · object | null"},"ray_job_id":{"oneOf":[{"type":"string","title":"Ray Job Id","description":"OPTIONAL. Ray/Anyscale job ID for tracking the infrastructure-level processing job. None if tier has not yet started or if the job ID was not returned by the engine. Set when tier processing is submitted to Ray/Anyscale via the Engine. Used for cancelling running jobs and monitoring infrastructure-level status. Format: 'raysubmit_' prefix followed by job identifier (e.g., 'raysubmit_9pDAyZbd5MN281TB'). This is the job ID that appears in the Ray/Anyscale dashboard.","example":"raysubmit_9pDAyZbd5MN281TB","uniqueKey":"tier_tasks.items.[INDEX].ray_job_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Ray Job Id","description":"OPTIONAL. Ray/Anyscale job ID for tracking the infrastructure-level processing job. None if tier has not yet started or if the job ID was not returned by the engine. Set when tier processing is submitted to Ray/Anyscale via the Engine. Used for cancelling running jobs and monitoring infrastructure-level status. Format: 'raysubmit_' prefix followed by job identifier (e.g., 'raysubmit_9pDAyZbd5MN281TB'). This is the job ID that appears in the Ray/Anyscale dashboard.","example":"raysubmit_9pDAyZbd5MN281TB","uniqueKey":"tier_tasks.items.[INDEX].ray_job_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].ray_job_id","typeLabel":"string | null"},"ray_job_started_at":{"oneOf":[{"type":"string","title":"Ray Job Started At","description":"OPTIONAL. When the Ray job itself began running, as reported by Ray/KubeRay (RayJob CR status.startTime), NOT when the API claimed the tier. Differs from started_at, which is stamped by the background worker before the job is even submitted, so started_at includes manifest build and submission time. Use this to separate control-plane overhead from actual engine execution.","format":"date-time","example":null,"uniqueKey":"tier_tasks.items.[INDEX].ray_job_started_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Ray Job Started At","description":"OPTIONAL. When the Ray job itself began running, as reported by Ray/KubeRay (RayJob CR status.startTime), NOT when the API claimed the tier. Differs from started_at, which is stamped by the background worker before the job is even submitted, so started_at includes manifest build and submission time. Use this to separate control-plane overhead from actual engine execution.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].ray_job_started_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].ray_job_started_at","typeLabel":"string\u003cdate-time> | null"},"ray_job_completed_at":{"oneOf":[{"type":"string","title":"Ray Job Completed At","description":"OPTIONAL. When the Ray job finished, as reported by Ray/KubeRay (RayJob CR status.endTime). Paired with ray_job_started_at; the difference between them is the engine-side execution time, whereas completed_at - started_at also contains API and background-worker time.","format":"date-time","example":null,"uniqueKey":"tier_tasks.items.[INDEX].ray_job_completed_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Ray Job Completed At","description":"OPTIONAL. When the Ray job finished, as reported by Ray/KubeRay (RayJob CR status.endTime). Paired with ray_job_started_at; the difference between them is the engine-side execution time, whereas completed_at - started_at also contains API and background-worker time.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].ray_job_completed_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].ray_job_completed_at","typeLabel":"string\u003cdate-time> | null"},"requires_gpu":{"oneOf":[{"type":"boolean","title":"Requires Gpu","description":"OPTIONAL. Whether this tier's Ray job was scheduled onto a GPU worker group. Populated at submit time from the engine's requires_gpu decision (built-in extractors default to GPU; custom plugins opt in via compute_profile). Lets users confirm a custom plugin actually landed on a GPU without kubectl access.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].requires_gpu.oneOf.0","typeLabel":"boolean"},{"type":"null","title":"Requires Gpu","description":"OPTIONAL. Whether this tier's Ray job was scheduled onto a GPU worker group. Populated at submit time from the engine's requires_gpu decision (built-in extractors default to GPU; custom plugins opt in via compute_profile). Lets users confirm a custom plugin actually landed on a GPU without kubectl access.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].requires_gpu.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].requires_gpu","typeLabel":"boolean | null"},"worker_groups":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].worker_groups.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Worker Groups","description":"OPTIONAL. Names of the Ray worker groups this tier's job is eligible to run on. Derived from the compute profile chosen at submit time (e.g., ['gpu-workers'] for requires_gpu=True, ['cpu-workers'] otherwise). Surfaces resource allocation in the batch response so users don't need kubectl access to debug scheduling.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].worker_groups.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Worker Groups","description":"OPTIONAL. Names of the Ray worker groups this tier's job is eligible to run on. Derived from the compute profile chosen at submit time (e.g., ['gpu-workers'] for requires_gpu=True, ['cpu-workers'] otherwise). Surfaces resource allocation in the batch response so users don't need kubectl access to debug scheduling.","example":null,"uniqueKey":"tier_tasks.items.[INDEX].worker_groups.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].worker_groups","typeLabel":"string[] | null"},"celery_task_id":{"oneOf":[{"type":"string","title":"Task Id","description":"OPTIONAL. Background task ID for tracking the worker processing this tier. None if tier has not yet started or is not processed via background task. Set when the tier processing task is triggered. Used for revoking pending/running tasks during batch cancellation or deletion. Format: UUID string (e.g., 'a1b2c3d4-e5f6-7890-abcd-ef1234567890').","example":"a1b2c3d4-e5f6-7890-abcd-ef1234567890","uniqueKey":"tier_tasks.items.[INDEX].celery_task_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Task Id","description":"OPTIONAL. Background task ID for tracking the worker processing this tier. None if tier has not yet started or is not processed via background task. Set when the tier processing task is triggered. Used for revoking pending/running tasks during batch cancellation or deletion. Format: UUID string (e.g., 'a1b2c3d4-e5f6-7890-abcd-ef1234567890').","example":"a1b2c3d4-e5f6-7890-abcd-ef1234567890","uniqueKey":"tier_tasks.items.[INDEX].celery_task_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].celery_task_id","typeLabel":"string | null"},"source_documents_fetched":{"oneOf":[{"type":"integer","title":"Source Documents Fetched","description":"OPTIONAL. Number of documents fetched from source collection(s) for Tier N processing. For Tier 0 (bucket source), this is the number of objects from the bucket. For Tier N+ (collection source), this is the count of documents from upstream collection(s). Set at the start of tier artifact building in build_tier_n_artifacts(). If 0, the source collection is empty - check upstream tier completion.","example":100,"uniqueKey":"tier_tasks.items.[INDEX].source_documents_fetched.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Source Documents Fetched","description":"OPTIONAL. Number of documents fetched from source collection(s) for Tier N processing. For Tier 0 (bucket source), this is the number of objects from the bucket. For Tier N+ (collection source), this is the count of documents from upstream collection(s). Set at the start of tier artifact building in build_tier_n_artifacts(). If 0, the source collection is empty - check upstream tier completion.","example":100,"uniqueKey":"tier_tasks.items.[INDEX].source_documents_fetched.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].source_documents_fetched","typeLabel":"integer | null"},"documents_after_source_filter":{"oneOf":[{"type":"integer","title":"Documents After Source Filter","description":"OPTIONAL. Number of documents remaining after applying source_filters. source_filters are optional conditions that exclude documents from processing. If this is 0 but source_documents_fetched > 0, your source_filters are too restrictive. Check that filter fields exist in source documents and conditions match expected values.","example":95,"uniqueKey":"tier_tasks.items.[INDEX].documents_after_source_filter.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents After Source Filter","description":"OPTIONAL. Number of documents remaining after applying source_filters. source_filters are optional conditions that exclude documents from processing. If this is 0 but source_documents_fetched > 0, your source_filters are too restrictive. Check that filter fields exist in source documents and conditions match expected values.","example":95,"uniqueKey":"tier_tasks.items.[INDEX].documents_after_source_filter.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_after_source_filter","typeLabel":"integer | null"},"documents_missing_input_fields":{"oneOf":[{"type":"integer","title":"Documents Missing Input Fields","description":"OPTIONAL. Number of documents missing required input_mapping fields. input_mappings define which fields from source documents map to extractor inputs. If this equals documents_after_source_filter, ALL documents are missing required fields. Common cause: upstream extractor didn't produce expected output (e.g., video_segment_url). Check upstream extractor configuration and verify output field names.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].documents_missing_input_fields.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Missing Input Fields","description":"OPTIONAL. Number of documents missing required input_mapping fields. input_mappings define which fields from source documents map to extractor inputs. If this equals documents_after_source_filter, ALL documents are missing required fields. Common cause: upstream extractor didn't produce expected output (e.g., video_segment_url). Check upstream extractor configuration and verify output field names.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].documents_missing_input_fields.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_missing_input_fields","typeLabel":"integer | null"},"documents_submitted_to_engine":{"oneOf":[{"type":"integer","title":"Documents Submitted To Engine","description":"OPTIONAL. Number of documents actually submitted to the Ray/Engine for processing. This is documents_after_source_filter minus documents_missing_input_fields. If 0, no documents were sent to the engine - check source_filters and input_mappings. If > 0 but documents_written = 0, the engine failed to process documents.","example":95,"uniqueKey":"tier_tasks.items.[INDEX].documents_submitted_to_engine.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Submitted To Engine","description":"OPTIONAL. Number of documents actually submitted to the Ray/Engine for processing. This is documents_after_source_filter minus documents_missing_input_fields. If 0, no documents were sent to the engine - check source_filters and input_mappings. If > 0 but documents_written = 0, the engine failed to process documents.","example":95,"uniqueKey":"tier_tasks.items.[INDEX].documents_submitted_to_engine.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_submitted_to_engine","typeLabel":"integer | null"},"documents_written":{"oneOf":[{"type":"integer","title":"Documents Written","description":"OPTIONAL. Documents THIS BATCH wrote in this tier. Sourced from the tier's extractor jobs (extractor_jobs[].documents_written), which report per batch. It is NOT a collection total: do not subtract documents_before_processing from it. If 0 but documents_submitted_to_engine > 0, check tier errors for processing failures.","example":95,"uniqueKey":"tier_tasks.items.[INDEX].documents_written.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Written","description":"OPTIONAL. Documents THIS BATCH wrote in this tier. Sourced from the tier's extractor jobs (extractor_jobs[].documents_written), which report per batch. It is NOT a collection total: do not subtract documents_before_processing from it. If 0 but documents_submitted_to_engine > 0, check tier errors for processing failures.","example":95,"uniqueKey":"tier_tasks.items.[INDEX].documents_written.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_written","typeLabel":"integer | null"},"documents_replaced":{"oneOf":[{"type":"integer","title":"Documents Replaced","description":"OPTIONAL. For dedup_strategy replace: documents this tier wrote for objects it replaced. Absent when the batch is not a replace.","uniqueKey":"tier_tasks.items.[INDEX].documents_replaced.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Replaced","description":"OPTIONAL. For dedup_strategy replace: documents this tier wrote for objects it replaced. Absent when the batch is not a replace.","uniqueKey":"tier_tasks.items.[INDEX].documents_replaced.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_replaced","typeLabel":"integer | null"},"documents_deleted":{"oneOf":[{"type":"integer","title":"Documents Deleted","description":"OPTIONAL. For dedup_strategy replace: existing documents of the replaced objects removed from this tier's collections after the new ones were written. Absent when the batch is not a replace.","uniqueKey":"tier_tasks.items.[INDEX].documents_deleted.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Deleted","description":"OPTIONAL. For dedup_strategy replace: existing documents of the replaced objects removed from this tier's collections after the new ones were written. Absent when the batch is not a replace.","uniqueKey":"tier_tasks.items.[INDEX].documents_deleted.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_deleted","typeLabel":"integer | null"},"documents_before_processing":{"oneOf":[{"type":"integer","title":"Documents Before Processing","description":"OPTIONAL. Document count in target collection(s) before this tier started processing. DIAGNOSTIC ONLY. Do NOT compute (documents_written - documents_before_processing): documents_written is already this batch's own output, and this field is a WHOLE-COLLECTION count, so the difference attributes concurrent batches' writes to this one . It is retained as the input to a fallback used only when no extractor reported a count.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].documents_before_processing.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Before Processing","description":"OPTIONAL. Document count in target collection(s) before this tier started processing. DIAGNOSTIC ONLY. Do NOT compute (documents_written - documents_before_processing): documents_written is already this batch's own output, and this field is a WHOLE-COLLECTION count, so the difference attributes concurrent batches' writes to this one . It is retained as the input to a fallback used only when no extractor reported a count.","example":0,"uniqueKey":"tier_tasks.items.[INDEX].documents_before_processing.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].documents_before_processing","typeLabel":"integer | null"},"last_activity_at":{"oneOf":[{"type":"string","title":"Last Activity At","description":"OPTIONAL. Timestamp of the last BatchJobPoller heartbeat confirming the tier's Ray job(s) were non-terminal (RUNNING or PENDING). Updated approximately every 10 seconds while the tier is IN_PROGRESS. A stale last_activity_at (minutes old) indicates the job may be stalled or lost. Use this to distinguish an actively running batch from one that is silently stuck.","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].last_activity_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Last Activity At","description":"OPTIONAL. Timestamp of the last BatchJobPoller heartbeat confirming the tier's Ray job(s) were non-terminal (RUNNING or PENDING). Updated approximately every 10 seconds while the tier is IN_PROGRESS. A stale last_activity_at (minutes old) indicates the job may be stalled or lost. Use this to distinguish an actively running batch from one that is silently stuck.","uniqueKey":"tier_tasks.items.[INDEX].last_activity_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].last_activity_at","typeLabel":"string\u003cdate-time> | null"},"ray_job_status":{"oneOf":[{"type":"string","title":"Ray Job Status","description":"OPTIONAL. Last observed Ray job status as of last_activity_at. Values: 'RUNNING' (actively processing), 'PENDING' (queued, not yet started), 'SUCCEEDED', 'FAILED', 'DEDUP_SKIPPED' (every object was already processed, so no job ran), 'REEXTRACT_GUARD_SKIPPED' (the organization does not allow re-extraction, so this tier did not run and wrote nothing; see status_message), 'SCOPED_EMPTY' (none of the batch's objects has documents in this tier's sources). For multi-extractor tiers, see extractor_jobs[].ray_job_status for per-job granularity.","example":"RUNNING","uniqueKey":"tier_tasks.items.[INDEX].ray_job_status.oneOf.0","typeLabel":"string"},{"type":"null","title":"Ray Job Status","description":"OPTIONAL. Last observed Ray job status as of last_activity_at. Values: 'RUNNING' (actively processing), 'PENDING' (queued, not yet started), 'SUCCEEDED', 'FAILED', 'DEDUP_SKIPPED' (every object was already processed, so no job ran), 'REEXTRACT_GUARD_SKIPPED' (the organization does not allow re-extraction, so this tier did not run and wrote nothing; see status_message), 'SCOPED_EMPTY' (none of the batch's objects has documents in this tier's sources). For multi-extractor tiers, see extractor_jobs[].ray_job_status for per-job granularity.","example":"RUNNING","uniqueKey":"tier_tasks.items.[INDEX].ray_job_status.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].ray_job_status","typeLabel":"string | null"},"ray_job_logs":{"oneOf":[{"type":"string","title":"Ray Job Logs","description":"OPTIONAL. Persisted Ray job logs captured when the job reached a terminal state (SUCCEEDED or FAILED). Contains the last 500 lines of the head pod's stdout. Populated automatically by the batch poller before pods are cleaned up by K8s, so logs remain available for debugging after the job's infrastructure is gone.","uniqueKey":"tier_tasks.items.[INDEX].ray_job_logs.oneOf.0","typeLabel":"string"},{"type":"null","title":"Ray Job Logs","description":"OPTIONAL. Persisted Ray job logs captured when the job reached a terminal state (SUCCEEDED or FAILED). Contains the last 500 lines of the head pod's stdout. Populated automatically by the batch poller before pods are cleaned up by K8s, so logs remain available for debugging after the job's infrastructure is gone.","uniqueKey":"tier_tasks.items.[INDEX].ray_job_logs.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].ray_job_logs","typeLabel":"string | null"},"ray_job_logs_captured_at":{"oneOf":[{"type":"string","title":"Ray Job Logs Captured At","description":"OPTIONAL. Timestamp when ray_job_logs were captured.","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].ray_job_logs_captured_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Ray Job Logs Captured At","description":"OPTIONAL. Timestamp when ray_job_logs were captured.","uniqueKey":"tier_tasks.items.[INDEX].ray_job_logs_captured_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].ray_job_logs_captured_at","typeLabel":"string\u003cdate-time> | null"},"submission_params":{"oneOf":[{"type":"object","title":"SubmissionParams","description":"OPTIONAL. Parameters used for the first Ray/GKE job submission in this tier. For per-extractor params, see extractor_jobs[].submission_params.","properties":{"entrypoint":{"oneOf":[{"type":"string","title":"Entrypoint","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.entrypoint.oneOf.0","typeLabel":"string"},{"type":"null","title":"Entrypoint","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.entrypoint.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.entrypoint","typeLabel":"string | null"},"deployment_mode":{"oneOf":[{"type":"string","title":"Deployment Mode","description":"'ray' (local/dev) or 'gke' (production).","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.deployment_mode.oneOf.0","typeLabel":"string"},{"type":"null","title":"Deployment Mode","description":"'ray' (local/dev) or 'gke' (production).","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.deployment_mode.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.deployment_mode","typeLabel":"string | null"},"requires_gpu":{"oneOf":[{"type":"boolean","title":"Requires Gpu","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.requires_gpu.oneOf.0","typeLabel":"boolean"},{"type":"null","title":"Requires Gpu","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.requires_gpu.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.requires_gpu","typeLabel":"boolean | null"},"num_cpus":{"oneOf":[{"type":"number","title":"Num Cpus","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.num_cpus.oneOf.0","typeLabel":"number"},{"type":"null","title":"Num Cpus","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.num_cpus.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.num_cpus","typeLabel":"number | null"},"num_gpus":{"oneOf":[{"type":"number","title":"Num Gpus","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.num_gpus.oneOf.0","typeLabel":"number"},{"type":"null","title":"Num Gpus","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.num_gpus.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.num_gpus","typeLabel":"number | null"},"memory_bytes":{"oneOf":[{"type":"integer","title":"Memory Bytes","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.memory_bytes.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Memory Bytes","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.memory_bytes.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.memory_bytes","typeLabel":"integer | null"},"priority":{"oneOf":[{"type":"integer","title":"Priority","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.priority.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Priority","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.priority.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.priority","typeLabel":"integer | null"},"plugin_archives":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_archives.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Plugin Archives","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_archives.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Plugin Archives","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_archives.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_archives","typeLabel":"string[] | null"},"plugin_dependencies":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_dependencies.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Plugin Dependencies","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_dependencies.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Plugin Dependencies","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_dependencies.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.plugin_dependencies","typeLabel":"string[] | null"},"image_uri":{"oneOf":[{"type":"string","title":"Image Uri","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.image_uri.oneOf.0","typeLabel":"string"},{"type":"null","title":"Image Uri","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.image_uri.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.image_uri","typeLabel":"string | null"},"extractor_name":{"oneOf":[{"type":"string","title":"Extractor Name","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.extractor_name.oneOf.0","typeLabel":"string"},{"type":"null","title":"Extractor Name","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.extractor_name.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.extractor_name","typeLabel":"string | null"},"extractor_version":{"oneOf":[{"type":"string","title":"Extractor Version","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.extractor_version.oneOf.0","typeLabel":"string"},{"type":"null","title":"Extractor Version","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.extractor_version.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.extractor_version","typeLabel":"string | null"},"env_vars_keys":{"oneOf":[{"type":"array","items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.env_vars_keys.oneOf.0.items.[INDEX]","typeLabel":"string"},"title":"Env Vars Keys","description":"Environment variable keys (values omitted for security).","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.env_vars_keys.oneOf.0","typeLabel":"string[]"},{"type":"null","title":"Env Vars Keys","description":"Environment variable keys (values omitted for security).","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.env_vars_keys.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.env_vars_keys","typeLabel":"string[] | null"},"manifest_key":{"oneOf":[{"type":"string","title":"Manifest Key","description":"S3 key of the batch manifest (metadata.json) used for this job. The manifest survives Ray cluster rebuilds, so the poller can resubmit an in-cluster (raysubmit_*) job killed by a redeploy.","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.manifest_key.oneOf.0","typeLabel":"string"},{"type":"null","title":"Manifest Key","description":"S3 key of the batch manifest (metadata.json) used for this job. The manifest survives Ray cluster rebuilds, so the poller can resubmit an in-cluster (raysubmit_*) job killed by a redeploy.","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.manifest_key.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.manifest_key","typeLabel":"string | null"},"submitted_at":{"oneOf":[{"type":"string","title":"Submitted At","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.submitted_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Submitted At","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.submitted_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0.submitted_at","typeLabel":"string\u003cdate-time> | null"}},"uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.0","typeLabel":"SubmissionParams · object"},{"type":"null","description":"OPTIONAL. Parameters used for the first Ray/GKE job submission in this tier. For per-extractor params, see extractor_jobs[].submission_params.","uniqueKey":"tier_tasks.items.[INDEX].submission_params.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].submission_params","typeLabel":"SubmissionParams · object | null"},"infrastructure_events":{"items":{"properties":{"event_type":{"type":"string","enum":["oom","preemption","node_failure","ray_bug","rolling_update","unknown"],"title":"InfraEventType","isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].event_type","typeLabel":"enum\u003cstring>"},"detected_at":{"oneOf":[{"type":"string","title":"Detected At","format":"date-time","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].detected_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Detected At","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].detected_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].detected_at","typeLabel":"string\u003cdate-time> | null"},"raw_signal":{"oneOf":[{"type":"string","title":"Raw Signal","description":"The error text that triggered classification.","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].raw_signal.oneOf.0","typeLabel":"string"},{"type":"null","title":"Raw Signal","description":"The error text that triggered classification.","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].raw_signal.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].raw_signal","typeLabel":"string | null"},"node_id":{"oneOf":[{"type":"string","title":"Node Id","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].node_id.oneOf.0","typeLabel":"string"},{"type":"null","title":"Node Id","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].node_id.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].node_id","typeLabel":"string | null"},"pod_name":{"oneOf":[{"type":"string","title":"Pod Name","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].pod_name.oneOf.0","typeLabel":"string"},{"type":"null","title":"Pod Name","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].pod_name.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX].pod_name","typeLabel":"string | null"}},"type":"object","required":["event_type"],"title":"InfrastructureDetail","description":"A single infrastructure event correlated with a tier execution.","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events.items.[INDEX]","typeLabel":"InfrastructureDetail · object"},"type":"array","title":"Infrastructure Events","description":"Infrastructure-level events correlated with this tier's execution (OOM, preemption, etc.).","uniqueKey":"tier_tasks.items.[INDEX].infrastructure_events","typeLabel":"InfrastructureDetail · object[]"},"audit":{"oneOf":[{"type":"object","title":"TierAuditInfo","description":"OPTIONAL. Tier completion invariant audit, persisted by the complete_tier callback. Phase 1 of INGESTION_RELIABILITY_PLAN.md. ``lost > 0`` means objects were submitted but absent from both processed_objects and failed_documents — investigate via the /audit endpoint.","required":["tier_num"],"properties":{"tier_num":{"type":"integer","title":"Tier Num","description":"Zero-based index of the tier.","isRequired":true,"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.tier_num","typeLabel":"integer"},"submitted":{"oneOf":[{"type":"integer","title":"Submitted","description":"Objects submitted to this tier. None when not yet recorded, which is the normal state for a tier still in flight.","uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.submitted.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Submitted","description":"Objects submitted to this tier. None when not yet recorded, which is the normal state for a tier still in flight.","uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.submitted.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.submitted","typeLabel":"integer | null"},"processed":{"type":"integer","title":"Processed","description":"Objects that produced a document.","default":0,"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.processed","typeLabel":"integer"},"failed":{"type":"integer","title":"Failed","description":"Objects recorded in failed_documents.","default":0,"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.failed","typeLabel":"integer"},"skipped":{"type":"integer","title":"Skipped","description":"Objects deliberately not processed.","default":0,"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.skipped","typeLabel":"integer"},"lost":{"type":"integer","title":"Lost","description":"max(0, submitted - processed - failed - skipped), and 0 when the tier balances. Above 0 means objects were submitted and are absent from BOTH processed and failed, which is the case worth escalating.","default":0,"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.lost","typeLabel":"integer"},"balanced":{"type":"boolean","title":"Balanced","description":"True only when submitted is known AND equals processed + failed + skipped.","default":false,"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.balanced","typeLabel":"boolean"},"notes":{"items":{"type":"string","uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.notes.items.[INDEX]","typeLabel":"string"},"type":"array","title":"Notes","description":"Human-readable notes recorded while auditing the tier.","uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.notes","typeLabel":"string[]"},"durability":{"oneOf":[{"type":"string","title":"Durability","description":"Write-durability verdict for this tier. 'confirmed' means the namespace's durable frontier had caught up to the highest write this batch recorded. 'lagging' means it had not, and that is the only value that downgrades the tier to COMPLETED_WITH_ERRORS, so it is how you tell a durability lag apart from lost rows. 'untracked' means the batch recorded no durably-tracked write to confirm. 'frontier_unknown' means the frontier could not be read, and a tier is never downgraded on that. Null means no verdict was recorded, which includes the single-tier finalize path that can downgrade without writing an audit at all. Null is not a statement that the writes are durable.","enum":["confirmed","lagging","untracked","frontier_unknown"],"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.durability.oneOf.0","typeLabel":"enum\u003cstring>"},{"type":"null","title":"Durability","description":"Write-durability verdict for this tier. 'confirmed' means the namespace's durable frontier had caught up to the highest write this batch recorded. 'lagging' means it had not, and that is the only value that downgrades the tier to COMPLETED_WITH_ERRORS, so it is how you tell a durability lag apart from lost rows. 'untracked' means the batch recorded no durably-tracked write to confirm. 'frontier_unknown' means the frontier could not be read, and a tier is never downgraded on that. Null means no verdict was recorded, which includes the single-tier finalize path that can downgrade without writing an audit at all. Null is not a statement that the writes are durable.","uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.durability.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0.durability","typeLabel":"enum\u003cstring> | null"}},"uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.0","typeLabel":"TierAuditInfo · object"},{"type":"null","description":"OPTIONAL. Tier completion invariant audit, persisted by the complete_tier callback. Phase 1 of INGESTION_RELIABILITY_PLAN.md. ``lost > 0`` means objects were submitted but absent from both processed_objects and failed_documents — investigate via the /audit endpoint.","uniqueKey":"tier_tasks.items.[INDEX].audit.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].audit","typeLabel":"TierAuditInfo · object | null"},"audit_override_reason":{"oneOf":[{"type":"string","title":"Audit Override Reason","description":"OPTIONAL. Set when the audit overrode the orchestrator-supplied tier status (e.g. promoted COMPLETED → COMPLETED_WITH_ERRORS because of lost objects).","uniqueKey":"tier_tasks.items.[INDEX].audit_override_reason.oneOf.0","typeLabel":"string"},{"type":"null","title":"Audit Override Reason","description":"OPTIONAL. Set when the audit overrode the orchestrator-supplied tier status (e.g. promoted COMPLETED → COMPLETED_WITH_ERRORS because of lost objects).","uniqueKey":"tier_tasks.items.[INDEX].audit_override_reason.oneOf.1","typeLabel":"null"}],"uniqueKey":"tier_tasks.items.[INDEX].audit_override_reason","typeLabel":"string | null"}},"type":"object","required":["tier_num","source_type"],"title":"TierTaskInfo","description":"Tracking information for a single collection processing tier's task.\n\nIn multi-tier collection processing, each tier represents a processing stage in the\ndecomposition pipeline (e.g., bucket → chunks → frames → scenes). Each tier gets its\nown independent task for granular monitoring, status tracking, and retry capabilities.\n\nUse Cases:\n - Monitor progress of individual tiers in a multi-tier batch\n - Retry failed tiers without reprocessing successful ones\n - Track lineage of processing through parent_task_id linkage\n - Query task status for specific processing stages\n\nTier Definitions:\n - Tier 0: Bucket → Collection (bucket objects as source)\n - Tier N (N > 0): Collection → Collection (upstream collection documents as source)\n\nLifecycle:\n 1. Created with status=PENDING when tier is scheduled\n 2. Updated to status=IN_PROGRESS when processing starts (task_id assigned)\n 3. Finalized to status=COMPLETED or status=FAILED when tier completes\n\nRequirements:\n - tier_num: REQUIRED. Zero-based tier number indicating processing stage\n - task_id: OPTIONAL. None until tier processing starts\n - status: REQUIRED. Defaults to PENDING\n - collection_ids: REQUIRED. Collections being processed in this tier\n - source_type: REQUIRED. \"bucket\" for tier 0, \"collection\" for tier N+\n - source_collection_ids: OPTIONAL. Required for tier N+ (collection sources)\n - parent_task_id: OPTIONAL. Links to previous tier for audit trail\n - started_at: OPTIONAL. Set when tier processing begins\n - completed_at: OPTIONAL. Set when tier processing finishes\n - error: OPTIONAL. Set if tier fails with error details","examples":[{"collection_ids":["col_chunks"],"completed_at":"2025-11-03T10:05:00Z","description":"Tier 0 task (completed): Bucket → Collection","errors":[],"ray_job_id":"raysubmit_ABC123","source_type":"bucket","started_at":"2025-11-03T10:00:00Z","status":"COMPLETED","task_id":"task_tier0_abc123","tier_num":0},{"collection_ids":["col_frames"],"description":"Tier 1 task (in progress): Collection → Collection","errors":[],"parent_task_id":"task_tier0_abc123","ray_job_id":"raysubmit_DEF456","source_collection_ids":["col_chunks"],"source_type":"collection","started_at":"2025-11-03T10:05:00Z","status":"IN_PROGRESS","task_id":"task_tier1_def456","tier_num":1},{"collection_ids":["col_scenes"],"description":"Tier 2 task (pending): Not yet started","errors":[],"source_collection_ids":["col_frames"],"source_type":"collection","status":"PENDING","tier_num":2},{"collection_ids":["col_frames"],"completed_at":"2025-11-03T10:06:30Z","description":"Tier 1 task (failed): Processing errors","error_summary":{"dependency":2},"errors":[{"affected_count":2,"affected_document_ids":["doc_123","doc_456"],"component":"VertexMultimodalService","error_type":"dependency","message":"Missing required package: google-genai","recovery_suggestion":"Install google-genai package: pip install google-genai","stage":"gemini_extraction","timestamp":"2025-11-03T10:06:00Z"}],"parent_task_id":"task_tier0_abc123","ray_job_id":"raysubmit_XYZ789","source_collection_ids":["col_chunks"],"source_type":"collection","started_at":"2025-11-03T10:05:00Z","status":"FAILED","task_id":"task_tier1_xyz789","tier_num":1}],"uniqueKey":"tier_tasks.items.[INDEX]","typeLabel":"TierTaskInfo · object"},"type":"array","title":"Tier Tasks","description":"OPTIONAL. List of tier task tracking information for multi-tier processing. Each element represents one tier in the processing pipeline. Empty array for simple single-tier batches. Populated during batch submission with tier 0 info, then appended as tiers progress. Each TierTaskInfo contains: tier_num, task_id, status, collection_ids, timestamps. Used for granular monitoring: 'Show me status of tier 2' or 'Retry tier 1'. Array index typically matches tier_num (tier_tasks[0] = tier 0, tier_tasks[1] = tier 1, etc.).","examples":[[],[{"collection_ids":["col_chunks"],"status":"COMPLETED","task_id":"task_tier0_abc","tier_num":0}]],"uniqueKey":"tier_tasks","typeLabel":"TierTaskInfo · object[]"},"current_tier":{"oneOf":[{"type":"integer","title":"Current Tier","description":"OPTIONAL. Zero-based index of the currently processing tier. None if batch hasn't started processing (status=DRAFT or PENDING). Updated as batch progresses through tiers. Used to show processing progress: 'Processing tier 2 of 5'. Set to last tier number when batch completes. Example: If processing tier 1 (frames), current_tier=1.","minimum":0,"example":0,"uniqueKey":"current_tier.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Current Tier","description":"OPTIONAL. Zero-based index of the currently processing tier. None if batch hasn't started processing (status=DRAFT or PENDING). Updated as batch progresses through tiers. Used to show processing progress: 'Processing tier 2 of 5'. Set to last tier number when batch completes. Example: If processing tier 1 (frames), current_tier=1.","example":0,"uniqueKey":"current_tier.oneOf.1","typeLabel":"null"}],"uniqueKey":"current_tier","typeLabel":"integer | null"},"total_tiers":{"type":"integer","minimum":1,"title":"Total Tiers","description":"OPTIONAL (defaults to 1). Total number of tiers in the collection DAG. Minimum 1 (tier 0 only = bucket → collection). Set during DAG resolution when batch is submitted. Equals len(dag_tiers) if dag_tiers is populated. Used to calculate progress: current_tier / total_tiers. Example: 5-tier pipeline (bucket → chunks → frames → scenes → summaries) has total_tiers=5.","default":1,"examples":[1,3,5],"uniqueKey":"total_tiers","typeLabel":"integer"},"dag_tiers":{"oneOf":[{"type":"array","items":{"items":{"type":"string","uniqueKey":"dag_tiers.oneOf.0.items.[INDEX].items.[INDEX]","typeLabel":"string"},"type":"array","uniqueKey":"dag_tiers.oneOf.0.items.[INDEX]","typeLabel":"string[]"},"title":"Dag Tiers","description":"OPTIONAL. Complete DAG tier structure for this batch. List of tiers, where each tier is a list of collection IDs to process at that stage. Tier 0 = bucket-sourced collections. Tier N (N > 0) = collection-sourced collections. Collections within same tier have no dependencies (can run in parallel). Collections in tier N+1 depend on collections in tier N. Populated during DAG resolution at batch submission. Used for tier-by-tier processing orchestration. Example: [['col_chunks'], ['col_frames', 'col_objects'], ['col_scenes']] = 3 tiers where frames and objects run in parallel at tier 1.","example":[["col_chunks"]],"uniqueKey":"dag_tiers.oneOf.0","typeLabel":"string[][]"},{"type":"null","title":"Dag Tiers","description":"OPTIONAL. Complete DAG tier structure for this batch. List of tiers, where each tier is a list of collection IDs to process at that stage. Tier 0 = bucket-sourced collections. Tier N (N > 0) = collection-sourced collections. Collections within same tier have no dependencies (can run in parallel). Collections in tier N+1 depend on collections in tier N. Populated during DAG resolution at batch submission. Used for tier-by-tier processing orchestration. Example: [['col_chunks'], ['col_frames', 'col_objects'], ['col_scenes']] = 3 tiers where frames and objects run in parallel at tier 1.","example":[["col_chunks"]],"uniqueKey":"dag_tiers.oneOf.1","typeLabel":"null"}],"uniqueKey":"dag_tiers","typeLabel":"string[][] | null"},"created_at":{"type":"string","format":"date-time","title":"Created At","description":"OPTIONAL (auto-set on creation). ISO 8601 timestamp when batch was created. Set using current_time() from shared.utilities.helpers. Immutable after creation. Used for batch age tracking and cleanup of old batches.","examples":["2025-11-03T10:00:00Z"],"uniqueKey":"created_at","typeLabel":"string\u003cdate-time>"},"progress":{"oneOf":[{"type":"object","title":"BatchProgress","description":"OPTIONAL. Live progress snapshot updated approximately every 10 seconds while the batch is IN_PROGRESS. Written by the Ray ProgressActor inside the engine job. None when status is DRAFT or PENDING (job not started), or after COMPLETED/FAILED. Use this to show real-time progress bars: processed/total objects, percent complete, throughput (items_per_second), and estimated time remaining (eta_seconds).","example":null,"properties":{"processed":{"type":"integer","title":"Processed","description":"Number of objects fully processed so far.","default":0,"uniqueKey":"progress.oneOf.0.processed","typeLabel":"integer"},"total":{"oneOf":[{"type":"integer","title":"Total","description":"Total objects to process. None until the dataset is loaded by the engine.","uniqueKey":"progress.oneOf.0.total.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Total","description":"Total objects to process. None until the dataset is loaded by the engine.","uniqueKey":"progress.oneOf.0.total.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.total","typeLabel":"integer | null"},"percent":{"oneOf":[{"type":"number","title":"Percent","description":"Completion percentage (0–100). None until total is known.","uniqueKey":"progress.oneOf.0.percent.oneOf.0","typeLabel":"number"},{"type":"null","title":"Percent","description":"Completion percentage (0–100). None until total is known.","uniqueKey":"progress.oneOf.0.percent.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.percent","typeLabel":"number | null"},"items_per_second":{"oneOf":[{"type":"number","title":"Items Per Second","description":"Current throughput. Averaged over elapsed time since job start.","uniqueKey":"progress.oneOf.0.items_per_second.oneOf.0","typeLabel":"number"},{"type":"null","title":"Items Per Second","description":"Current throughput. Averaged over elapsed time since job start.","uniqueKey":"progress.oneOf.0.items_per_second.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.items_per_second","typeLabel":"number | null"},"eta_seconds":{"oneOf":[{"type":"number","title":"Eta Seconds","description":"Estimated seconds remaining. None until both total and throughput are known.","uniqueKey":"progress.oneOf.0.eta_seconds.oneOf.0","typeLabel":"number"},{"type":"null","title":"Eta Seconds","description":"Estimated seconds remaining. None until both total and throughput are known.","uniqueKey":"progress.oneOf.0.eta_seconds.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.eta_seconds","typeLabel":"number | null"},"batch_count":{"type":"integer","title":"Batch Count","description":"Number of Ray map_batch micro-batches completed.","default":0,"uniqueKey":"progress.oneOf.0.batch_count","typeLabel":"integer"},"errors":{"type":"integer","title":"Errors","description":"Number of items that errored during processing.","default":0,"uniqueKey":"progress.oneOf.0.errors","typeLabel":"integer"},"first_error":{"oneOf":[{"type":"string","title":"First Error","description":"First item-level error captured during processing (truncated to ~500 chars), e.g. 'ValueError: cannot embed empty text'. Explains WHAT the `errors` counter is counting. None when no item-level error has been captured (or the engine image pre-dates this field). See also batch-level error_summary (error-type counts) and failed_objects (per-object detail).","x-mixpeek-redacted":"driver_text","uniqueKey":"progress.oneOf.0.first_error.oneOf.0","typeLabel":"string"},{"type":"null","title":"First Error","description":"First item-level error captured during processing (truncated to ~500 chars), e.g. 'ValueError: cannot embed empty text'. Explains WHAT the `errors` counter is counting. None when no item-level error has been captured (or the engine image pre-dates this field). See also batch-level error_summary (error-type counts) and failed_objects (per-object detail).","x-mixpeek-redacted":"driver_text","uniqueKey":"progress.oneOf.0.first_error.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.first_error","typeLabel":"string | null"},"extraction_first_error":{"oneOf":[{"type":"string","title":"Extraction First Error","description":"First map/extraction-stage error (rows dropped before the datasink). Distinct from datasink write errors. Typically equals first_error unless a later write-stage error was also captured.","x-mixpeek-redacted":"driver_text","uniqueKey":"progress.oneOf.0.extraction_first_error.oneOf.0","typeLabel":"string"},{"type":"null","title":"Extraction First Error","description":"First map/extraction-stage error (rows dropped before the datasink). Distinct from datasink write errors. Typically equals first_error unless a later write-stage error was also captured.","x-mixpeek-redacted":"driver_text","uniqueKey":"progress.oneOf.0.extraction_first_error.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.extraction_first_error","typeLabel":"string | null"},"documents_skipped":{"type":"integer","title":"Documents Skipped","description":"Number of input rows the processor explicitly marked skipped (metadata-only objects, missing required embedding fields, content-flag filtered, null-text chunks). Skipped rows are not failures and not successes; they count toward the tier-completion invariant via the audit. Phase 1.3 of INGESTION_RELIABILITY_PLAN.md.","default":0,"uniqueKey":"progress.oneOf.0.documents_skipped","typeLabel":"integer"},"objects_extracted":{"oneOf":[{"type":"integer","title":"Objects Extracted","description":"Objects that finished extraction (their rows reached the write stage). The batch headline (`processed`, `percent`, the status message) counts these, never the dispatch counter, so a batch whose objects all fail extraction reads 0 here and 0 percent. Null when the engine has not reported extraction progress yet.","example":0,"uniqueKey":"progress.oneOf.0.objects_extracted.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Objects Extracted","description":"Objects that finished extraction (their rows reached the write stage). The batch headline (`processed`, `percent`, the status message) counts these, never the dispatch counter, so a batch whose objects all fail extraction reads 0 here and 0 percent. Null when the engine has not reported extraction progress yet.","example":0,"uniqueKey":"progress.oneOf.0.objects_extracted.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.objects_extracted","typeLabel":"integer | null"},"objects_skipped":{"oneOf":[{"type":"integer","title":"Objects Skipped","description":"Objects the pipeline deliberately skipped (nothing to embed, already extracted under dedup_strategy skip). Skipped objects are neither extracted nor failed. Null when `objects_extracted` is null.","example":0,"uniqueKey":"progress.oneOf.0.objects_skipped.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Objects Skipped","description":"Objects the pipeline deliberately skipped (nothing to embed, already extracted under dedup_strategy skip). Skipped objects are neither extracted nor failed. Null when `objects_extracted` is null.","example":0,"uniqueKey":"progress.oneOf.0.objects_skipped.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.objects_skipped","typeLabel":"integer | null"},"objects_failed":{"oneOf":[{"type":"integer","title":"Objects Failed","description":"Objects whose extraction failed. `failed_objects` on the batch lists them (bounded) and carries each error. Null when `objects_extracted` is null.","example":0,"uniqueKey":"progress.oneOf.0.objects_failed.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Objects Failed","description":"Objects whose extraction failed. `failed_objects` on the batch lists them (bounded) and carries each error. Null when `objects_extracted` is null.","example":0,"uniqueKey":"progress.oneOf.0.objects_failed.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.objects_failed","typeLabel":"integer | null"},"current_stage":{"oneOf":[{"type":"object","title":"BatchStageInfo","description":"Current processing stage as reported by the Ray engine. Populated once the engine begins work (may lag by ~10s). Useful to distinguish 'loading model' from 'no activity' during early processing.","required":["name","index","total"],"properties":{"name":{"type":"string","title":"Name","description":"Honest active streaming phase: 'dispatch', 'extraction', or 'write'. Equals the one phase in BatchProgress.phases whose status is 'active'.","examples":["dispatch","extraction","write"],"isRequired":true,"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.name","typeLabel":"string"},"index":{"type":"integer","title":"Index","description":"1-based stage index within the current job.","isRequired":true,"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.index","typeLabel":"integer"},"total":{"type":"integer","title":"Total","description":"Total stages in the current job.","isRequired":true,"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.total","typeLabel":"integer"},"stage_elapsed_seconds":{"type":"number","title":"Stage Elapsed Seconds","description":"Seconds spent in this stage so far.","default":0,"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.stage_elapsed_seconds","typeLabel":"number"},"sub_stage":{"oneOf":[{"type":"string","title":"Sub Stage","description":"Sub-stage within 'processing': 'model_loading' while models are initializing, 'inferring' once the first batch completes, 'preparing_write' while the results are handed to the datasink. Helps explain why processed=0.","example":"model_loading","uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.sub_stage.oneOf.0","typeLabel":"string"},{"type":"null","title":"Sub Stage","description":"Sub-stage within 'processing': 'model_loading' while models are initializing, 'inferring' once the first batch completes, 'preparing_write' while the results are handed to the datasink. Helps explain why processed=0.","example":"model_loading","uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.sub_stage.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.sub_stage","typeLabel":"string | null"},"sub_stage_elapsed_seconds":{"oneOf":[{"type":"number","title":"Sub Stage Elapsed Seconds","description":"Seconds spent in the CURRENT sub-stage, which is a different clock from stage_elapsed_seconds: a batch can be 90s into 'processing' and 85s into 'model_loading'. Null when sub_stage is null, and null on an engine image that predates this field; null means not reported, never zero seconds.","example":85,"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.sub_stage_elapsed_seconds.oneOf.0","typeLabel":"number"},{"type":"null","title":"Sub Stage Elapsed Seconds","description":"Seconds spent in the CURRENT sub-stage, which is a different clock from stage_elapsed_seconds: a batch can be 90s into 'processing' and 85s into 'model_loading'. Null when sub_stage is null, and null on an engine image that predates this field; null means not reported, never zero seconds.","example":85,"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.sub_stage_elapsed_seconds.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0.sub_stage_elapsed_seconds","typeLabel":"number | null"}},"uniqueKey":"progress.oneOf.0.current_stage.oneOf.0","typeLabel":"BatchStageInfo · object"},{"type":"null","description":"Current processing stage as reported by the Ray engine. Populated once the engine begins work (may lag by ~10s). Useful to distinguish 'loading model' from 'no activity' during early processing.","uniqueKey":"progress.oneOf.0.current_stage.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.current_stage","typeLabel":"BatchStageInfo · object | null"},"phases":{"oneOf":[{"type":"object","title":"BatchPhases","description":"Honest streaming-phase breakdown: dispatch → extraction → write, each with its own processed/total/status. Exactly one phase is 'active' and its key equals current_stage.name. The SOLE per-stage source of truth — use this rather than current_stage for per-phase counters. None on engine images that pre-date this field.","properties":{"dispatch":{"properties":{"processed":{"type":"integer","title":"Processed","description":"Items completed in this phase.","default":0,"uniqueKey":"progress.oneOf.0.phases.oneOf.0.dispatch.processed","typeLabel":"integer"},"total":{"oneOf":[{"type":"integer","title":"Total","description":"Items expected in this phase; null when indeterminate.","uniqueKey":"progress.oneOf.0.phases.oneOf.0.dispatch.total.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Total","description":"Items expected in this phase; null when indeterminate.","uniqueKey":"progress.oneOf.0.phases.oneOf.0.dispatch.total.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.phases.oneOf.0.dispatch.total","typeLabel":"integer | null"},"status":{"type":"string","enum":["pending","active","done","error"],"title":"Status","description":"Phase status: 'pending', 'active', 'done', or 'error'.","default":"pending","examples":["pending","active","done","error"],"uniqueKey":"progress.oneOf.0.phases.oneOf.0.dispatch.status","typeLabel":"enum\u003cstring>"}},"type":"object","title":"BatchPhaseDetail","description":"Per-phase progress for one streaming phase.\n\nRay Data streams, so extraction and write run concurrently. Each phase\ncarries its own honest counters. ``status`` is one of pending|active|\ndone|error. Exactly one phase is ``active`` at a time, and it matches\n``BatchStageInfo.name``. ``total`` is null when indeterminate (the write\nphase stays null until extraction drains — expanding pipelines emit N\noutput points per input object).","uniqueKey":"progress.oneOf.0.phases.oneOf.0.dispatch","typeLabel":"BatchPhaseDetail · object"},"extraction":{"properties":{"processed":{"type":"integer","title":"Processed","description":"Items completed in this phase.","default":0,"uniqueKey":"progress.oneOf.0.phases.oneOf.0.extraction.processed","typeLabel":"integer"},"total":{"oneOf":[{"type":"integer","title":"Total","description":"Items expected in this phase; null when indeterminate.","uniqueKey":"progress.oneOf.0.phases.oneOf.0.extraction.total.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Total","description":"Items expected in this phase; null when indeterminate.","uniqueKey":"progress.oneOf.0.phases.oneOf.0.extraction.total.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.phases.oneOf.0.extraction.total","typeLabel":"integer | null"},"status":{"type":"string","enum":["pending","active","done","error"],"title":"Status","description":"Phase status: 'pending', 'active', 'done', or 'error'.","default":"pending","examples":["pending","active","done","error"],"uniqueKey":"progress.oneOf.0.phases.oneOf.0.extraction.status","typeLabel":"enum\u003cstring>"}},"type":"object","title":"BatchPhaseDetail","description":"Per-phase progress for one streaming phase.\n\nRay Data streams, so extraction and write run concurrently. Each phase\ncarries its own honest counters. ``status`` is one of pending|active|\ndone|error. Exactly one phase is ``active`` at a time, and it matches\n``BatchStageInfo.name``. ``total`` is null when indeterminate (the write\nphase stays null until extraction drains — expanding pipelines emit N\noutput points per input object).","uniqueKey":"progress.oneOf.0.phases.oneOf.0.extraction","typeLabel":"BatchPhaseDetail · object"},"write":{"properties":{"processed":{"type":"integer","title":"Processed","description":"Items completed in this phase.","default":0,"uniqueKey":"progress.oneOf.0.phases.oneOf.0.write.processed","typeLabel":"integer"},"total":{"oneOf":[{"type":"integer","title":"Total","description":"Items expected in this phase; null when indeterminate.","uniqueKey":"progress.oneOf.0.phases.oneOf.0.write.total.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Total","description":"Items expected in this phase; null when indeterminate.","uniqueKey":"progress.oneOf.0.phases.oneOf.0.write.total.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.phases.oneOf.0.write.total","typeLabel":"integer | null"},"status":{"type":"string","enum":["pending","active","done","error"],"title":"Status","description":"Phase status: 'pending', 'active', 'done', or 'error'.","default":"pending","examples":["pending","active","done","error"],"uniqueKey":"progress.oneOf.0.phases.oneOf.0.write.status","typeLabel":"enum\u003cstring>"}},"type":"object","title":"BatchPhaseDetail","description":"Per-phase progress for one streaming phase.\n\nRay Data streams, so extraction and write run concurrently. Each phase\ncarries its own honest counters. ``status`` is one of pending|active|\ndone|error. Exactly one phase is ``active`` at a time, and it matches\n``BatchStageInfo.name``. ``total`` is null when indeterminate (the write\nphase stays null until extraction drains — expanding pipelines emit N\noutput points per input object).","uniqueKey":"progress.oneOf.0.phases.oneOf.0.write","typeLabel":"BatchPhaseDetail · object"}},"uniqueKey":"progress.oneOf.0.phases.oneOf.0","typeLabel":"BatchPhases · object"},{"type":"null","description":"Honest streaming-phase breakdown: dispatch → extraction → write, each with its own processed/total/status. Exactly one phase is 'active' and its key equals current_stage.name. The SOLE per-stage source of truth — use this rather than current_stage for per-phase counters. None on engine images that pre-date this field.","uniqueKey":"progress.oneOf.0.phases.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.phases","typeLabel":"BatchPhases · object | null"},"active_step":{"oneOf":[{"type":"object","title":"BatchStepInfo","description":"Active pipeline step for multi-step extractors. Shows which processor is currently running (e.g., 'GroundingDINOProcessor 1/3'). None for single-step pipelines.","required":["name","index","total","step_elapsed_seconds"],"properties":{"name":{"type":"string","title":"Name","description":"Processor class name (e.g., 'GroundingDINOProcessor', 'SigLIPProcessor').","isRequired":true,"uniqueKey":"progress.oneOf.0.active_step.oneOf.0.name","typeLabel":"string"},"index":{"type":"integer","title":"Index","description":"1-based step index within the pipeline.","isRequired":true,"uniqueKey":"progress.oneOf.0.active_step.oneOf.0.index","typeLabel":"integer"},"total":{"type":"integer","title":"Total","description":"Total number of steps in the pipeline.","isRequired":true,"uniqueKey":"progress.oneOf.0.active_step.oneOf.0.total","typeLabel":"integer"},"step_elapsed_seconds":{"type":"number","title":"Step Elapsed Seconds","description":"Seconds spent in this step so far.","isRequired":true,"uniqueKey":"progress.oneOf.0.active_step.oneOf.0.step_elapsed_seconds","typeLabel":"number"}},"uniqueKey":"progress.oneOf.0.active_step.oneOf.0","typeLabel":"BatchStepInfo · object"},{"type":"null","description":"Active pipeline step for multi-step extractors. Shows which processor is currently running (e.g., 'GroundingDINOProcessor 1/3'). None for single-step pipelines.","uniqueKey":"progress.oneOf.0.active_step.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.active_step","typeLabel":"BatchStepInfo · object | null"},"overshoot_percent":{"oneOf":[{"type":"number","title":"Overshoot Percent","description":"When processed exceeds total (due to Ray Data retries or pipeline data expansion), this field shows the excess percentage above 100%. For example, 32.0 means 132% of items have been processed. None when processed \u003c= total.","uniqueKey":"progress.oneOf.0.overshoot_percent.oneOf.0","typeLabel":"number"},{"type":"null","title":"Overshoot Percent","description":"When processed exceeds total (due to Ray Data retries or pipeline data expansion), this field shows the excess percentage above 100%. For example, 32.0 means 132% of items have been processed. None when processed \u003c= total.","uniqueKey":"progress.oneOf.0.overshoot_percent.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.overshoot_percent","typeLabel":"number | null"},"queue_position":{"oneOf":[{"type":"integer","title":"Queue Position","description":"1-based position in the Ray job submission queue. Non-null only while the batch is waiting for a concurrency slot. Once a slot is acquired (job submitted to Ray), this becomes null.","uniqueKey":"progress.oneOf.0.queue_position.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Queue Position","description":"1-based position in the Ray job submission queue. Non-null only while the batch is waiting for a concurrency slot. Once a slot is acquired (job submitted to Ray), this becomes null.","uniqueKey":"progress.oneOf.0.queue_position.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.queue_position","typeLabel":"integer | null"},"queue_reason":{"oneOf":[{"type":"string","title":"Queue Reason","description":"Why the batch is waiting, while it is QUEUED: the cluster's extractor capacity (how many of its extractor jobs are running and how many this batch needs) or the tier's concurrent-batch limit. Null once the batch starts.","uniqueKey":"progress.oneOf.0.queue_reason.oneOf.0","typeLabel":"string"},{"type":"null","title":"Queue Reason","description":"Why the batch is waiting, while it is QUEUED: the cluster's extractor capacity (how many of its extractor jobs are running and how many this batch needs) or the tier's concurrent-batch limit. Null once the batch starts.","uniqueKey":"progress.oneOf.0.queue_reason.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.queue_reason","typeLabel":"string | null"},"stage_history":{"oneOf":[{"type":"array","items":{"properties":{"name":{"type":"string","title":"Name","description":"Stage name as the extractor reported it, for example 'loading'.","examples":["model_loading"],"isRequired":true,"uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX].name","typeLabel":"string"},"index":{"type":"integer","title":"Index","description":"1-based position of this stage within the job.","examples":[2],"isRequired":true,"uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX].index","typeLabel":"integer"},"total":{"type":"integer","title":"Total","description":"Total stages the job declared.","examples":[4],"isRequired":true,"uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX].total","typeLabel":"integer"},"started_at":{"type":"number","title":"Started At","description":"Epoch seconds when the stage began.","examples":[1789630000.12],"isRequired":true,"uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX].started_at","typeLabel":"number"},"ended_at":{"type":"number","title":"Ended At","description":"Epoch seconds when the stage ended.","examples":[1789630042.51],"isRequired":true,"uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX].ended_at","typeLabel":"number"},"duration_seconds":{"type":"number","title":"Duration Seconds","description":"Wall-clock seconds the stage took, rounded to 2dp by the writer.","examples":[42.39],"isRequired":true,"uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX].duration_seconds","typeLabel":"number"}},"type":"object","required":["name","index","total","started_at","ended_at","duration_seconds"],"title":"BatchStageTiming","description":"One completed stage of a batch, with the time it took.\n\nThese entries were typed ``Dict[str, Any]``, so codegen emitted\n``{[key: string]: any}[]`` and every client hand-declared the shape to read\na duration the server already measured.\n\nMIRRORED, like ``TierAuditInfo`` above and for the same reason: the entries\nare built by ``ProgressTracker.set_stage`` in\n``engine/monitoring/performance/utils.py``, which this module cannot import.\n``test_stage_history_wire_contract.py`` asserts the writer's keys and this\nmodel's fields stay identical, so a rename on either side fails a test\nrather than silently emptying a field for every consumer.\n\nTimestamps are epoch SECONDS as floats, which is what ``time.time()`` returns\nat the writer. They are not ISO strings and not milliseconds.","uniqueKey":"progress.oneOf.0.stage_history.oneOf.0.items.[INDEX]","typeLabel":"BatchStageTiming · object"},"title":"Stage History","description":"Completed stage timing breakdown, newest last. Populated as stages complete, capped at 50 entries by the writer; the stage still running is in current_stage and is NOT here. Null on the batch LIST path, which strips it for payload size, and present on GET /v1/buckets/{bucket_id}/batches/{batch_id}: null here means not returned on this path, not that no stages ran.","uniqueKey":"progress.oneOf.0.stage_history.oneOf.0","typeLabel":"BatchStageTiming · object[]"},{"type":"null","title":"Stage History","description":"Completed stage timing breakdown, newest last. Populated as stages complete, capped at 50 entries by the writer; the stage still running is in current_stage and is NOT here. Null on the batch LIST path, which strips it for payload size, and present on GET /v1/buckets/{bucket_id}/batches/{batch_id}: null here means not returned on this path, not that no stages ran.","uniqueKey":"progress.oneOf.0.stage_history.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.stage_history","typeLabel":"BatchStageTiming · object[] | null"},"documents_written":{"oneOf":[{"type":"integer","title":"Documents Written","description":"Derived documents written across completed tier/extractor jobs.","uniqueKey":"progress.oneOf.0.documents_written.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Written","description":"Derived documents written across completed tier/extractor jobs.","uniqueKey":"progress.oneOf.0.documents_written.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.documents_written","typeLabel":"integer | null"},"chunk_stats":{"oneOf":[{"type":"object","title":"BatchChunkStats","description":"Realized chunking statistics (unit, total_chunks, mean units per chunk) reported by the text chunker. None for non-chunking extractors or engine images that pre-date this field.","required":["unit"],"properties":{"unit":{"type":"string","title":"Unit","description":"Realized unit the chunker grouped by: characters, words, sentences, paragraphs, pages, seconds (time_segments), or none (no chunking; one chunk per input row).","isRequired":true,"uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.0.unit","typeLabel":"string"},"total_chunks":{"type":"integer","title":"Total Chunks","description":"Total chunks produced across all processed rows.","default":0,"uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.0.total_chunks","typeLabel":"integer"},"mean_units_per_chunk":{"oneOf":[{"type":"number","title":"Mean Units Per Chunk","description":"Mean realized chunk size, measured in `unit`. A value near 1 for a count unit (sentences/paragraphs/pages) with a larger configured chunk_size means the configuration is not taking effect as intended.","uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.0.mean_units_per_chunk.oneOf.0","typeLabel":"number"},{"type":"null","title":"Mean Units Per Chunk","description":"Mean realized chunk size, measured in `unit`. A value near 1 for a count unit (sentences/paragraphs/pages) with a larger configured chunk_size means the configuration is not taking effect as intended.","uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.0.mean_units_per_chunk.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.0.mean_units_per_chunk","typeLabel":"number | null"}},"uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.0","typeLabel":"BatchChunkStats · object"},{"type":"null","description":"Realized chunking statistics (unit, total_chunks, mean units per chunk) reported by the text chunker. None for non-chunking extractors or engine images that pre-date this field.","uniqueKey":"progress.oneOf.0.chunk_stats.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.chunk_stats","typeLabel":"BatchChunkStats · object | null"},"first_record_at":{"oneOf":[{"type":"number","title":"First Record At","description":"Unix time (seconds) at which the tier's pipeline processed its first record. None until a record has been processed.","uniqueKey":"progress.oneOf.0.first_record_at.oneOf.0","typeLabel":"number"},{"type":"null","title":"First Record At","description":"Unix time (seconds) at which the tier's pipeline processed its first record. None until a record has been processed.","uniqueKey":"progress.oneOf.0.first_record_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.first_record_at","typeLabel":"number | null"},"seconds_to_first_record":{"oneOf":[{"type":"number","title":"Seconds To First Record","description":"Seconds from the tier's job start to its first processed record. None until a record has been processed.","uniqueKey":"progress.oneOf.0.seconds_to_first_record.oneOf.0","typeLabel":"number"},{"type":"null","title":"Seconds To First Record","description":"Seconds from the tier's job start to its first processed record. None until a record has been processed.","uniqueKey":"progress.oneOf.0.seconds_to_first_record.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.seconds_to_first_record","typeLabel":"number | null"},"extractors":{"oneOf":[{"type":"object","title":"Extractors","description":"Per-extractor progress for a tier that runs more than one extractor, keyed by '{extractor_name}:{collection_id}'. Lists an extractor that has reported activity but no processed batch at processed 0. None for a single-extractor tier.","additionalProperties":{"properties":{"processed":{"type":"integer","title":"Processed","description":"Objects this extractor has fully processed so far.","default":0,"uniqueKey":"progress.oneOf.0.extractors.oneOf.0.additionalProperties.processed","typeLabel":"integer"},"seconds_since_update":{"oneOf":[{"type":"number","title":"Seconds Since Update","description":"Seconds since this extractor last reported activity, at the time the snapshot was written. A value that keeps growing while processed stays flat names the stuck step.","uniqueKey":"progress.oneOf.0.extractors.oneOf.0.additionalProperties.seconds_since_update.oneOf.0","typeLabel":"number"},{"type":"null","title":"Seconds Since Update","description":"Seconds since this extractor last reported activity, at the time the snapshot was written. A value that keeps growing while processed stays flat names the stuck step.","uniqueKey":"progress.oneOf.0.extractors.oneOf.0.additionalProperties.seconds_since_update.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.extractors.oneOf.0.additionalProperties.seconds_since_update","typeLabel":"number | null"}},"type":"object","title":"BatchExtractorProgress","description":"One extractor's progress within a multi-extractor tier.","uniqueKey":"progress.oneOf.0.extractors.oneOf.0.additionalProperties","typeLabel":"BatchExtractorProgress · object"},"uniqueKey":"progress.oneOf.0.extractors.oneOf.0","typeLabel":"Extractors · object"},{"type":"null","title":"Extractors","description":"Per-extractor progress for a tier that runs more than one extractor, keyed by '{extractor_name}:{collection_id}'. Lists an extractor that has reported activity but no processed batch at processed 0. None for a single-extractor tier.","uniqueKey":"progress.oneOf.0.extractors.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress.oneOf.0.extractors","typeLabel":"Extractors · object | null"},"status_warnings":{"items":{"type":"string","uniqueKey":"progress.oneOf.0.status_warnings.items.[INDEX]","typeLabel":"string"},"type":"array","title":"Status Warnings","description":"Read-time warnings that explain ambiguous terminal status or accounting gaps.","uniqueKey":"progress.oneOf.0.status_warnings","typeLabel":"string[]"}},"uniqueKey":"progress.oneOf.0","typeLabel":"BatchProgress · object"},{"type":"null","description":"OPTIONAL. Live progress snapshot updated approximately every 10 seconds while the batch is IN_PROGRESS. Written by the Ray ProgressActor inside the engine job. None when status is DRAFT or PENDING (job not started), or after COMPLETED/FAILED. Use this to show real-time progress bars: processed/total objects, percent complete, throughput (items_per_second), and estimated time remaining (eta_seconds).","example":null,"uniqueKey":"progress.oneOf.1","typeLabel":"null"}],"uniqueKey":"progress","typeLabel":"BatchProgress · object | null"},"documents_written":{"oneOf":[{"type":"integer","title":"Documents Written","description":"OPTIONAL. Read-time aggregate of documents_written from tier_tasks and extractor_jobs. None means the completion callback has not reported write accounting yet.","example":null,"uniqueKey":"documents_written.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Written","description":"OPTIONAL. Read-time aggregate of documents_written from tier_tasks and extractor_jobs. None means the completion callback has not reported write accounting yet.","example":null,"uniqueKey":"documents_written.oneOf.1","typeLabel":"null"}],"uniqueKey":"documents_written","typeLabel":"integer | null"},"documents_replaced":{"oneOf":[{"type":"integer","title":"Documents Replaced","description":"OPTIONAL. For dedup_strategy replace: documents this batch wrote for the objects it replaced, summed over its tiers. Absent when the batch is not a replace.","uniqueKey":"documents_replaced.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Replaced","description":"OPTIONAL. For dedup_strategy replace: documents this batch wrote for the objects it replaced, summed over its tiers. Absent when the batch is not a replace.","uniqueKey":"documents_replaced.oneOf.1","typeLabel":"null"}],"uniqueKey":"documents_replaced","typeLabel":"integer | null"},"documents_deleted":{"oneOf":[{"type":"integer","title":"Documents Deleted","description":"OPTIONAL. For dedup_strategy replace: the replaced objects' existing documents removed after the new ones were written, summed over its tiers. A failed or cancelled batch removes none. Absent when the batch is not a replace.","uniqueKey":"documents_deleted.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Documents Deleted","description":"OPTIONAL. For dedup_strategy replace: the replaced objects' existing documents removed after the new ones were written, summed over its tiers. A failed or cancelled batch removes none. Absent when the batch is not a replace.","uniqueKey":"documents_deleted.oneOf.1","typeLabel":"null"}],"uniqueKey":"documents_deleted","typeLabel":"integer | null"},"pages_dropped":{"oneOf":[{"type":"integer","title":"Pages Dropped","description":"OPTIONAL. Read-time aggregate of pages NOT indexed across extractor jobs — multi-page inputs (PDFs) capped by max_document_pages or dropped by per-page failures. Only present when > 0: a COMPLETED batch with pages_dropped > 0 indexed its inputs PARTIALLY. See pages_dropped_reasons; raise the collection's max_document_pages to index more pages.","example":null,"uniqueKey":"pages_dropped.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Pages Dropped","description":"OPTIONAL. Read-time aggregate of pages NOT indexed across extractor jobs — multi-page inputs (PDFs) capped by max_document_pages or dropped by per-page failures. Only present when > 0: a COMPLETED batch with pages_dropped > 0 indexed its inputs PARTIALLY. See pages_dropped_reasons; raise the collection's max_document_pages to index more pages.","example":null,"uniqueKey":"pages_dropped.oneOf.1","typeLabel":"null"}],"uniqueKey":"pages_dropped","typeLabel":"integer | null"},"pages_dropped_reasons":{"oneOf":[{"type":"object","title":"Pages Dropped Reasons","description":"OPTIONAL. Read-time aggregate of dropped-page counts by reason: max_document_pages_cap (input exceeded the collection's max_document_pages) or page_processing_failure (per-page extract/embed errors).","example":null,"additionalProperties":{"type":"integer","uniqueKey":"pages_dropped_reasons.oneOf.0.additionalProperties","typeLabel":"integer"},"uniqueKey":"pages_dropped_reasons.oneOf.0","typeLabel":"Pages Dropped Reasons · object"},{"type":"null","title":"Pages Dropped Reasons","description":"OPTIONAL. Read-time aggregate of dropped-page counts by reason: max_document_pages_cap (input exceeded the collection's max_document_pages) or page_processing_failure (per-page extract/embed errors).","example":null,"uniqueKey":"pages_dropped_reasons.oneOf.1","typeLabel":"null"}],"uniqueKey":"pages_dropped_reasons","typeLabel":"Pages Dropped Reasons · object | null"},"segments_dropped":{"oneOf":[{"type":"integer","title":"Segments Dropped","description":"OPTIONAL. Read-time aggregate of video segments NOT indexed across extractor jobs. Only present when > 0: a COMPLETED batch with segments_dropped > 0 indexed its videos PARTIALLY. segments_dropped_reasons says why: for config_cap_max_video_segments, raise max_video_segments to cover more of each video; segment_embedding_failure segments were not written because their embedding failed, which raising max_video_segments does not recover.","example":null,"uniqueKey":"segments_dropped.oneOf.0","typeLabel":"integer"},{"type":"null","title":"Segments Dropped","description":"OPTIONAL. Read-time aggregate of video segments NOT indexed across extractor jobs. Only present when > 0: a COMPLETED batch with segments_dropped > 0 indexed its videos PARTIALLY. segments_dropped_reasons says why: for config_cap_max_video_segments, raise max_video_segments to cover more of each video; segment_embedding_failure segments were not written because their embedding failed, which raising max_video_segments does not recover.","example":null,"uniqueKey":"segments_dropped.oneOf.1","typeLabel":"null"}],"uniqueKey":"segments_dropped","typeLabel":"integer | null"},"segments_dropped_reasons":{"oneOf":[{"type":"object","title":"Segments Dropped Reasons","description":"OPTIONAL. Read-time aggregate of dropped-segment counts by reason: config_cap_max_video_segments (video exceeded the collection's max_video_segments) or segment_embedding_failure (a segment's embedding failed, so the segment was not written).","example":null,"additionalProperties":{"type":"integer","uniqueKey":"segments_dropped_reasons.oneOf.0.additionalProperties","typeLabel":"integer"},"uniqueKey":"segments_dropped_reasons.oneOf.0","typeLabel":"Segments Dropped Reasons · object"},{"type":"null","title":"Segments Dropped Reasons","description":"OPTIONAL. Read-time aggregate of dropped-segment counts by reason: config_cap_max_video_segments (video exceeded the collection's max_video_segments) or segment_embedding_failure (a segment's embedding failed, so the segment was not written).","example":null,"uniqueKey":"segments_dropped_reasons.oneOf.1","typeLabel":"null"}],"uniqueKey":"segments_dropped_reasons","typeLabel":"Segments Dropped Reasons · object | null"},"status_diagnostics":{"additionalProperties":true,"type":"object","title":"Status Diagnostics","description":"Read-time diagnostics explaining terminal status, error indicators, and document-write accounting.","uniqueKey":"status_diagnostics","typeLabel":"Status Diagnostics · object"},"documents_resolvable":{"type":"boolean","title":"Documents Resolvable","description":"ALWAYS PRESENT. True only after the server CONFIRMED this terminal batch's just-written documents are actually queryable via the filtered document get path: a confirmation, never a prediction, timestamp, or elapsed time. A COMPLETED batch with documents_resolvable=false is still finalizing: its documents may 404 on direct GET even though they are durable (Studio renders status==COMPLETED \u0026\u0026 documents_resolvable==false as a 'Finalizing, indexing your results' state and keeps polling). Batches created before the feature shipped emit true (historical, long since resolvable); post-ship batches with no stored value emit false (fail toward not-ready). If the bounded post-completion confirm cannot succeed, documents_resolvable_error is set and this flag stays false.","default":false,"uniqueKey":"documents_resolvable","typeLabel":"boolean"},"documents_resolvable_error":{"oneOf":[{"type":"string","title":"Documents Resolvable Error","description":"Terminal failure signal for the bounded resolvability confirm. Set when the post-completion confirm loop could not confirm this batch's documents were queryable within its bound, the case where documents may NEVER become resolvable. When set, documents_resolvable stays false and will not flip: stop polling and surface a 'taking longer than expected' state instead of a spinner. Null while the confirm is pending or after it succeeded.","x-mixpeek-redacted":"driver_text","uniqueKey":"documents_resolvable_error.oneOf.0","typeLabel":"string"},{"type":"null","title":"Documents Resolvable Error","description":"Terminal failure signal for the bounded resolvability confirm. Set when the post-completion confirm loop could not confirm this batch's documents were queryable within its bound, the case where documents may NEVER become resolvable. When set, documents_resolvable stays false and will not flip: stop polling and surface a 'taking longer than expected' state instead of a spinner. Null while the confirm is pending or after it succeeded.","x-mixpeek-redacted":"driver_text","uniqueKey":"documents_resolvable_error.oneOf.1","typeLabel":"null"}],"uniqueKey":"documents_resolvable_error","typeLabel":"string | null"},"health":{"oneOf":[{"type":"string","title":"Health","description":"OPTIONAL. Computed health status for actively processing batches. Only populated when status is PROCESSING or IN_PROGRESS. Values: 'healthy' (recent activity detected), 'stalled' (no activity for 5+ minutes), 'unknown' (no heartbeat data yet). Computed from tier_tasks[].last_activity_at and updated_at. Use this to detect stuck batches before the internal stall detector kills them.","example":"healthy","uniqueKey":"health.oneOf.0","typeLabel":"string"},{"type":"null","title":"Health","description":"OPTIONAL. Computed health status for actively processing batches. Only populated when status is PROCESSING or IN_PROGRESS. Values: 'healthy' (recent activity detected), 'stalled' (no activity for 5+ minutes), 'unknown' (no heartbeat data yet). Computed from tier_tasks[].last_activity_at and updated_at. Use this to detect stuck batches before the internal stall detector kills them.","example":"healthy","uniqueKey":"health.oneOf.1","typeLabel":"null"}],"uniqueKey":"health","typeLabel":"string | null"},"cost":{"oneOf":[{"type":"object","title":"BatchCost","description":"OPTIONAL. Dollar cost of this batch, derived at read time from the credits recorded against it in usage_records (credits_consumed * $0.001/credit). Lets callers answer 'what did this batch cost?' from the batch response without GCP billing labels. None when cost could not be looked up (e.g. the usage store was unavailable); 0 credits/USD when no usage has been recorded for the batch yet.","properties":{"credits_consumed":{"type":"integer","minimum":0,"title":"Credits Consumed","description":"Total credits recorded against this batch in usage_records (resource_type='batch'). 0 if no usage has been recorded yet (e.g. DRAFT batches, ENTERPRISE tiers that skip credit consumption, or batches submitted before metering existed).","default":0,"examples":[0,100,4820],"uniqueKey":"cost.oneOf.0.credits_consumed","typeLabel":"integer"},"cost_usd":{"type":"number","minimum":0,"title":"Cost Usd","description":"Derived dollar cost = credits_consumed * $0.001/credit (CREDIT_RATE_USD). Same rate used by monthly invoicing, so this matches what the batch contributes to the bill. Reads 0 for org classes the credits system floors or skips entirely (enterprise, internal, failed batches) -- see metered_model_usd for what this batch's work actually cost in that case.","default":0,"examples":[0,0.1,4.82],"uniqueKey":"cost.oneOf.0.cost_usd","typeLabel":"number"},"credit_rate_usd":{"type":"number","title":"Credit Rate Usd","description":"USD per credit used to derive cost_usd ($0.001/credit).","default":0.001,"uniqueKey":"cost.oneOf.0.credit_rate_usd","typeLabel":"number"},"metered_model_usd":{"oneOf":[{"type":"number","title":"Metered Model Usd","description":"The batch's real metered provider cost (e.g. Gemini token spend), from its stored cost_plus block, when the batch has finished and that block exists. This is the figure that still reflects real spend for org classes cost_usd reads 0 for (enterprise, internal, failed batches) -- see shared/billing/cost_plus_floor.py. None, not 0, when no cost_plus block exists yet (not yet computed, or the batch predates the cost-plus floor).","minimum":0,"example":null,"uniqueKey":"cost.oneOf.0.metered_model_usd.oneOf.0","typeLabel":"number"},{"type":"null","title":"Metered Model Usd","description":"The batch's real metered provider cost (e.g. Gemini token spend), from its stored cost_plus block, when the batch has finished and that block exists. This is the figure that still reflects real spend for org classes cost_usd reads 0 for (enterprise, internal, failed batches) -- see shared/billing/cost_plus_floor.py. None, not 0, when no cost_plus block exists yet (not yet computed, or the batch predates the cost-plus floor).","example":null,"uniqueKey":"cost.oneOf.0.metered_model_usd.oneOf.1","typeLabel":"null"}],"uniqueKey":"cost.oneOf.0.metered_model_usd","typeLabel":"number | null"}},"uniqueKey":"cost.oneOf.0","typeLabel":"BatchCost · object"},{"type":"null","description":"OPTIONAL. Dollar cost of this batch, derived at read time from the credits recorded against it in usage_records (credits_consumed * $0.001/credit). Lets callers answer 'what did this batch cost?' from the batch response without GCP billing labels. None when cost could not be looked up (e.g. the usage store was unavailable); 0 credits/USD when no usage has been recorded for the batch yet.","uniqueKey":"cost.oneOf.1","typeLabel":"null"}],"uniqueKey":"cost","typeLabel":"BatchCost · object | null"},"last_activity_at":{"oneOf":[{"type":"string","title":"Last Activity At","description":"OPTIONAL. Timestamp of the most recent activity across all tier tasks. Aggregated from tier_tasks[].last_activity_at — the latest heartbeat from any tier. Updated approximately every 10 seconds by the BatchJobPoller while processing. A stale value (minutes old) while status is PROCESSING indicates the batch may be stalled. None for batches that have not started processing or have no heartbeat data.","format":"date-time","uniqueKey":"last_activity_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Last Activity At","description":"OPTIONAL. Timestamp of the most recent activity across all tier tasks. Aggregated from tier_tasks[].last_activity_at — the latest heartbeat from any tier. Updated approximately every 10 seconds by the BatchJobPoller while processing. A stale value (minutes old) while status is PROCESSING indicates the batch may be stalled. None for batches that have not started processing or have no heartbeat data.","uniqueKey":"last_activity_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"last_activity_at","typeLabel":"string\u003cdate-time> | null"},"retry_count":{"type":"integer","minimum":0,"title":"Retry Count","description":"OPTIONAL (defaults to 0). Number of times this batch has been auto-retried due to transient infrastructure failures (spot node preemption, OOM, actor death). Incremented each time the batch is automatically requeued after a retryable failure. User-facing: lets users see that retries happened transparently.","default":0,"examples":[0,1,2,3],"uniqueKey":"retry_count","typeLabel":"integer"},"max_retries":{"type":"integer","minimum":0,"title":"Max Retries","description":"OPTIONAL (defaults to 3). Maximum number of automatic retries for transient failures. When retry_count reaches max_retries, the batch stays in FAILED state. Only transient/infrastructure failures trigger retries — validation and data errors do not.","default":3,"examples":[3,5],"uniqueKey":"max_retries","typeLabel":"integer"},"last_retry_at":{"oneOf":[{"type":"string","title":"Last Retry At","description":"OPTIONAL. ISO 8601 timestamp of the most recent auto-retry attempt. None if the batch has never been retried. Used to calculate exponential backoff for subsequent retries.","format":"date-time","example":null,"uniqueKey":"last_retry_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Last Retry At","description":"OPTIONAL. ISO 8601 timestamp of the most recent auto-retry attempt. None if the batch has never been retried. Used to calculate exponential backoff for subsequent retries.","example":null,"uniqueKey":"last_retry_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"last_retry_at","typeLabel":"string\u003cdate-time> | null"},"retry_reason":{"oneOf":[{"type":"string","title":"Retry Reason","description":"OPTIONAL. Human-readable reason for the most recent auto-retry. None if the batch has never been retried. Describes the transient failure that triggered the retry (e.g., 'Spot node preempted', 'Ray actor died', 'OOM killed').","x-mixpeek-redacted":"driver_text","example":null,"uniqueKey":"retry_reason.oneOf.0","typeLabel":"string"},{"type":"null","title":"Retry Reason","description":"OPTIONAL. Human-readable reason for the most recent auto-retry. None if the batch has never been retried. Describes the transient failure that triggered the retry (e.g., 'Spot node preempted', 'Ray actor died', 'OOM killed').","x-mixpeek-redacted":"driver_text","example":null,"uniqueKey":"retry_reason.oneOf.1","typeLabel":"null"}],"uniqueKey":"retry_reason","typeLabel":"string | null"},"node_death_resubmit_count":{"type":"integer","minimum":0,"title":"Node Death Resubmit Count","description":"OPTIONAL (defaults to 0). Number of times Mixpeek resubmitted this batch on its own after its job was lost to node loss (a spot preemption, a node that died, or the job reaped for its resources). Each resubmit resumes from the objects already finished, so completed work is not re-extracted. Compare with max_node_death_resubmits to see how many automatic resubmits remain.","default":0,"examples":[0,1,3],"uniqueKey":"node_death_resubmit_count","typeLabel":"integer"},"deploy_resubmit_count":{"type":"integer","minimum":0,"title":"Deploy Resubmit Count","description":"OPTIONAL (defaults to 0). Number of times Mixpeek resubmitted this batch on its own after an engine deploy interrupted its job. Compare with max_deploy_resubmits to see how many automatic resubmits remain.","default":0,"examples":[0,1,2],"uniqueKey":"deploy_resubmit_count","typeLabel":"integer"},"quota_block_first_seen_at":{"oneOf":[{"type":"string","title":"Quota Block First Seen At","description":"OPTIONAL. Set the first time this batch's Ray submission hit a quota-exhaustion or engine-deploy-lock gate (K8s admission 429/503), and cleared once a submission succeeds or the block's requeue window expires. Distinct from retry_count/last_retry_at above, which track the transient-infrastructure-failure retry mechanism (worker died, OOM, spot preemption) rather than this pre-submission gate. Used to bound the requeue loop to a fixed wall-clock window instead of retrying indefinitely.","format":"date-time","example":null,"uniqueKey":"quota_block_first_seen_at.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Quota Block First Seen At","description":"OPTIONAL. Set the first time this batch's Ray submission hit a quota-exhaustion or engine-deploy-lock gate (K8s admission 429/503), and cleared once a submission succeeds or the block's requeue window expires. Distinct from retry_count/last_retry_at above, which track the transient-infrastructure-failure retry mechanism (worker died, OOM, spot preemption) rather than this pre-submission gate. Used to bound the requeue loop to a fixed wall-clock window instead of retrying indefinitely.","example":null,"uniqueKey":"quota_block_first_seen_at.oneOf.1","typeLabel":"null"}],"uniqueKey":"quota_block_first_seen_at","typeLabel":"string\u003cdate-time> | null"},"webhook_url":{"oneOf":[{"type":"string","title":"Webhook Url","description":"OPTIONAL. URL to receive an HTTP POST notification when the batch reaches a terminal state (COMPLETED, FAILED, or CANCELED). Set at submit time via SubmitBatchRequest. The webhook is fire-and-forget: delivery failures are logged but never affect batch processing.","example":"https://example.com/webhooks/batch-complete","uniqueKey":"webhook_url.oneOf.0","typeLabel":"string"},{"type":"null","title":"Webhook Url","description":"OPTIONAL. URL to receive an HTTP POST notification when the batch reaches a terminal state (COMPLETED, FAILED, or CANCELED). Set at submit time via SubmitBatchRequest. The webhook is fire-and-forget: delivery failures are logged but never affect batch processing.","example":"https://example.com/webhooks/batch-complete","uniqueKey":"webhook_url.oneOf.1","typeLabel":"null"}],"uniqueKey":"webhook_url","typeLabel":"string | null"},"updated_at":{"type":"string","format":"date-time","title":"Updated At","description":"OPTIONAL (auto-updated). ISO 8601 timestamp when batch was last modified. Updated using current_time() whenever batch status or tier_tasks change. Used to track batch activity and identify stale batches.","examples":["2025-11-03T10:30:00Z"],"uniqueKey":"updated_at","typeLabel":"string\u003cdate-time>"},"status_message":{"oneOf":[{"type":"string","title":"Status Message","description":"COMPUTED. Human-readable description of the current batch state. Examples: 'Processing 724/50,000 objects (1.4%)', 'Queued — 2 batches ahead', 'Completed in 5m 23s', 'Loading model (stage 1/3)'. Computed on read, not stored in the database.","uniqueKey":"status_message.oneOf.0","typeLabel":"string"},{"type":"null","title":"Status Message","description":"COMPUTED. Human-readable description of the current batch state. Examples: 'Processing 724/50,000 objects (1.4%)', 'Queued — 2 batches ahead', 'Completed in 5m 23s', 'Loading model (stage 1/3)'. Computed on read, not stored in the database.","uniqueKey":"status_message.oneOf.1","typeLabel":"null"}],"uniqueKey":"status_message","typeLabel":"string | null"},"estimated_completion":{"oneOf":[{"type":"string","title":"Estimated Completion","description":"COMPUTED. Estimated completion timestamp based on current throughput. Derived from progress.eta_seconds + now. None if throughput data is unavailable. Computed on read, not stored in the database.","format":"date-time","uniqueKey":"estimated_completion.oneOf.0","typeLabel":"string\u003cdate-time>"},{"type":"null","title":"Estimated Completion","description":"COMPUTED. Estimated completion timestamp based on current throughput. Derived from progress.eta_seconds + now. None if throughput data is unavailable. Computed on read, not stored in the database.","uniqueKey":"estimated_completion.oneOf.1","typeLabel":"null"}],"uniqueKey":"estimated_completion","typeLabel":"string\u003cdate-time> | null"}},"type":"object","required":["bucket_id","max_node_death_resubmits","max_deploy_resubmits","lifecycle","lifecycle_reason"],"title":"BatchModel","description":"Model representing a batch of objects for processing through collections.\n\nA batch groups bucket objects together for processing through one or more collections.\nBatches support multi-tier processing where collections are processed in dependency order\n(e.g., bucket → chunks → frames → scenes). Each tier has independent task tracking.\n\nUse Cases:\n - Process multiple objects through collections in a single batch\n - Track progress of multi-tier decomposition pipelines\n - Monitor and retry individual processing tiers\n - Query batch status and tier-specific task information\n\nLifecycle:\n 1. Created in DRAFT status with object_ids\n 2. Submitted for processing → status changes to PENDING\n 3. Each tier processes sequentially (tier 0 → tier 1 → ... → tier N)\n 4. Batch completes when all tiers finish (status=COMPLETED) or any tier fails (status=FAILED)\n\nMulti-Tier Processing:\n - Tier 0: Bucket objects → Collections (bucket as source)\n - Tier N (N > 0): Collection documents → Collections (upstream collection as source)\n - Each tier gets independent task tracking via tier_tasks array\n - Processing proceeds tier-by-tier with automatic chaining\n\nRequirements:\n - batch_id: OPTIONAL (auto-generated if not provided)\n - bucket_id: REQUIRED\n - status: OPTIONAL (defaults to DRAFT)\n - object_ids: REQUIRED for processing (must have at least 1 object)\n - collection_ids: OPTIONAL (discovered via DAG resolution)\n - tier_tasks: OPTIONAL (populated during processing)\n - current_tier: OPTIONAL (set during processing)\n - total_tiers: OPTIONAL (defaults to 1, set during DAG resolution)\n - dag_tiers: OPTIONAL (populated during DAG resolution)","examples":[{"batch_id":"btch_simple_001","bucket_id":"bkt_videos","collection_ids":["col_chunks"],"dag_tiers":[["col_chunks"]],"description":"Simple single-tier batch (DRAFT)","metadata":{"campaign_id":"Q4_2025"},"object_ids":["obj_video_001","obj_video_002"],"status":"DRAFT","tier_tasks":[],"total_tiers":1,"type":"BUCKET"},{"batch_id":"btch_multitier_002","bucket_id":"bkt_videos","collection_ids":["col_chunks","col_frames","col_scenes"],"current_tier":1,"dag_tiers":[["col_chunks"],["col_frames"],["col_scenes"]],"description":"Multi-tier batch (IN_PROGRESS): Currently processing tier 1","manifest_key":"ns_abc/org_123/manifests/tier_1.parquet","metadata":{"user_email":"user@example.com"},"object_ids":["obj_video_001"],"status":"IN_PROGRESS","task_id":"task_tier0_abc123","tier_tasks":[{"collection_ids":["col_chunks"],"completed_at":"2025-11-03T10:05:00Z","source_type":"bucket","started_at":"2025-11-03T10:00:00Z","status":"COMPLETED","task_id":"task_tier0_abc123","tier_num":0},{"collection_ids":["col_frames"],"parent_task_id":"task_tier0_abc123","source_collection_ids":["col_chunks"],"source_type":"collection","started_at":"2025-11-03T10:05:00Z","status":"IN_PROGRESS","task_id":"task_tier1_def456","tier_num":1},{"collection_ids":["col_scenes"],"source_collection_ids":["col_frames"],"source_type":"collection","status":"PENDING","tier_num":2}],"total_tiers":3,"type":"BUCKET"},{"batch_id":"btch_complete_003","bucket_id":"bkt_videos","collection_ids":["col_chunks","col_frames","col_scenes"],"current_tier":2,"dag_tiers":[["col_chunks"],["col_frames"],["col_scenes"]],"description":"Multi-tier batch (COMPLETED): All 3 tiers finished","object_ids":["obj_video_001"],"status":"COMPLETED","tier_tasks":[{"collection_ids":["col_chunks"],"completed_at":"2025-11-03T10:05:00Z","status":"COMPLETED","task_id":"task_tier0_abc","tier_num":0},{"collection_ids":["col_frames"],"completed_at":"2025-11-03T10:10:00Z","status":"COMPLETED","task_id":"task_tier1_def","tier_num":1},{"collection_ids":["col_scenes"],"completed_at":"2025-11-03T10:15:00Z","status":"COMPLETED","task_id":"task_tier2_ghi","tier_num":2}],"total_tiers":3,"type":"BUCKET"},{"batch_id":"btch_failed_004","bucket_id":"bkt_videos","collection_ids":["col_chunks","col_frames"],"current_tier":1,"dag_tiers":[["col_chunks"],["col_frames"]],"description":"Multi-tier batch (FAILED): Tier 1 failed with dependency error","error":"Tier 1 failed: Missing required package: google-genai","error_summary":{"authentication":2,"dependency":5},"object_ids":["obj_video_001"],"status":"FAILED","tier_tasks":[{"collection_ids":["col_chunks"],"completed_at":"2025-11-03T10:05:00Z","errors":[],"status":"COMPLETED","task_id":"task_tier0_abc","tier_num":0},{"collection_ids":["col_frames"],"completed_at":"2025-11-03T10:06:30Z","error_summary":{"authentication":2,"dependency":5},"errors":[{"affected_count":5,"component":"VertexMultimodalService","error_type":"dependency","message":"Missing required package: google-genai","recovery_suggestion":"Install google-genai package","stage":"gemini_extraction"},{"affected_count":2,"component":"VertexMultimodalService","error_type":"authentication","message":"Invalid Vertex AI credentials"}],"status":"FAILED","task_id":"task_tier1_def","tier_num":1}],"total_tiers":2,"type":"BUCKET"}],"uniqueKey":"","typeLabel":"BatchModel · object"},"examples":{"Example":{"title":"Example","value":{"batch_id":"btch_simple_001","bucket_id":"bkt_videos","collection_ids":["col_chunks"],"dag_tiers":[["col_chunks"]],"description":"Simple single-tier batch (DRAFT)","metadata":{"campaign_id":"Q4_2025"},"object_ids":["obj_video_001","obj_video_002"],"status":"DRAFT","tier_tasks":[],"total_tiers":1,"type":"BUCKET"}}},"exampleType":"generated"}},"exampleType":"generated"},"77626a0a-930d-4d53-9c2c-f349dc2b08f0":{"description":"Bad Request","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"94a84887-1a34-44f9-b7be-df47da91d6a2":{"description":"Unauthorized","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"0ceb9ddc-4a80-4cb8-a366-366b1f0116a1":{"description":"Forbidden","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"d0a731f3-948c-42fd-9abf-e12600566965":{"description":"Not Found","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"6a62aeb5-4cc7-408b-b2d9-e3191b75113f":{"description":"Unprocessable Entity","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"ef7ab2f0-842c-49aa-b34f-1162ba929db3":{"description":"Too Many Requests","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"d51fa5be-7e9d-4afe-9569-b200abfcb0a0":{"description":"Internal Server Error","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"},"b6bec9b4-6471-4d9d-8b75-0014e936673d":{"description":"Service Unavailable","content":{"application/json":{"schema":{"properties":{"status":{"type":"integer","title":"Status","description":"HTTP status code for this error","isRequired":true,"uniqueKey":"status","typeLabel":"integer"},"error":{"properties":{"message":{"type":"string","title":"Message","description":"Human-readable error message","isRequired":true,"uniqueKey":"error.message","typeLabel":"string"},"type":{"type":"string","title":"Type","description":"Stable error type identifier (machine-readable)","isRequired":true,"uniqueKey":"error.type","typeLabel":"string"},"code":{"oneOf":[{"type":"string","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.0","typeLabel":"string"},{"type":"null","title":"Code","description":"Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition.","uniqueKey":"error.code.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.code","typeLabel":"string | null"},"details":{"oneOf":[{"type":"object","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","additionalProperties":true,"uniqueKey":"error.details.oneOf.0","typeLabel":"Details · object"},{"type":"null","title":"Details","description":"Optional structured details to help debugging (validation errors, IDs, etc.)","uniqueKey":"error.details.oneOf.1","typeLabel":"null"}],"uniqueKey":"error.details","typeLabel":"Details · object | null"}},"type":"object","required":["message","type"],"title":"ErrorDetail","description":"Error details payload","isRequired":true,"uniqueKey":"error","typeLabel":"ErrorDetail · object"},"success":{"type":"boolean","title":"Success","description":"Always false for error responses","default":false,"uniqueKey":"success","typeLabel":"boolean"}},"type":"object","required":["status","error"],"title":"ErrorResponse","description":"Error response model.","examples":[{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}],"uniqueKey":"","typeLabel":"ErrorResponse · object"},"examples":{"Example":{"title":"Example","value":{"error":{"details":{"id":"ns_123","resource":"namespace"},"message":"Namespace not found","type":"NotFoundError"},"status":404,"success":false}}},"exampleType":"generated"}},"exampleType":"generated"}},"security":{"ef19c76f-e833-4f86-97f8-9ef3a310e317":{"BearerAuth":{"type":"http","scheme":"bearer","description":"Mixpeek API key, sent as `Authorization: Bearer mxp_sk_...`. Create one in Studio under Settings → API Keys, or with an admin key via `POST /v1/organizations/users/{user_email}/api-keys`. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403.","scopes":[],"schemeName":"BearerAuth"},"NamespaceHeader":{"type":"apiKey","in":"header","name":"X-Namespace","description":"Namespace id (`ns_...`), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked `x-mixpeek-namespace-scoped`.","scopes":[],"schemeName":"NamespaceHeader"}}},"servers":{"a3ca4d02-9041-4306-8eae-afdf7d554be4":{"url":"https://api.mixpeek.com","description":"Production"}},"processedSecurityOptions":[{"schemes":[{"scheme":{"type":"http","scheme":"bearer","description":"Mixpeek API key, sent as `Authorization: Bearer mxp_sk_...`. Create one in Studio under Settings → API Keys, or with an admin key via `POST /v1/organizations/users/{user_email}/api-keys`. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403.","scopes":[],"schemeName":"BearerAuth"},"schema":{"title":"Bearer Auth","type":"string","isRequired":true,"description":"Mixpeek API key, sent as `Authorization: Bearer mxp_sk_...`. Create one in Studio under Settings → API Keys, or with an admin key via `POST /v1/organizations/users/{user_email}/api-keys`. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403.","format":"bearer","typeLabel":"string\u003cbearer>","deprecated":false},"schemeName":"BearerAuth","name":"Authorization"},{"scheme":{"type":"apiKey","in":"header","name":"X-Namespace","description":"Namespace id (`ns_...`), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked `x-mixpeek-namespace-scoped`.","scopes":[],"schemeName":"NamespaceHeader"},"schema":{"title":"X-Namespace","type":"string","description":"Namespace id (`ns_...`), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked `x-mixpeek-namespace-scoped`.","isRequired":true,"typeLabel":"string","deprecated":false},"schemeName":"NamespaceHeader","name":"X-Namespace"}],"title":"BearerAuth \u0026 NamespaceHeader"}]},"originalFileLocation":"api-reference/openapi.json"},"subdomain":"mixpeek","siteName":"Mixpeek","title":"Partially Update Batch - Mixpeek","navMaps":{"firstHrefInVersion":{"v1":{"href":"/api-reference/bucket-batches/partially-update-batch","matchLevel":0}},"firstHrefInLanguage":{}},"lastModified":"2026-01-23T23:19:11.092Z","originalFileLocation":"api-reference/bucket-batches/partially-update-batch.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"This endpoint partially updates a batch (PATCH operation). Only provided fields will be updated. At minimum, metadata can always be updated. Immutable fields like batch_id and timestamps cannot be modified.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Mixpeek\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Partially Update Batch - Mixpeek\">\u003cmeta data-mint-head property=\"og:description\" content=\"This endpoint partially updates a batch (PATCH operation). Only provided fields will be updated. At minimum, metadata can always be updated. Immutable fields like batch_id and timestamps cannot be modified.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Mixpeek\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://mixpeek.mintlify.site/docs/_mintlify/api/og/v2?division=Batches\u0026amp;title=Partially+Update+Batch\u0026amp;description=This+endpoint+partially+updates+a+batch+%28PATCH+operation%29.%0A++++Only+provided+fields+will+be+updated.+At+minimum%2C+metadata+can+always+be+updated.%0A++++Immutable+f\u0026amp;theme=c63607064cbc270563da1421\u0026amp;signature=7aea3f99e4c36b3a0bb9ba18677f8d2675d98c4e7ab6ef09b3d3b26a623dc09f\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Partially Update Batch - Mixpeek\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"This endpoint partially updates a batch (PATCH operation). Only provided fields will be updated. At minimum, metadata can always be updated. Immutable fields like batch_id and timestamps cannot be modified.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://mixpeek.mintlify.site/docs/_mintlify/api/og/v2?division=Batches\u0026amp;title=Partially+Update+Batch\u0026amp;description=This+endpoint+partially+updates+a+batch+%28PATCH+operation%29.%0A++++Only+provided+fields+will+be+updated.+At+minimum%2C+metadata+can+always+be+updated.%0A++++Immutable+f\u0026amp;theme=c63607064cbc270563da1421\u0026amp;signature=7aea3f99e4c36b3a0bb9ba18677f8d2675d98c4e7ab6ef09b3d3b26a623dc09f\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/docs/mintlify-assets/_mintlify/favicons/mixpeek/FL3jtq6uKM5AAW-Z/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Mixpeek\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#FC5185\">\u003clink data-mint-head rel=\"canonical\" href=\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/docs/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/api-reference/bucket-batches/partially-update-batch.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://docs.mixpeek.com/#organization\",\"name\":\"Mixpeek\",\"url\":\"https://docs.mixpeek.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/mixpeek/QQyAGObvvz8PjpE0/logo/light.png?fit=max\u0026auto=format\u0026n=QQyAGObvvz8PjpE0\u0026q=85\u0026s=ac5ab550180057404fe833886513775f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://docs.mixpeek.com/docs#website\",\"name\":\"Mixpeek\",\"url\":\"https://docs.mixpeek.com/docs\",\"publisher\":{\"@id\":\"https://docs.mixpeek.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch#webpage\",\"url\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch\",\"name\":\"Partially Update Batch\",\"description\":\"This endpoint partially updates a batch (PATCH operation). Only provided fields will be updated. At minimum, metadata can always be updated. Immutable fields like batch_id and timestamps cannot be modified.\",\"dateModified\":\"2026-01-23T23:19:11.092Z\",\"isPartOf\":{\"@id\":\"https://docs.mixpeek.com/docs#website\"},\"breadcrumb\":{\"@id\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Buckets\",\"item\":\"https://docs.mixpeek.com/docs/api-reference/buckets/create-bucket\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Batches\",\"item\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/create-batch\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Partially Update Batch\",\"item\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch\"}]},{\"@type\":[\"TechArticle\",\"APIReference\"],\"@id\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch#apireference\",\"headline\":\"Partially Update Batch\",\"name\":\"Partially Update Batch\",\"description\":\"This endpoint partially updates a batch (PATCH operation). Only provided fields will be updated. At minimum, metadata can always be updated. Immutable fields like batch_id and timestamps cannot be modified.\",\"url\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch\",\"mainEntityOfPage\":{\"@id\":\"https://docs.mixpeek.com/docs/api-reference/bucket-batches/partially-update-batch#webpage\"},\"dateModified\":\"2026-01-23T23:19:11.092Z\",\"publisher\":{\"@id\":\"https://docs.mixpeek.com/#organization\"},\"isPartOf\":{\"@id\":\"https://docs.mixpeek.com/docs#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"Mixpeek\",\"creator\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","siteName":"Mixpeek","nav":{"global":{"anchors":[{"anchor":"Homepage","icon":"house","href":"https://mixpeek.com"},{"anchor":"Engineering Blog","icon":"newspaper","href":"https://mixpeek.com/blog"},{"anchor":"API Swagger","icon":"file-code","href":"https://api.mixpeek.com/docs"}]},"versions":[{"version":"v1","default":true,"tabs":[{"tab":"Documentation","pages":[{"title":"Documentation","href":"/overview/introduction"}]},{"tab":"Vector Store","pages":[{"title":"Vector Store","href":"/vector-store/overview"}]},{"tab":"Guides","pages":[{"title":"Guides","href":"/tutorials/semantic-search"}]},{"tab":"API Reference","groups":[{"group":"Namespaces","pages":[{"openapi":"post /v1/namespaces","title":"Create Namespace","href":"/api-reference/namespaces/create-namespace"},{"openapi":"get /v1/namespaces/{namespace_identifier}","title":"Get Namespace","href":"/api-reference/namespaces/get-namespace"},{"openapi":"put /v1/namespaces/{namespace_identifier}","title":"Update Namespace","href":"/api-reference/namespaces/update-namespace"},{"openapi":"patch /v1/namespaces/{namespace_identifier}","title":"Partially Update Namespace","href":"/api-reference/namespaces/partially-update-namespace"},{"openapi":"post /v1/namespaces/list","title":"List Namespaces","href":"/api-reference/namespaces/list-namespaces"},{"openapi":"get /v1/namespaces/{namespace_identifier}/stats","title":"Get Namespace Stats","href":"/api-reference/namespaces/get-namespace-stats"},{"openapi":"delete /v1/namespaces/{namespace_identifier}","title":"Delete Namespace","href":"/api-reference/namespaces/delete-namespace"},{"group":"Clone","pages":[{"openapi":"post /v1/namespaces/{namespace_identifier}/clone","title":"Clone Namespace","href":"/api-reference/namespace-clone/clone-namespace"}]},{"group":"Migrations","pages":[{"openapi":"post /v1/namespaces/migrations/","title":"Create Migration","href":"/api-reference/namespace-migrations/create-migration"},{"openapi":"get /v1/namespaces/migrations/{migration_id}","title":"Get Migration","href":"/api-reference/namespace-migrations/get-migration"},{"openapi":"post /v1/namespaces/migrations/list","title":"List Migrations","href":"/api-reference/namespace-migrations/list-migrations"},{"openapi":"post /v1/namespaces/migrations/{migration_id}/start","title":"Start Migration","href":"/api-reference/namespace-migrations/start-migration"},{"openapi":"post /v1/namespaces/migrations/validate","title":"Validate Migration","href":"/api-reference/namespace-migrations/validate-migration"},{"openapi":"post /v1/namespaces/migrations/{migration_id}/cancel","title":"Cancel Migration","href":"/api-reference/namespace-migrations/cancel-migration"},{"openapi":"delete /v1/namespaces/migrations/{migration_id}","title":"Delete Migration","href":"/api-reference/namespace-migrations/delete-migration"}]},{"group":"Extractors","pages":[{"openapi":"get /v1/namespaces/{namespace_id}/extractors","title":"List all extractors available to namespace","href":"/api-reference/namespace-extractors/list-all-extractors-available-to-namespace"},{"openapi":"get /v1/namespaces/{namespace_id}/extractors/{extractor_id}","title":"Get extractor details","href":"/api-reference/namespace-extractors/get-extractor-details"}]},{"group":"Models","pages":[{"openapi":"get /v1/namespaces/{namespace_id}/models","title":"List models in namespace","href":"/api-reference/custom-models/list-models-in-namespace"},{"openapi":"post /v1/namespaces/{namespace_id}/models","title":"Upload a custom model","href":"/api-reference/custom-models/upload-a-custom-model"},{"openapi":"get /v1/namespaces/{namespace_id}/models/{model_id}","title":"Get model details","href":"/api-reference/custom-models/get-model-details"},{"openapi":"post /v1/namespaces/{namespace_id}/models/{model_id}/deploy","title":"Deploy model to Ray object store","href":"/api-reference/custom-models/deploy-model-to-ray-object-store"},{"openapi":"post /v1/namespaces/{namespace_id}/models/org/{model_id}/enable","title":"Enable org model for namespace","href":"/api-reference/custom-models/enable-org-model-for-namespace"},{"openapi":"post /v1/namespaces/{namespace_id}/models/org/{model_id}/disable","title":"Disable org model for namespace","href":"/api-reference/custom-models/disable-org-model-for-namespace"},{"openapi":"get /v1/namespaces/{namespace_id}/models/available","title":"List available org models","href":"/api-reference/custom-models/list-available-org-models"},{"openapi":"delete /v1/namespaces/{namespace_id}/models/{model_id}","title":"Delete a model","href":"/api-reference/custom-models/delete-a-model"}]},{"group":"Snapshots","pages":[{"openapi":"post /v1/namespaces/{namespace_identifier}/snapshots","title":"Create Namespace Snapshot","href":"/api-reference/namespace-snapshots/create-namespace-snapshot"},{"openapi":"post /v1/namespaces/{namespace_identifier}/snapshots/list","title":"List Namespace Snapshots","href":"/api-reference/namespace-snapshots/list-namespace-snapshots"},{"openapi":"get /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}","title":"Get Snapshot","href":"/api-reference/namespace-snapshots/get-snapshot"},{"openapi":"delete /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}","title":"Delete Snapshot","href":"/api-reference/namespace-snapshots/delete-snapshot"},{"openapi":"get /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}/verify","title":"Verify Snapshot Restore-Readiness","href":"/api-reference/namespace-snapshots/verify-snapshot-restore-readiness"},{"openapi":"post /v1/namespaces/{namespace_identifier}/snapshots/{snapshot_id}/restore","title":"Restore From Snapshot","href":"/api-reference/namespace-snapshots/restore-from-snapshot"}]},{"openapi":"get /v1/namespaces","title":"List Namespaces","href":"/api-reference/namespaces/list-namespaces-get"},{"openapi":"post /v1/namespaces/stats/batch","title":"Get Namespace Stats (Batch)","href":"/api-reference/namespaces/get-namespace-stats-batch"},{"openapi":"post /v1/namespaces/{namespace_identifier}/ensure-indexes","title":"Ensure Namespace Payload Indexes","href":"/api-reference/namespaces/ensure-namespace-payload-indexes"},{"group":"BYO vectors","pages":[{"openapi":"post /v1/namespaces/{namespace_id}/documents/upsert","title":"Upsert documents with BYO vectors","href":"/api-reference/byo-documents/upsert-documents-byo-vectors"},{"openapi":"get /v1/namespaces/{namespace_id}/documents/{document_id}","title":"Get a document by ID (path-scoped namespace)","href":"/api-reference/byo-documents/get-document-id"},{"openapi":"post /v1/namespaces/standalone","title":"Create standalone namespace (BYO vectors)","href":"/api-reference/byo-documents/create-standalone-namespace"},{"openapi":"post /v1/namespaces/{namespace_id}/promote","title":"Promote standalone namespace to managed mode","href":"/api-reference/byo-documents/promote-standalone-namespace-managed-mode"},{"openapi":"post /v1/namespaces/{namespace_id}/promote/validate","title":"Dry-run validate a standalone → managed promotion","href":"/api-reference/byo-documents/dry-run-validate-standalone-managed-promotion"}]}]},{"group":"Buckets","pages":[{"openapi":"post /v1/buckets","title":"Create Bucket","href":"/api-reference/buckets/create-bucket"},{"openapi":"get /v1/buckets/{bucket_identifier}","title":"Get Bucket","href":"/api-reference/buckets/get-bucket"},{"openapi":"put /v1/buckets/{bucket_identifier}","title":"Update Bucket","href":"/api-reference/buckets/update-bucket"},{"openapi":"patch /v1/buckets/{bucket_identifier}","title":"Partially Update Bucket","href":"/api-reference/buckets/partially-update-bucket"},{"openapi":"post /v1/buckets/list","title":"List Buckets","href":"/api-reference/buckets/list-buckets"},{"openapi":"delete /v1/buckets/{bucket_identifier}","title":"Delete Bucket","href":"/api-reference/buckets/delete-bucket"},{"group":"Syncs","pages":[{"openapi":"post /v1/buckets/{bucket_id}/syncs/list","title":"List Sync Configurations","href":"/api-reference/bucket-syncs/list-sync-configurations"},{"openapi":"post /v1/buckets/{bucket_id}/syncs","title":"Create Sync Configuration","href":"/api-reference/bucket-syncs/create-sync-configuration"},{"openapi":"get /v1/buckets/{bucket_id}/syncs/{sync_config_id}","title":"Get Sync Configuration","href":"/api-reference/bucket-syncs/get-sync-configuration"},{"openapi":"patch /v1/buckets/{bucket_id}/syncs/{sync_config_id}","title":"Update Sync Configuration","href":"/api-reference/bucket-syncs/update-sync-configuration"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/pause","title":"Pause Sync Configuration","href":"/api-reference/bucket-syncs/pause-sync-configuration"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/resume","title":"Resume Sync Configuration","href":"/api-reference/bucket-syncs/resume-sync-configuration"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/trigger","title":"Trigger Sync Configuration","href":"/api-reference/bucket-syncs/trigger-sync-configuration"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/unlock","title":"Force Unlock Sync Configuration","href":"/api-reference/bucket-syncs/force-unlock-sync-configuration"},{"openapi":"delete /v1/buckets/{bucket_id}/syncs/{sync_config_id}","title":"Delete Sync Configuration","href":"/api-reference/bucket-syncs/delete-sync-configuration"},{"openapi":"get /v1/buckets/{bucket_id}/syncs/{sync_config_id}/metrics","title":"Get Sync Metrics","href":"/api-reference/bucket-syncs/get-sync-metrics"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/jobs","title":"List Sync Jobs","href":"/api-reference/bucket-syncs/list-sync-jobs"},{"openapi":"get /v1/buckets/{bucket_id}/syncs/{sync_config_id}/jobs/{sync_job_id}","title":"Get Sync Job","href":"/api-reference/bucket-syncs/get-sync-job"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/dlq","title":"List Dlq Entries","href":"/api-reference/bucket-syncs/list-dlq-entries"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/dlq/requeue","title":"Requeue Dlq Entries","href":"/api-reference/bucket-syncs/requeue-dlq-entries"},{"openapi":"post /v1/buckets/{bucket_id}/syncs/{sync_config_id}/dlq/bulk-mark-failed","title":"Bulk Mark Dlq Permanently Failed","href":"/api-reference/bucket-syncs/bulk-mark-dlq-permanently-failed"}]},{"group":"Source adapter","pages":[{"openapi":"get /v1/buckets/{bucket_id}/source-adapter/status","title":"Get Adapter Status","href":"/api-reference/bucket-source-adapter/get-source-adapter-status"},{"openapi":"post /v1/buckets/{bucket_id}/source-adapter/refresh","title":"Refresh Source Adapter","href":"/api-reference/bucket-source-adapter/refresh-source-adapter"}]},{"group":"Objects","pages":[{"openapi":"post /v1/buckets/{bucket_identifier}/objects","title":"Create Object","href":"/api-reference/bucket-objects/create-object"},{"openapi":"post /v1/buckets/{bucket_identifier}/objects/batch","title":"Create Objects in Batch","href":"/api-reference/bucket-objects/create-objects-in-batch"},{"openapi":"get /v1/buckets/{bucket_identifier}/objects/{object_identifier}","title":"Get Object","href":"/api-reference/bucket-objects/get-object"},{"openapi":"put /v1/buckets/{bucket_identifier}/objects/{object_identifier}","title":"Update Object","href":"/api-reference/bucket-objects/update-object"},{"openapi":"patch /v1/buckets/{bucket_identifier}/objects/{object_identifier}","title":"Partially Update Object","href":"/api-reference/bucket-objects/partially-update-object"},{"openapi":"post /v1/buckets/{bucket_identifier}/objects/list","title":"List Objects","href":"/api-reference/bucket-objects/list-objects"},{"openapi":"post /v1/buckets/{bucket_identifier}/objects/aggregate","title":"Aggregate Objects","href":"/api-reference/bucket-objects/aggregate-objects"},{"openapi":"delete /v1/buckets/{bucket_identifier}/objects/{object_identifier}","title":"Delete Object","href":"/api-reference/bucket-objects/delete-object"},{"openapi":"get /v1/buckets/{bucket_identifier}/objects","title":"List Objects (GET)","href":"/api-reference/bucket-objects/list-objects-get"},{"openapi":"post /v1/buckets/{bucket_identifier}/objects/bulk-soft-delete","title":"Bulk Soft-Delete Objects","href":"/api-reference/bucket-objects/bulk-soft-delete-objects"}]},{"group":"Uploads","pages":[{"openapi":"post /v1/buckets/{bucket_identifier}/uploads","title":"Create Upload","href":"/api-reference/bucket-uploads/create-upload"},{"openapi":"post /v1/buckets/{bucket_identifier}/uploads/batch","title":"Batch Create Uploads","href":"/api-reference/bucket-uploads/batch-create-uploads"},{"openapi":"get /v1/buckets/{bucket_identifier}/uploads/{upload_id}","title":"Get Upload","href":"/api-reference/bucket-uploads/get-upload"},{"openapi":"post /v1/buckets/{bucket_identifier}/uploads/{upload_id}/confirm","title":"Confirm Upload","href":"/api-reference/bucket-uploads/confirm-upload"},{"openapi":"post /v1/buckets/{bucket_identifier}/uploads/confirm/batch","title":"Batch Confirm Uploads","href":"/api-reference/bucket-uploads/batch-confirm-uploads"},{"openapi":"post /v1/buckets/{bucket_identifier}/uploads/list","title":"List Uploads","href":"/api-reference/bucket-uploads/list-uploads"},{"openapi":"delete /v1/buckets/{bucket_identifier}/uploads/{upload_id}","title":"Delete Upload","href":"/api-reference/bucket-uploads/delete-upload"}]},{"group":"Batches","pages":[{"openapi":"post /v1/buckets/{bucket_identifier}/batches","title":"Create Batch","href":"/api-reference/bucket-batches/create-batch"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/objects","title":"Add Objects to Batch","href":"/api-reference/bucket-batches/add-objects-to-batch"},{"openapi":"get /v1/buckets/{bucket_identifier}/batches/{batch_id}","title":"Get Batch Configuration","description":"Retrieve batch configuration and historical data from MongoDB. Status is automatically synchronized from Task API. For real-time monitoring, use GET /v1/tasks/{task_id} (Redis, faster).","href":"/api-reference/bucket-batches/get-batch-configuration"},{"openapi":"patch /v1/buckets/{bucket_identifier}/batches/{batch_id}","title":"Partially Update Batch","href":"/api-reference/bucket-batches/partially-update-batch"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/submit","title":"Submit Batch for Processing","description":"Submit a draft batch for asynchronous processing. The batch must be in 'DRAFT' status and contain objects.","href":"/api-reference/bucket-batches/submit-batch-for-processing"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/submit-all","title":"Submit All Draft Batches","href":"/api-reference/bucket-batches/submit-all-draft-batches"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/list","title":"List Batches","href":"/api-reference/bucket-batches/list-batches"},{"openapi":"get /v1/buckets/{bucket_identifier}/batches/{batch_id}/logs","title":"Get Ray Job Logs for Batch","href":"/api-reference/bucket-batches/get-ray-job-logs-for-batch"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/cancel","title":"Cancel Batch","href":"/api-reference/bucket-batches/cancel-batch"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/cancel","title":"Cancel Tier","href":"/api-reference/bucket-batches/cancel-tier"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/jobs/{ray_job_id}/cancel","title":"Cancel Job","href":"/api-reference/bucket-batches/cancel-job"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/retry","title":"Retry Tier with Resource Overrides","href":"/api-reference/bucket-batches/retry-tier"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/retry","title":"Retry Failed Documents","href":"/api-reference/bucket-batches/retry-failed-documents"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/resubmit","title":"Resubmit Failed Batch","href":"/api-reference/bucket-batches/resubmit-failed-batch"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/tiers/{tier_num}/heal","title":"Manual Self-Healing","href":"/api-reference/bucket-batches/heal-tier"},{"openapi":"get /v1/buckets/{bucket_identifier}/batches/{batch_id}/diagnose","title":"Diagnose Batch","href":"/api-reference/bucket-batches/diagnose-batch"},{"openapi":"get /v1/buckets/{bucket_identifier}/batches/{batch_id}/failed-documents","title":"Get Failed Documents for Batch","href":"/api-reference/bucket-batches/get-failed-documents-for-batch"},{"openapi":"delete /v1/buckets/{bucket_identifier}/batches/{batch_id}/documents","title":"Delete Documents Produced by Batch","href":"/api-reference/bucket-batches/delete-documents-produced-by-batch"},{"openapi":"delete /v1/buckets/{bucket_identifier}/batches/{batch_id}","title":"Delete Batch","href":"/api-reference/bucket-batches/delete-batch"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/{batch_id}/estimate-cost","title":"Estimate Batch Cost (pre-flight, dry-run)","href":"/api-reference/bucket-batches/estimate-batch-cost"},{"openapi":"post /v1/buckets/{bucket_identifier}/batches/bulk-submit","title":"Bulk-submit a whole bucket as N auto-chunked, auto-queued batches","href":"/api-reference/bucket-batches/bulk-submit-whole-bucket-n-auto"},{"openapi":"get /v1/buckets/{bucket_identifier}/batches/{batch_id}/audit","title":"Tier-completion invariant audit","href":"/api-reference/bucket-batches/tier-completion-invariant-audit"}]},{"openapi":"get /v1/buckets","title":"List Buckets","href":"/api-reference/buckets/list-buckets-get"}]},{"group":"Collections","pages":[{"openapi":"post /v1/collections","title":"Create Collection","href":"/api-reference/collections/create-collection"},{"openapi":"post /v1/collections/{collection_identifier}/clone","title":"Clone Collection","href":"/api-reference/collections/clone-collection"},{"openapi":"get /v1/collections/{collection_identifier}","title":"Get Collection","href":"/api-reference/collections/get-collection"},{"openapi":"patch /v1/collections/{collection_identifier}","title":"Update Collection","href":"/api-reference/collections/update-collection"},{"openapi":"get /v1/collections/{collection_identifier}/features","title":"Describe collection features","href":"/api-reference/collections/describe-collection-features"},{"openapi":"post /v1/collections/list","title":"List Collections","href":"/api-reference/collections/list-collections"},{"openapi":"post /v1/collections/{collection_identifier}/export","title":"Export Collection","href":"/api-reference/collections/export-collection"},{"openapi":"post /v1/collections/{collection_identifier}/trigger","title":"Trigger Collection Processing","href":"/api-reference/collections/trigger-collection-processing"},{"openapi":"delete /v1/collections/{collection_identifier}","title":"Delete Collection","href":"/api-reference/collections/delete-collection"},{"openapi":"get /v1/collections/{collection_identifier}/reconciliation","title":"Reconcile objects vs documents","href":"/api-reference/collections/reconcile-collection"},{"openapi":"get /v1/collections/{collection_identifier}/documents/{document_id}/diagnostics","title":"Per-document support diagnostics (visibility, failures, history).","href":"/api-reference/collections/document-diagnostics"},{"group":"Schema","pages":[{"openapi":"post /v1/collections/{collection_id}/sync-schema","title":"Sync Collection Schema","href":"/api-reference/collection-schema/sync-collection-schema"}]},{"group":"Documents","pages":[{"openapi":"post /v1/collections/{collection_identifier}/documents","title":"Create a document.","href":"/api-reference/collection-documents/create-a-document"},{"openapi":"get /v1/collections/{collection_identifier}/documents/{document_id}","title":"Get a document by ID.","href":"/api-reference/collection-documents/get-a-document-by-id"},{"openapi":"get /v1/collections/{collection_identifier}/documents/{document_id}/download","title":"Download frame-accurate scene media for this document.","href":"/api-reference/collection-documents/download-document-media"},{"openapi":"put /v1/collections/{collection_identifier}/documents/{document_id}","title":"Update Document","href":"/api-reference/collection-documents/update-document"},{"openapi":"patch /v1/collections/{collection_identifier}/documents/{document_id}","title":"Patch Document","href":"/api-reference/collection-documents/patch-document"},{"openapi":"patch /v1/collections/{collection_identifier}/documents/{document_id}/acl","title":"Update a document's access control list (ACL).","href":"/api-reference/collection-documents/update-a-documents-access-control-list-acl"},{"openapi":"post /v1/collections/{collection_identifier}/documents/list","title":"List documents.","href":"/api-reference/collection-documents/list-documents"},{"openapi":"post /v1/collections/{collection_identifier}/documents/aggregate","title":"Aggregate Documents","href":"/api-reference/collection-documents/aggregate-documents"},{"openapi":"post /v1/collections/{collection_identifier}/documents/distinct","title":"Return distinct values for a single field.","href":"/api-reference/collection-documents/return-distinct-values-for-a-single-field"},{"openapi":"patch /v1/collections/{collection_identifier}/documents/bulk","title":"Bulk Update Documents","href":"/api-reference/collection-documents/bulk-update-documents"},{"openapi":"post /v1/collections/{collection_identifier}/documents/batch","title":"Batch Update Documents","href":"/api-reference/collection-documents/batch-update-documents"},{"openapi":"delete /v1/collections/{collection_identifier}/documents/batch","title":"Batch Delete Documents","href":"/api-reference/collection-documents/batch-delete-documents"},{"openapi":"delete /v1/collections/{collection_identifier}/documents/{document_id}","title":"Delete a document by ID.","href":"/api-reference/collection-documents/delete-a-document-by-id"},{"openapi":"post /v1/collections/{collection_identifier}/documents/schema/validate","title":"Validate document against collection schema","href":"/api-reference/collection-documents/validate-document-schema"},{"openapi":"get /v1/collections/{collection_identifier}/documents/count","title":"Count documents in a collection.","href":"/api-reference/collection-documents/count-documents-collection"},{"openapi":"get /v1/collections/{collection_identifier}/documents/{document_id}/ancestors","title":"Get document ancestors (lineage chain root → parent).","href":"/api-reference/collection-documents/get-document-ancestors"},{"openapi":"get /v1/collections/{collection_identifier}/documents/{document_id}/descendants","title":"Get direct downstream documents (depth=1 children).","href":"/api-reference/collection-documents/get-direct-downstream-documents"},{"openapi":"get /v1/collections/{collection_identifier}/documents","title":"List documents.","href":"/api-reference/collection-documents/list-documents-get"}]},{"group":"Lineage","pages":[{"openapi":"get /v1/collections/{collection_id}/documents/{document_id}/lineage","title":"Get document lineage","href":"/api-reference/document-lineage/get-document-lineage"},{"openapi":"get /v1/objects/{object_id}/documents","title":"Get all documents derived from an object","href":"/api-reference/document-lineage/get-all-documents-derived-from-an-object"},{"openapi":"get /v1/objects/{object_id}/decomposition-tree","title":"Get decomposition tree visualization","href":"/api-reference/document-lineage/get-decomposition-tree-visualization"}]},{"group":"Lifecycle","pages":[{"openapi":"get /v1/collections/{collection_identifier}/lifecycle","title":"Get Lifecycle Status","href":"/api-reference/collection-lifecycle/get-lifecycle-status"},{"openapi":"patch /v1/collections/{collection_identifier}/lifecycle","title":"Transition Lifecycle","href":"/api-reference/collection-lifecycle/transition-lifecycle"}]},{"group":"Taxonomies","pages":[{"openapi":"post /v1/collections/{collection_identifier}/apply-taxonomy","title":"Apply Taxonomy to Existing Documents","href":"/api-reference/collection-taxonomies/apply-taxonomy-to-existing-documents"}]},{"group":"Import","pages":[{"openapi":"post /v1/collections/import/infer","title":"Infer import schema","href":"/api-reference/collections/infer-import-schema"},{"openapi":"post /v1/collections/import/validate-mapping","title":"Validate an import mapping","href":"/api-reference/collections/validate-import-mapping"},{"openapi":"post /v1/collections/{collection_identifier}/import","title":"Import external documents into a collection","href":"/api-reference/collections/import-collection-documents"}]},{"openapi":"get /v1/collections/features","title":"Discover Features","href":"/api-reference/features/discover-features"}]},{"group":"Feature Extractors","pages":[{"openapi":"get /v1/collections/features/extractors","title":"List Feature Extractors","href":"/api-reference/feature-extractors/list-feature-extractors"},{"openapi":"get /v1/collections/features/extractors/{feature_extractor_id}","title":"Get Feature Extractor by Name","href":"/api-reference/feature-extractors/get-feature-extractor-by-name"},{"group":"Marketplace","pages":[{"openapi":"get /v1/public/extractors/","title":"List Public Extractors","href":"/api-reference/public-extractor-marketplace/list-public-extractors"},{"openapi":"get /v1/public/extractors/{public_name}","title":"Get Public Extractor","href":"/api-reference/public-extractor-marketplace/get-public-extractor"}]},{"group":"Publishing","pages":[{"openapi":"post /v1/extractors/{extractor_id}/publish","title":"Publish an extractor to the marketplace","href":"/api-reference/extractor-publishing/publish-extractor-marketplace"},{"openapi":"get /v1/extractors/{extractor_id}/publish","title":"Get published extractor status","href":"/api-reference/extractor-publishing/get-published-extractor-status"},{"openapi":"patch /v1/extractors/{extractor_id}/publish","title":"Update published extractor","href":"/api-reference/extractor-publishing/update-published-extractor"},{"openapi":"delete /v1/extractors/{extractor_id}/publish","title":"Unpublish an extractor","href":"/api-reference/extractor-publishing/unpublish-extractor"},{"openapi":"post /v1/extractors/install","title":"Install a public extractor","href":"/api-reference/extractor-publishing/install-public-extractor"}]},{"group":"Submissions","pages":[{"openapi":"get /v1/extractors","title":"List all extractors available to organization","href":"/api-reference/extractor-submissions/list-all-extractors-available-organization"},{"openapi":"post /v1/extractors/submissions/uploads","title":"Get presigned URL for extractor submission","href":"/api-reference/extractor-submissions/get-presigned-url-extractor-submission"},{"openapi":"post /v1/extractors/submissions/uploads/{submission_id}/confirm","title":"Confirm extractor submission upload","href":"/api-reference/extractor-submissions/confirm-extractor-submission-upload"},{"openapi":"get /v1/extractors/submissions","title":"List extractor submissions","href":"/api-reference/extractor-submissions/list-extractor-submissions"},{"openapi":"get /v1/extractors/submissions/{submission_id}","title":"Get submission details","href":"/api-reference/extractor-submissions/get-submission-details"}]}]},{"group":"Documents","pages":[{"openapi":"get /v1/documents/{document_id}","title":"Get a document by ID (namespace-scoped).","href":"/api-reference/documents/get-a-document-by-id-namespace-scoped"},{"openapi":"put /v1/documents/{document_id}","title":"Update a document by ID (namespace-scoped).","href":"/api-reference/documents/update-a-document-by-id-namespace-scoped"},{"openapi":"patch /v1/documents/{document_id}","title":"Partially update a document by ID (namespace-scoped).","href":"/api-reference/documents/partially-update-a-document-by-id-namespace-scoped"},{"openapi":"post /v1/documents/list","title":"List documents across all collections (namespace-scoped).","href":"/api-reference/documents/list-documents-across-all-collections-namespace-scoped"},{"openapi":"post /v1/documents/batch-get","title":"Batch get documents by IDs","href":"/api-reference/documents/batch-get-documents-by-ids"},{"openapi":"post /v1/documents/list/batch","title":"Run multiple list-documents queries concurrently.","href":"/api-reference/documents/run-multiple-list-documents-queries-concurrently"}]},{"group":"Retrievers","pages":[{"openapi":"post /v1/retrievers","title":"Create Retriever","href":"/api-reference/retrievers/create-retriever"},{"openapi":"post /v1/retrievers/list","title":"List Retrievers","href":"/api-reference/retrievers/list-retrievers"},{"openapi":"get /v1/retrievers/{retriever_id}","title":"Get Retriever","href":"/api-reference/retrievers/get-retriever"},{"openapi":"patch /v1/retrievers/{retriever_id}","title":"Patch Retriever","href":"/api-reference/retrievers/patch-retriever"},{"openapi":"post /v1/retrievers/{retriever_id}/clone","title":"Clone Retriever","href":"/api-reference/retrievers/clone-retriever"},{"openapi":"delete /v1/retrievers/{retriever_id}","title":"Delete Retriever","href":"/api-reference/retrievers/delete-retriever"},{"openapi":"post /v1/retrievers/{retriever_id}/execute","title":"Execute Retriever (Auto-Optimized)","href":"/api-reference/retrievers/execute-retriever-auto-optimized"},{"openapi":"post /v1/retrievers/{retriever_id}/execute/batch","title":"Batch Execute Retriever","href":"/api-reference/retrievers/batch-execute-retriever"},{"openapi":"post /v1/retrievers/{retriever_id}/executions/list","title":"List Executions","href":"/api-reference/retrievers/list-executions"},{"openapi":"get /v1/retrievers/{retriever_id}/executions/{execution_id}","title":"Get Execution","href":"/api-reference/retrievers/get-execution"},{"openapi":"get /v1/retrievers/{retriever_id}/executions/{execution_id}/status","title":"Get Live Execution Status","href":"/api-reference/retrievers/get-live-execution-status"},{"openapi":"post /v1/retrievers/{retriever_id}/execute/explain","title":"Explain Retriever Execution Plan","href":"/api-reference/retrievers/explain-retriever-execution-plan"},{"openapi":"get /v1/retrievers/{retriever_id}/executions/{execution_id}/profile","title":"Get Execution Query Profile (explain-on-execute read path)","href":"/api-reference/retrievers/get-execution-profile"},{"openapi":"get /v1/retrievers/typeahead","title":"Typeahead","href":"/api-reference/retrievers/typeahead"},{"group":"Stages","pages":[{"openapi":"get /v1/retrievers/stages","title":"List Available Retriever Stages","href":"/api-reference/retriever-stages/list-available-retriever-stages"}]},{"group":"Interactions","pages":[{"openapi":"post /v1/retrievers/interactions","title":"Create Interaction","href":"/api-reference/retriever-interactions/create-interaction"},{"openapi":"post /v1/retrievers/interactions/list","title":"List Interactions","href":"/api-reference/retriever-interactions/list-interactions"},{"openapi":"get /v1/retrievers/interactions/{interaction_id}","title":"Get Interaction","href":"/api-reference/retriever-interactions/get-interaction"},{"openapi":"delete /v1/retrievers/interactions/{interaction_id}","title":"Delete Interaction","href":"/api-reference/retriever-interactions/delete-interaction"},{"openapi":"post /v1/retrievers/interactions/batch","title":"Create Interactions Bulk","href":"/api-reference/retriever-interactions/create-interactions-bulk"}]},{"group":"Evaluations","pages":[{"openapi":"get /v1/retrievers/evaluations/datasets","title":"List evaluation datasets","href":"/api-reference/retriever-evaluations/list-evaluation-datasets"},{"openapi":"post /v1/retrievers/evaluations/datasets","title":"Create evaluation dataset","href":"/api-reference/retriever-evaluations/create-evaluation-dataset"},{"openapi":"get /v1/retrievers/evaluations/datasets/{dataset_identifier}","title":"Get evaluation dataset","href":"/api-reference/retriever-evaluations/get-evaluation-dataset"},{"openapi":"get /v1/retrievers/{retriever_id}/evaluations","title":"List evaluations","href":"/api-reference/retriever-evaluations/list-evaluations"},{"openapi":"post /v1/retrievers/{retriever_id}/evaluations","title":"Run evaluation","href":"/api-reference/retriever-evaluations/run-evaluation"},{"openapi":"get /v1/retrievers/{retriever_id}/evaluations/{evaluation_id}","title":"Get evaluation results","href":"/api-reference/retriever-evaluations/get-evaluation-results"},{"openapi":"post /v1/evaluations/score","title":"Score predictions vs ground truth (stateless)","href":"/api-reference/retriever-evaluations/score-predictions"},{"openapi":"get /v1/evaluations","title":"Discover evaluation endpoints","href":"/api-reference/retriever-evaluations/discover-evaluation-endpoints"},{"openapi":"post /v1/retrievers/{retriever_id}/evaluations/generate-from-interactions","title":"Generate evaluation dataset from interactions","href":"/api-reference/retriever-evaluations/generate-from-interactions"}]},{"group":"Benchmarks","pages":[{"openapi":"get /v1/retrievers/benchmarks","title":"List benchmarks","href":"/api-reference/retriever-benchmarks/list-benchmarks"},{"openapi":"post /v1/retrievers/benchmarks","title":"Create benchmark","href":"/api-reference/retriever-benchmarks/create-benchmark"},{"openapi":"get /v1/retrievers/benchmarks/{benchmark_id}","title":"Get benchmark","href":"/api-reference/retriever-benchmarks/get-benchmark"},{"openapi":"delete /v1/retrievers/benchmarks/{benchmark_id}","title":"Delete benchmark","href":"/api-reference/retriever-benchmarks/delete-benchmark"}]},{"group":"API Keys","pages":[{"openapi":"post /v1/retrievers/{retriever_id}/api-keys","title":"Create Retriever API Key","href":"/api-reference/retriever-api-keys/create-retriever-api-key"},{"openapi":"get /v1/retrievers/{retriever_id}/api-keys","title":"List Retriever API Keys","href":"/api-reference/retriever-api-keys/list-retriever-api-keys"},{"openapi":"delete /v1/retrievers/{retriever_id}/api-keys/{key_id}","title":"Revoke Retriever API Key","href":"/api-reference/retriever-api-keys/revoke-retriever-api-key"}]},{"group":"Adhoc","pages":[{"openapi":"post /v1/retrievers/execute","title":"Execute Adhoc Retriever","href":"/api-reference/adhoc-retrievers/execute-adhoc-retriever"},{"openapi":"post /v1/retrievers/executions/list","title":"List Adhoc Executions","href":"/api-reference/adhoc-retrievers/list-adhoc-executions"},{"openapi":"get /v1/retrievers/executions/{execution_id}","title":"Get Adhoc Execution","href":"/api-reference/adhoc-retrievers/get-adhoc-execution"}]},{"group":"Publishing","pages":[{"openapi":"post /v1/retrievers/{retriever_id}/publish","title":"Publish Retriever","href":"/api-reference/published-retrievers/publish-retriever"},{"openapi":"get /v1/retrievers/published","title":"List Published Retrievers","href":"/api-reference/published-retrievers/list-published-retrievers"},{"openapi":"get /v1/retrievers/{retriever_id}/publish","title":"Get Published Retriever","href":"/api-reference/published-retrievers/get-published-retriever"},{"openapi":"patch /v1/retrievers/{retriever_id}/publish","title":"Update Published Retriever","href":"/api-reference/published-retrievers/update-published-retriever"},{"openapi":"delete /v1/retrievers/{retriever_id}/publish","title":"Unpublish Retriever","href":"/api-reference/published-retrievers/unpublish-retriever"},{"openapi":"get /v1/retrievers/{retriever_id}/publish/availability","title":"Check Name Availability","href":"/api-reference/published-retrievers/check-name-availability"},{"openapi":"get /v1/retrievers/publish/stats","title":"Get Organization Publish Stats","href":"/api-reference/published-retrievers/get-organization-publish-stats"},{"openapi":"post /v1/retrievers/{retriever_id}/publish/reconcile","title":"Reconcile Publish State","href":"/api-reference/published-retrievers/reconcile-publish-state"}]},{"group":"Public Retriever API","pages":[{"openapi":"get /v1/public/retrievers/","title":"List Public Retrievers","href":"/api-reference/public-retriever-api/list-public-retrievers"},{"openapi":"get /v1/public/retrievers/{public_name}/config","title":"Get Public Retriever Config","href":"/api-reference/public-retriever-api/get-public-retriever-config"},{"openapi":"get /v1/public/retrievers/{public_name}/template","title":"Get Public Retriever Template","href":"/api-reference/public-retriever-api/get-public-retriever-template"},{"openapi":"post /v1/public/retrievers/{public_name}/execute","title":"Execute Public Retriever","href":"/api-reference/public-retriever-api/execute-public-retriever"},{"openapi":"post /v1/public/retrievers/{public_name}/verify","title":"Verify Password","href":"/api-reference/public-retriever-api/verify-password"},{"openapi":"post /v1/public/retrievers/{public_name}/interactions","title":"Track Interaction","href":"/api-reference/public-retriever-api/track-interaction"},{"openapi":"post /v1/public/retrievers/{public_name}/interactions/batch","title":"Track Interaction Batch","href":"/api-reference/public-retriever-api/track-interaction-batch"}]},{"group":"Feature search","pages":[{"openapi":"post /v1/features/search","title":"Feature Search","href":"/api-reference/feature-search/feature-search"}]}]},{"group":"Enrich \u0026 Improve","pages":[{"group":"Taxonomies","pages":[{"openapi":"post /v1/taxonomies","title":"Create Taxonomy","href":"/api-reference/taxonomies/create-taxonomy"},{"openapi":"get /v1/taxonomies/{taxonomy_identifier}","title":"Get Taxonomy","href":"/api-reference/taxonomies/get-taxonomy"},{"openapi":"patch /v1/taxonomies/{taxonomy_identifier}","title":"Partially Update Taxonomy","href":"/api-reference/taxonomies/partially-update-taxonomy"},{"openapi":"post /v1/taxonomies/list","title":"List Taxonomies","href":"/api-reference/taxonomies/list-taxonomies"},{"openapi":"post /v1/taxonomies/execute/{taxonomy_identifier}","title":"Test taxonomy configuration (validation only) — DEPRECATED path","href":"/api-reference/taxonomies/test-taxonomy-configuration-validation-only"},{"openapi":"get /v1/taxonomies/{taxonomy_id}/versions","title":"List Taxonomy Versions","href":"/api-reference/taxonomies/list-taxonomy-versions"},{"openapi":"post /v1/taxonomies/{taxonomy_id}/versions","title":"Create Taxonomy Version","href":"/api-reference/taxonomies/create-taxonomy-version"},{"openapi":"post /v1/taxonomies/{taxonomy_identifier}/clone","title":"Clone Taxonomy","href":"/api-reference/taxonomies/clone-taxonomy"},{"openapi":"delete /v1/taxonomies/{taxonomy_identifier}","title":"Delete Taxonomy","href":"/api-reference/taxonomies/delete-taxonomy"},{"group":"Analytics","pages":[{"openapi":"get /v1/taxonomies/{taxonomy_id}/analytics/available-steps","title":"Get Available Steps","href":"/api-reference/taxonomy-analytics/get-available-steps"},{"openapi":"post /v1/taxonomies/{taxonomy_id}/analytics/transitions","title":"Compute step transition analytics","href":"/api-reference/taxonomy-analytics/compute-step-transition-analytics"},{"openapi":"post /v1/taxonomies/{taxonomy_id}/analytics/paths","title":"Analyze multi-step transition paths","href":"/api-reference/taxonomy-analytics/analyze-multi-step-transition-paths"},{"openapi":"get /v1/taxonomies/{taxonomy_id}/analytics/runs","title":"List taxonomy analytics run history","href":"/api-reference/taxonomy-analytics/list-taxonomy-analytics-run-history"},{"openapi":"get /v1/taxonomies/{taxonomy_id}/analytics/runs/{run_id}","title":"Get a taxonomy analytics run","href":"/api-reference/taxonomy-analytics/get-taxonomy-analytics-run"},{"openapi":"post /v1/taxonomies/{taxonomy_id}/analytics/runs/{run_id}/rerun","title":"Re-run a stored taxonomy analytics run","href":"/api-reference/taxonomy-analytics/re-run-stored-taxonomy-analytics-run"}]},{"openapi":"get /v1/taxonomies","title":"List taxonomies (GET alias for POST /list).","href":"/api-reference/taxonomies/list-taxonomies-get"},{"openapi":"post /v1/taxonomies/from-documents","title":"Create a taxonomy from a document selection or a cluster run (one gesture)","href":"/api-reference/taxonomies/create-taxonomy-from-document-selection"},{"openapi":"post /v1/taxonomies/{taxonomy_identifier}/nodes/from-documents","title":"Add a taxonomy node derived from a document selection","href":"/api-reference/taxonomies/add-taxonomy-node-derived-from-document"},{"openapi":"post /v1/taxonomies/{taxonomy_identifier}/execute","title":"Test taxonomy configuration (validation only)","href":"/api-reference/taxonomies/test-taxonomy-configuration"}]},{"group":"Clusters","pages":[{"openapi":"post /v1/clusters","title":"Create Cluster","href":"/api-reference/clusters/create-cluster"},{"openapi":"get /v1/clusters/{cluster_identifier}","title":"Get Cluster","href":"/api-reference/clusters/get-cluster"},{"openapi":"patch /v1/clusters/{cluster_identifier}","title":"Partially Update Cluster","href":"/api-reference/clusters/partially-update-cluster"},{"openapi":"post /v1/clusters/list","title":"List Clusters","href":"/api-reference/clusters/list-clusters"},{"openapi":"post /v1/clusters/{cluster_id}/execute","title":"Execute Clustering","href":"/api-reference/clusters/execute-clustering"},{"openapi":"get /v1/clusters/{cluster_id}/groups","title":"List Cluster Groups","href":"/api-reference/clusters/list-cluster-groups"},{"openapi":"delete /v1/clusters/{cluster_id}","title":"Delete Cluster","href":"/api-reference/clusters/delete-cluster"},{"group":"Triggers","pages":[{"openapi":"post /v1/clusters/triggers","title":"Create Cluster Trigger","href":"/api-reference/cluster-triggers/create-cluster-trigger"},{"openapi":"post /v1/clusters/triggers/list","title":"List Cluster Triggers","href":"/api-reference/cluster-triggers/list-cluster-triggers"},{"openapi":"get /v1/clusters/triggers/{trigger_id}","title":"Get Cluster Trigger","href":"/api-reference/cluster-triggers/get-cluster-trigger"},{"openapi":"patch /v1/clusters/triggers/{trigger_id}","title":"Update Cluster Trigger","href":"/api-reference/cluster-triggers/update-cluster-trigger"},{"openapi":"post /v1/clusters/triggers/{trigger_id}/pause","title":"Pause Cluster Trigger","href":"/api-reference/cluster-triggers/pause-cluster-trigger"},{"openapi":"post /v1/clusters/triggers/{trigger_id}/resume","title":"Resume Cluster Trigger","href":"/api-reference/cluster-triggers/resume-cluster-trigger"},{"openapi":"post /v1/clusters/triggers/{trigger_id}/history","title":"Get Trigger Execution History","href":"/api-reference/cluster-triggers/get-trigger-execution-history"},{"openapi":"delete /v1/clusters/triggers/{trigger_id}","title":"Delete Cluster Trigger","href":"/api-reference/cluster-triggers/delete-cluster-trigger"}]},{"group":"Executions","pages":[{"openapi":"get /v1/clusters/{cluster_id}/executions","title":"Get Latest Cluster Execution","href":"/api-reference/cluster-executions/get-latest-cluster-execution"},{"openapi":"get /v1/clusters/{cluster_id}/executions/{run_id}","title":"Get Specific Cluster Execution","href":"/api-reference/cluster-executions/get-specific-cluster-execution"},{"openapi":"post /v1/clusters/{cluster_id}/executions/list","title":"List Cluster Execution History","href":"/api-reference/cluster-executions/list-cluster-execution-history"},{"openapi":"post /v1/clusters/{cluster_id}/executions/{run_id}/cancel","title":"Cancel Execution","href":"/api-reference/cluster-executions/cancel-execution"},{"openapi":"get /v1/clusters/{cluster_id}/executions/{run_id}/export","title":"Export Cluster Run Data (CSV / Parquet)","href":"/api-reference/cluster-executions/export-cluster-run-data"},{"openapi":"patch /v1/clusters/{cluster_id}/executions/{run_id}/labels","title":"Rename Cluster Labels (Per-Execution)","href":"/api-reference/cluster-executions/rename-cluster-labels"},{"openapi":"patch /v1/clusters/{cluster_id}/executions/{run_id}/name","title":"Name an Execution Run","href":"/api-reference/cluster-executions/name-execution-run"}]},{"openapi":"get /v1/clusters","title":"List Clusters (GET)","href":"/api-reference/clusters/list-clusters-get"},{"openapi":"get /v1/clusters/{cluster_id}/visualization","title":"Cluster Visualization Data","href":"/api-reference/clusters/cluster-visualization-data"}]},{"group":"Annotations","pages":[{"openapi":"post /v1/annotations","title":"Create Annotation","href":"/api-reference/annotations/create-annotation"},{"openapi":"get /v1/annotations/{annotation_id}","title":"Get Annotation","href":"/api-reference/annotations/get-annotation"},{"openapi":"get /v1/annotations/{annotation_id}/versions","title":"Get Annotation Versions","href":"/api-reference/annotations/get-annotation-versions"},{"openapi":"get /v1/annotations/{annotation_id}/versions/{version}","title":"Get Annotation At Version","href":"/api-reference/annotations/get-annotation-at-version"},{"openapi":"post /v1/annotations/list","title":"List Annotations","href":"/api-reference/annotations/list-annotations"},{"openapi":"get /v1/annotations/stats","title":"Annotation Stats","href":"/api-reference/annotations/annotation-stats"},{"openapi":"patch /v1/annotations/{annotation_id}","title":"Patch Annotation","href":"/api-reference/annotations/update-annotation"},{"openapi":"delete /v1/annotations/{annotation_id}","title":"Delete Annotation","href":"/api-reference/annotations/delete-annotation"},{"openapi":"post /v1/annotations/bulk","title":"Bulk Annotations","href":"/api-reference/annotations/bulk-annotations"}]}]},{"group":"Automate","pages":[{"group":"Triggers","pages":[{"openapi":"post /v1/triggers","title":"Create Trigger","href":"/api-reference/create-trigger"},{"openapi":"post /v1/triggers/list","title":"List Triggers","href":"/api-reference/list-triggers"},{"openapi":"get /v1/triggers/{trigger_id}","title":"Get Trigger","href":"/api-reference/get-trigger"},{"openapi":"patch /v1/triggers/{trigger_id}","title":"Update Trigger","href":"/api-reference/update-trigger"},{"openapi":"delete /v1/triggers/{trigger_id}","title":"Delete Trigger","href":"/api-reference/delete-trigger"},{"openapi":"post /v1/triggers/{trigger_id}/execute","title":"Execute Trigger Now","href":"/api-reference/execute-trigger-now"},{"openapi":"post /v1/triggers/{trigger_id}/pause","title":"Pause Trigger","href":"/api-reference/pause-trigger"},{"openapi":"post /v1/triggers/{trigger_id}/resume","title":"Resume Trigger","href":"/api-reference/resume-trigger"},{"openapi":"get /v1/triggers/{trigger_id}/history","title":"Get Trigger Execution History","href":"/api-reference/get-trigger-execution-history"},{"openapi":"get /v1/triggers","title":"List Triggers (GET alias for POST /list).","href":"/api-reference/triggers/list-triggers"}]},{"group":"Webhooks","pages":[{"openapi":"post /v1/organizations/webhooks","title":"Create Webhook","href":"/api-reference/webhooks/create-webhook"},{"openapi":"post /v1/organizations/webhooks/list","title":"List Webhooks","href":"/api-reference/webhooks/list-webhooks"},{"openapi":"get /v1/organizations/webhooks/{identifier}","title":"Get Webhook","href":"/api-reference/webhooks/get-webhook"},{"openapi":"put /v1/organizations/webhooks/{identifier}","title":"Update Webhook","href":"/api-reference/webhooks/update-webhook"},{"openapi":"delete /v1/organizations/webhooks/{identifier}","title":"Delete Webhook","href":"/api-reference/webhooks/delete-webhook"}]},{"group":"Alerts","pages":[{"openapi":"post /v1/alerts","title":"Create Alert","href":"/api-reference/alerts/create-alert"},{"openapi":"get /v1/alerts/{alert_identifier}","title":"Get Alert","href":"/api-reference/alerts/get-alert"},{"openapi":"patch /v1/alerts/{alert_identifier}","title":"Update Alert","href":"/api-reference/alerts/update-alert"},{"openapi":"post /v1/alerts/list","title":"List Alerts","href":"/api-reference/alerts/list-alerts"},{"openapi":"delete /v1/alerts/{alert_identifier}","title":"Delete Alert","href":"/api-reference/alerts/delete-alert"},{"openapi":"get /v1/alerts/{alert_identifier}/executions","title":"List Alert Executions","href":"/api-reference/alerts/list-alert-executions"},{"openapi":"get /v1/alerts/executions/{execution_id}","title":"Get Alert Execution","href":"/api-reference/alerts/get-alert-execution"},{"openapi":"get /v1/alerts/{alert_identifier}/results","title":"Get Latest Alert Result","href":"/api-reference/alerts/get-latest-alert-result"},{"openapi":"get /v1/alerts/executions","title":"List All Executions","href":"/api-reference/alerts/list-all-executions"},{"openapi":"get /v1/alerts","title":"List Alerts (GET alias for POST /list).","href":"/api-reference/alerts/list-alerts-get"},{"openapi":"post /v1/organizations/alerts","title":"Create Org-Scoped Alert","href":"/api-reference/alerts/create-org-scoped-alert"},{"openapi":"get /v1/organizations/alerts","title":"List Org-Scoped Alerts (GET alias for POST /list).","href":"/api-reference/alerts/list-org-scoped-alerts"},{"openapi":"post /v1/organizations/alerts/list","title":"List Org-Scoped Alerts","href":"/api-reference/alerts/list-org-scoped-alerts-list"},{"openapi":"get /v1/organizations/alerts/{alert_identifier}","title":"Get Org-Scoped Alert","href":"/api-reference/alerts/get-org-scoped-alert"},{"openapi":"patch /v1/organizations/alerts/{alert_identifier}","title":"Update Org-Scoped Alert","href":"/api-reference/alerts/update-org-scoped-alert"},{"openapi":"delete /v1/organizations/alerts/{alert_identifier}","title":"Delete Org-Scoped Alert","href":"/api-reference/alerts/delete-org-scoped-alert"}]},{"group":"Notifications","pages":[{"openapi":"post /v1/notifications/list","title":"List Notifications","href":"/api-reference/notifications/list-notifications"},{"openapi":"get /v1/notifications/{notification_id}","title":"Get Notification","href":"/api-reference/notifications/get-notification"},{"openapi":"post /v1/notifications/{notification_id}/read","title":"Mark As Read","href":"/api-reference/notifications/mark-as-read"},{"openapi":"post /v1/notifications/read/all","title":"Mark All As Read","href":"/api-reference/notifications/mark-all-as-read"},{"openapi":"get /v1/notifications/unread/count","title":"Get Unread Count","href":"/api-reference/notifications/get-unread-count"},{"openapi":"delete /v1/notifications/{notification_id}","title":"Delete Notification","href":"/api-reference/notifications/delete-notification"},{"openapi":"get /v1/notifications/preferences","title":"Get Preferences","href":"/api-reference/notifications/get-preferences"},{"openapi":"put /v1/notifications/preferences","title":"Update Preferences","href":"/api-reference/notifications/update-preferences"},{"openapi":"get /v1/notifications/preferences/reminders","title":"Get Reminder Preferences","href":"/api-reference/notifications/get-reminder-preferences"},{"openapi":"put /v1/notifications/preferences/reminders","title":"Update Reminder Preferences","href":"/api-reference/notifications/update-reminder-preferences"},{"openapi":"get /v1/notifications/funnel/state","title":"Get Funnel State","href":"/api-reference/notifications/get-funnel-state"},{"openapi":"post /v1/notifications/funnel/advance","title":"Advance Funnel Stage","href":"/api-reference/notifications/advance-funnel-stage"}]},{"group":"Tasks","pages":[{"openapi":"post /v1/tasks/list","title":"List Tasks","href":"/api-reference/tasks/list-tasks"},{"openapi":"get /v1/tasks/{task_id}","title":"Get Task Information","href":"/api-reference/tasks/get-task-information"},{"openapi":"delete /v1/tasks/{task_id}","title":"Kill Task","href":"/api-reference/tasks/kill-task"}]},{"group":"Batch Queue","pages":[{"openapi":"post /v1/batches/list","title":"List All Batches","href":"/api-reference/batches/list-all-batches"},{"openapi":"get /v1/batches/{batch_id}","title":"Get Batch by ID","href":"/api-reference/batches/get-batch-by-id"},{"openapi":"post /v1/batches/{batch_id}/cancel","title":"Cancel Batch","href":"/api-reference/batches/cancel-batch"},{"openapi":"get /v1/batches/queue","title":"Batch Queue Status","href":"/api-reference/batches/batch-queue-status"}]}]},{"group":"Administer","pages":[{"group":"Organization","pages":[{"openapi":"get /v1/me","title":"Whoami","href":"/api-reference/account/whoami"},{"openapi":"get /v1/organizations","title":"Get Organization","href":"/api-reference/organizations/get-organization"},{"openapi":"patch /v1/organizations","title":"Update Organization","href":"/api-reference/organizations/update-organization"},{"openapi":"get /v1/organizations/targets","title":"Get Organization Targets","href":"/api-reference/organizations/get-organization-targets"},{"group":"Users","pages":[{"openapi":"get /v1/organizations/users","title":"List Users","href":"/api-reference/organization-users/list-users"},{"openapi":"post /v1/organizations/users","title":"Create User","href":"/api-reference/organization-users/create-user"},{"openapi":"get /v1/organizations/users/{user_email}","title":"Get User","href":"/api-reference/organization-users/get-user"},{"openapi":"patch /v1/organizations/users/{user_email}","title":"Update User","href":"/api-reference/organization-users/update-user"},{"openapi":"delete /v1/organizations/users/{user_email}","title":"Delete User","href":"/api-reference/organization-users/delete-user"}]},{"group":"API Keys","pages":[{"openapi":"get /v1/organizations/users/{user_email}/api-keys","title":"List Api Keys","href":"/api-reference/organization-api-keys/list-api-keys"},{"openapi":"post /v1/organizations/users/{user_email}/api-keys","title":"Create Api Key","href":"/api-reference/organization-api-keys/create-api-key"},{"openapi":"patch /v1/organizations/users/{user_email}/api-keys/{key_name}","title":"Update Api Key","href":"/api-reference/organization-api-keys/update-api-key"},{"openapi":"post /v1/organizations/users/{user_email}/api-keys/{key_name}/rotate","title":"Rotate Api Key","href":"/api-reference/organization-api-keys/rotate-api-key"},{"openapi":"delete /v1/organizations/users/{user_email}/api-keys/{key_name}","title":"Delete Api Key","href":"/api-reference/organization-api-keys/delete-api-key"},{"openapi":"post /v1/auth/token-exchange","title":"Exchange a workload OIDC token for a short-lived API key","href":"/api-reference/organization-api-keys/exchange-oidc-token"}]},{"group":"Storage Connections","pages":[{"openapi":"post /v1/organizations/connections/list","title":"List Storage Connections","href":"/api-reference/organization-connections/list-storage-connections"},{"openapi":"post /v1/organizations/connections","title":"Create Storage Connection","href":"/api-reference/organization-connections/create-storage-connection"},{"openapi":"get /v1/organizations/connections/{connection_identifier}","title":"Get Storage Connection","href":"/api-reference/organization-connections/get-storage-connection"},{"openapi":"patch /v1/organizations/connections/{connection_identifier}","title":"Update Storage Connection","href":"/api-reference/organization-connections/update-storage-connection"},{"openapi":"post /v1/organizations/connections/{connection_identifier}/test","title":"Test Storage Connection","href":"/api-reference/organization-connections/test-storage-connection"},{"openapi":"delete /v1/organizations/connections/{connection_identifier}","title":"Delete Storage Connection","href":"/api-reference/organization-connections/delete-storage-connection"},{"openapi":"get /v1/organizations/connections/{connection_identifier}/files","title":"List Google Drive Files","href":"/api-reference/organization-connections/list-google-drive-files"},{"openapi":"get /v1/organizations/connections/{connection_identifier}/folders","title":"List Google Drive Folders","href":"/api-reference/organization-connections/list-google-drive-folders"}]},{"group":"Secrets","pages":[{"openapi":"get /v1/organizations/secrets","title":"List Secrets","href":"/api-reference/organization-secrets/list-secrets"},{"openapi":"post /v1/organizations/secrets","title":"Create Secret","href":"/api-reference/organization-secrets/create-secret"},{"openapi":"put /v1/organizations/secrets/{secret_name}","title":"Update Secret","href":"/api-reference/organization-secrets/update-secret"},{"openapi":"delete /v1/organizations/secrets/{secret_name}","title":"Delete Secret","href":"/api-reference/organization-secrets/delete-secret"}]},{"group":"Billing","pages":[{"openapi":"get /v1/organizations/billing/balance","title":"Get Credit Balance","href":"/api-reference/organization-billing/get-credit-balance"},{"openapi":"get /v1/organizations/billing/usage/current","title":"Get Current Usage","href":"/api-reference/organization-billing/get-current-usage"},{"openapi":"get /v1/organizations/billing/usage/vector-backend","title":"Get Vector Backend Usage","href":"/api-reference/organization-billing/get-vector-backend-usage"},{"openapi":"get /v1/organizations/billing/usage/breakdown","title":"Get Usage Breakdown","href":"/api-reference/organization-billing/get-usage-breakdown"},{"openapi":"get /v1/organizations/billing/spending-caps","title":"Get Spending Caps","href":"/api-reference/organization-billing/get-spending-caps"},{"openapi":"post /v1/organizations/billing/spending-caps","title":"Update Spending Caps","href":"/api-reference/organization-billing/update-spending-caps"},{"openapi":"post /v1/organizations/billing/setup-payment-method","title":"Setup Payment Method","href":"/api-reference/organization-billing/setup-payment-method"},{"openapi":"post /v1/organizations/billing/confirm-payment-method","title":"Confirm Payment Method","href":"/api-reference/organization-billing/confirm-payment-method"},{"openapi":"get /v1/organizations/billing/payment-method","title":"Get Payment Method","href":"/api-reference/organization-billing/get-payment-method"},{"openapi":"get /v1/organizations/billing/invoices","title":"List Invoices","href":"/api-reference/organization-billing/list-invoices"},{"openapi":"post /v1/organizations/billing/generate-invoice","title":"Generate Invoice Now","href":"/api-reference/organization-billing/generate-invoice-now"},{"openapi":"post /v1/organizations/billing/enable-auto-billing","title":"Enable Auto Billing","href":"/api-reference/organization-billing/enable-auto-billing"},{"openapi":"post /v1/organizations/billing/disable-auto-billing","title":"Disable Auto Billing","href":"/api-reference/organization-billing/disable-auto-billing"},{"openapi":"post /v1/organizations/billing/estimate","title":"Estimate Cost","href":"/api-reference/organization-billing/estimate-cost"},{"openapi":"get /v1/organizations/billing/transactions","title":"List Transactions","href":"/api-reference/organization-billing/list-transactions"},{"openapi":"post /v1/organizations/billing/checkout","title":"Create Checkout","href":"/api-reference/organization-billing/create-checkout"},{"openapi":"post /v1/organizations/billing/subscribe","title":"Subscribe To Plan","href":"/api-reference/organization-billing/subscribe-plan"},{"openapi":"post /v1/organizations/billing/portal","title":"Open Billing Portal","href":"/api-reference/organization-billing/open-billing-portal"},{"openapi":"get /v1/organizations/billing/subscription","title":"Get Subscription Status","href":"/api-reference/organization-billing/get-subscription-status"},{"openapi":"post /v1/organizations/billing/subscription/cancel","title":"Cancel Subscription","href":"/api-reference/organization-billing/cancel-subscription"},{"openapi":"post /v1/organizations/billing/apply-promo","title":"Apply Promo","href":"/api-reference/organization-billing/apply-promo"},{"openapi":"post /v1/organizations/billing/redeem-promo","title":"Redeem Promo","href":"/api-reference/organization-billing/redeem-promo"}]},{"group":"Usage","pages":[{"openapi":"get /v1/organizations/usage","title":"Get Org Usage","href":"/api-reference/organization-usage/get-org-usage"},{"openapi":"get /v1/organizations/api-keys/{key_id}/usage","title":"Get Api Key Usage","href":"/api-reference/organization-usage/get-api-key-usage"},{"openapi":"get /v1/organizations/api-keys/{key_id}/usage/endpoints","title":"Get Api Key Endpoint Breakdown","href":"/api-reference/organization-usage/get-api-key-endpoint-breakdown"}]},{"group":"Audit Logs","pages":[{"openapi":"get /v1/organizations/audit/logs","title":"List Audit Logs","href":"/api-reference/organization-audit/list-audit-logs"},{"openapi":"get /v1/organizations/audit/logs/export","title":"Export Audit Logs","href":"/api-reference/organization-audit/export-audit-logs"},{"openapi":"get /v1/organizations/audit/logs/{audit_id}","title":"Get Audit Log","href":"/api-reference/organization-audit/get-audit-log"},{"openapi":"get /v1/organizations/audit/settings","title":"Get Audit Settings","href":"/api-reference/organization-audit/get-audit-settings"},{"openapi":"patch /v1/organizations/audit/settings","title":"Update Audit Settings","href":"/api-reference/organization-audit/update-audit-settings"}]},{"group":"Privacy (DSAR)","pages":[{"openapi":"post /v1/organizations/dsar/export","title":"Request Data Export","href":"/api-reference/data-subject-access-requests/request-data-export"},{"openapi":"post /v1/organizations/dsar/delete","title":"Request Data Deletion","href":"/api-reference/data-subject-access-requests/request-data-deletion"},{"openapi":"get /v1/organizations/dsar/status","title":"List Dsar Requests","href":"/api-reference/data-subject-access-requests/list-dsar-requests"}]},{"openapi":"get /v1/organizations/credits","title":"Get Credits","href":"/api-reference/organizations/get-credits"},{"group":"Infrastructure","pages":[{"openapi":"get /v1/organizations/infrastructure/summary","title":"Get Infrastructure","href":"/api-reference/organization-infrastructure/get-infrastructure"}]}]},{"group":"Pricing","pages":[{"openapi":"get /v1/billing/pricing","title":"Get Pricing Configuration","href":"/api-reference/pricing/get-pricing-configuration"}]},{"group":"Support","pages":[{"openapi":"post /v1/support/tickets","title":"Create Support Ticket","href":"/api-reference/support/create-support-ticket"},{"openapi":"get /v1/support/tickets","title":"List Support Tickets","href":"/api-reference/support/list-support-tickets"},{"openapi":"post /v1/support/tickets/list","title":"List Support Tickets (org + global).","href":"/api-reference/support/list-support-tickets-list"},{"openapi":"get /v1/support/tickets/{ticket_identifier}","title":"Get Support Ticket (with message thread).","href":"/api-reference/support/get-support-ticket"},{"openapi":"patch /v1/support/tickets/{ticket_identifier}","title":"Update a support ticket (status/priority).","href":"/api-reference/support/update-support-ticket"},{"openapi":"post /v1/support/tickets/{ticket_identifier}/messages","title":"Reply to a support ticket.","href":"/api-reference/support/reply-support-ticket"},{"openapi":"post /v1/support/comms/list","title":"List correspondence (ticket messages + broadcasts, org + global).","href":"/api-reference/support/list-correspondence"},{"openapi":"get /v1/support/comms","title":"List Correspondence","href":"/api-reference/support/list-correspondence-get"},{"openapi":"post /v1/support/comms/broadcast","title":"Create a broadcast communication (Mixpeek admin only).","href":"/api-reference/support/create-broadcast-communication"},{"openapi":"post /v1/support/videos/upload-url","title":"Presigned PUT to upload a support video to cold storage (admin only).","href":"/api-reference/support/presigned-put-upload-support-video-cold"},{"openapi":"post /v1/support/videos","title":"Register a support video (admin only); triggers Gemini review for uploads.","href":"/api-reference/support/register-support-video-triggers-gemini-review"},{"openapi":"get /v1/support/videos","title":"List Support Videos","href":"/api-reference/support/list-support-videos"},{"openapi":"post /v1/support/videos/list","title":"List curated support videos (org + global).","href":"/api-reference/support/list-curated-support-videos"},{"openapi":"get /v1/support/videos/{video_identifier}","title":"Get a support video (with presigned playback URL + chapters).","href":"/api-reference/support/get-support-video"}]}]},{"group":"Advanced","pages":[{"group":"Apps","pages":[{"openapi":"post /v1/apps","title":"Create an App","href":"/api-reference/apps/create-app"},{"openapi":"get /v1/apps","title":"List Apps","href":"/api-reference/apps/list-apps"},{"openapi":"get /v1/apps/{app_id}","title":"Get an App","href":"/api-reference/apps/get-app"},{"openapi":"patch /v1/apps/{app_id}","title":"Update an App","href":"/api-reference/apps/update-app"},{"openapi":"delete /v1/apps/{app_id}","title":"Delete an App","href":"/api-reference/apps/delete-app"},{"openapi":"post /v1/apps/{app_id}/publish","title":"Publish an App","href":"/api-reference/apps/publish-app"},{"openapi":"post /v1/apps/{app_id}/rollback","title":"Rollback an App","href":"/api-reference/apps/rollback-app"},{"group":"Deploy","pages":[{"openapi":"post /v1/apps/{app_id}/deploy","title":"Deploy an App","href":"/api-reference/apps/deploy-app"},{"openapi":"post /v1/apps/{app_id}/deploy/upload-url","title":"Get presigned upload URL","href":"/api-reference/apps/get-deploy-upload-url"},{"title":"Get Deploy Status","openapi":"GET /v1/apps/{app_id}/deploys/{deploy_id}","href":"/api-reference/apps/get-deploy-status"},{"openapi":"get /v1/apps/{app_id}/deploys/{deploy_id}/logs/stream","title":"Stream deploy logs (SSE)","href":"/api-reference/apps/stream-deploy-logs-sse"},{"openapi":"post /v1/apps/{app_id}/deploy/verify-upload","title":"Verify a presigned-URL upload landed in S3","href":"/api-reference/apps/verify-a-presigned-url-upload-landed-in-s3"},{"openapi":"post /v1/apps/{app_id}/promote","title":"Promote staging to production","href":"/api-reference/apps/promote-staging-to-production"}]},{"group":"GitHub","pages":[{"openapi":"post /v1/apps/{app_id}/connect-repo","title":"Connect a GitHub repository","href":"/api-reference/apps/connect-a-github-repository"},{"openapi":"delete /v1/apps/{app_id}/connect-repo","title":"Disconnect GitHub repository","href":"/api-reference/apps/disconnect-github-repository"}]},{"group":"Users","pages":[{"openapi":"get /v1/apps/{app_id}/users","title":"List app users","href":"/api-reference/apps/list-app-users"},{"openapi":"post /v1/apps/{app_id}/users/invite","title":"Invite a user","href":"/api-reference/apps/invite-a-user"},{"openapi":"get /v1/apps/{app_id}/users/invitations","title":"List pending invitations","href":"/api-reference/apps/list-pending-invitations"},{"openapi":"patch /v1/apps/{app_id}/users/{user_id}/role","title":"Update user role","href":"/api-reference/apps/update-user-role"},{"openapi":"delete /v1/apps/{app_id}/users/{user_id}","title":"Remove a user","href":"/api-reference/apps/remove-a-user"},{"openapi":"delete /v1/apps/{app_id}/users/invitations/{invitation_id}","title":"Revoke an invitation","href":"/api-reference/apps/revoke-an-invitation"}]},{"group":"Versions","pages":[{"openapi":"get /v1/apps/{app_id}/versions","title":"List version history","href":"/api-reference/apps/list-versions"},{"title":"Get Version Details","openapi":"GET /v1/apps/{app_id}/versions/{version}","href":"/api-reference/apps/get-version"},{"title":"Diff Two Versions","openapi":"GET /v1/apps/{app_id}/versions/{from_version}/diff/{to_version}","href":"/api-reference/apps/diff-versions"},{"title":"Download Version Bundle","openapi":"GET /v1/apps/{app_id}/versions/{version}/download","href":"/api-reference/apps/download-version"},{"title":"Restore a Version","openapi":"POST /v1/apps/{app_id}/versions/{version}/restore","href":"/api-reference/apps/restore-version"}]},{"group":"Domains","pages":[{"openapi":"post /v1/apps/{app_id}/domains","title":"Add a custom domain","href":"/api-reference/apps/add-domain"},{"openapi":"get /v1/apps/{app_id}/domains","title":"List custom domains","href":"/api-reference/apps/list-domains"},{"openapi":"delete /v1/apps/{app_id}/domains/{domain}","title":"Remove a domain","href":"/api-reference/apps/delete-domain"},{"openapi":"post /v1/apps/{app_id}/domains/{domain}/verify","title":"Trigger DNS verification","href":"/api-reference/apps/verify-domain"}]},{"group":"Analytics","pages":[{"openapi":"get /v1/apps/{app_id}/analytics","title":"Get app analytics overview","href":"/api-reference/apps/get-app-analytics-overview"},{"openapi":"get /v1/apps/{app_id}/error-rate","title":"Get recent error rate","href":"/api-reference/apps/get-recent-error-rate"},{"openapi":"get /v1/apps/{app_id}/logs","title":"Get runtime or request logs","href":"/api-reference/apps/get-runtime-or-request-logs"}]},{"group":"Public","pages":[{"openapi":"get /v1/public/apps/{slug}/config","title":"Get public App config","href":"/api-reference/public-apps-api/get-app-config"},{"openapi":"post /v1/public/apps/{slug}/search","title":"Execute an App tab search","href":"/api-reference/public-apps-api/search-app"},{"openapi":"get /v1/public/apps/{slug}/gallery","title":"Get App featured gallery","href":"/api-reference/public-apps-api/get-app-gallery"},{"openapi":"get /v1/public/apps/{slug}/taxonomies/{taxonomy_id}/tree","title":"Get taxonomy node tree for an App","href":"/api-reference/public-apps-api/get-app-taxonomies"},{"openapi":"post /v1/public/apps/{slug}/interactions","title":"Track an App interaction event","href":"/api-reference/public-apps-api/track-app-interaction"}]},{"openapi":"get /v1/apps/{app_id}/github/install-url","title":"Get the GitHub App install URL for this app","href":"/api-reference/apps/get-github-app-install-url-app"}]},{"group":"Agent Sessions","pages":[{"openapi":"post /v1/agents/sessions","title":"Create Session","href":"/api-reference/agent-sessions/create-session"},{"openapi":"post /v1/agents/sessions/list","title":"List Sessions","href":"/api-reference/agent-sessions/list-sessions"},{"openapi":"get /v1/agents/sessions/{session_id}","title":"Get Session","href":"/api-reference/agent-sessions/get-session"},{"openapi":"patch /v1/agents/sessions/{session_id}","title":"Patch Session","href":"/api-reference/agent-sessions/patch-session"},{"openapi":"post /v1/agents/sessions/{session_id}/messages","title":"Send Message","href":"/api-reference/agent-sessions/send-message"},{"openapi":"get /v1/agents/sessions/{session_id}/history","title":"Get History","href":"/api-reference/agent-sessions/get-history"},{"openapi":"get /v1/agents/sessions/tools","title":"List Tools","href":"/api-reference/agent-sessions/list-tools"},{"openapi":"post /v1/agents/sessions/intent/detect","title":"Detect Intent","href":"/api-reference/agent-sessions/detect-intent"},{"openapi":"post /v1/agents/sessions/{session_id}/confirmations/{confirmation_id}","title":"Respond To Confirmation","href":"/api-reference/agent-sessions/respond-to-confirmation"},{"openapi":"post /v1/agents/sessions/{session_id}/feedback","title":"Submit Feedback","href":"/api-reference/agent-sessions/submit-feedback"},{"openapi":"delete /v1/agents/sessions/{session_id}","title":"Terminate Session","href":"/api-reference/agent-sessions/terminate-session"}]},{"group":"Templates","pages":[{"group":"Namespace","pages":[{"openapi":"get /v1/templates/namespaces","title":"List Namespace Templates","href":"/api-reference/templates/list-namespace-templates"},{"openapi":"get /v1/templates/namespaces/{template_id}","title":"Get Namespace Template","href":"/api-reference/templates/get-namespace-template"},{"openapi":"post /v1/templates/namespaces/from-namespace/{namespace_id}","title":"Create Namespace Template","href":"/api-reference/templates/create-namespace-template"}]},{"group":"Retriever","pages":[{"openapi":"post /v1/templates/retrievers","title":"List Retriever Templates","href":"/api-reference/templates/list-retriever-templates"},{"openapi":"get /v1/templates/retrievers/{template_id}","title":"Get Retriever Template","href":"/api-reference/templates/get-retriever-template"},{"openapi":"post /v1/templates/retrievers/from-retriever/{retriever_id}","title":"Create Retriever Template","href":"/api-reference/templates/create-retriever-template"},{"openapi":"post /v1/templates/retrievers/{template_id}/instantiate","title":"Instantiate Retriever Template","href":"/api-reference/templates/instantiate-retriever-template"}]},{"group":"Collection","pages":[{"openapi":"post /v1/templates/collections","title":"List Collection Templates","href":"/api-reference/templates/list-collection-templates"},{"openapi":"get /v1/templates/collections/{template_id}","title":"Get Collection Template","href":"/api-reference/templates/get-collection-template"},{"openapi":"post /v1/templates/collections/from-collection/{collection_id}","title":"Create Collection Template","href":"/api-reference/templates/create-collection-template"},{"openapi":"post /v1/templates/collections/{template_id}/instantiate","title":"Instantiate Collection Template","href":"/api-reference/templates/instantiate-collection-template"}]},{"group":"Bucket","pages":[{"openapi":"post /v1/templates/buckets","title":"List Bucket Templates","href":"/api-reference/templates/list-bucket-templates"},{"openapi":"get /v1/templates/buckets/{template_id}","title":"Get Bucket Template","href":"/api-reference/templates/get-bucket-template"},{"openapi":"post /v1/templates/buckets/from-bucket/{bucket_id}","title":"Create Bucket Template","href":"/api-reference/templates/create-bucket-template"},{"openapi":"post /v1/templates/buckets/{template_id}/instantiate","title":"Instantiate Bucket Template","href":"/api-reference/templates/instantiate-bucket-template"}]},{"group":"Cluster","pages":[{"openapi":"post /v1/templates/clusters","title":"List Cluster Templates","href":"/api-reference/templates/list-cluster-templates"},{"openapi":"get /v1/templates/clusters/{template_id}","title":"Get Cluster Template","href":"/api-reference/templates/get-cluster-template"},{"openapi":"post /v1/templates/clusters/from-cluster/{cluster_id}","title":"Create Cluster Template","href":"/api-reference/templates/create-cluster-template"},{"openapi":"post /v1/templates/clusters/{template_id}/instantiate","title":"Instantiate Cluster Template","href":"/api-reference/templates/instantiate-cluster-template"}]},{"group":"Taxonomy","pages":[{"openapi":"post /v1/templates/taxonomies","title":"List Taxonomy Templates","href":"/api-reference/templates/list-taxonomy-templates"},{"openapi":"get /v1/templates/taxonomies/{template_id}","title":"Get Taxonomy Template","href":"/api-reference/templates/get-taxonomy-template"},{"openapi":"post /v1/templates/taxonomies/from-taxonomy/{taxonomy_id}","title":"Create Taxonomy Template","href":"/api-reference/templates/create-taxonomy-template"},{"openapi":"post /v1/templates/taxonomies/{template_id}/instantiate","title":"Instantiate Taxonomy Template","href":"/api-reference/templates/instantiate-taxonomy-template"}]},{"group":"Public Templates","pages":[{"openapi":"get /v1/public/templates/namespaces","title":"List Public Namespace Templates","href":"/api-reference/public-templates-api/list-public-namespace-templates"},{"openapi":"get /v1/public/templates/namespaces/{template_id}","title":"Get Public Namespace Template","href":"/api-reference/public-templates-api/get-public-namespace-template"},{"openapi":"get /v1/public/templates/retrievers","title":"List Public Retriever Templates","href":"/api-reference/public-templates-api/list-public-retriever-templates"},{"openapi":"get /v1/public/templates/retrievers/{template_id}","title":"Get Public Retriever Template","href":"/api-reference/public-templates-api/get-public-retriever-template"},{"openapi":"get /v1/public/templates/scaffolds","title":"List Scaffolds","href":"/api-reference/public-templates-api/list-scaffolds"},{"openapi":"get /v1/public/templates/scaffolds/{template_id}","title":"Get Scaffold","href":"/api-reference/public-templates-api/get-scaffold"},{"openapi":"get /v1/public/templates/scaffolds/{template_id}/sample-data","title":"Get Scaffold Sample Data","href":"/api-reference/public-templates-api/get-scaffold-sample-data"},{"openapi":"get /v1/public/templates/scaffolds/{template_id}/manifest","title":"Get Scaffold Manifest","href":"/api-reference/public-templates-api/get-scaffold-manifest"}]},{"group":"Scaffold Templates","pages":[{"openapi":"post /v1/templates/scaffolds/default","title":"Get Or Create Default Namespace","href":"/api-reference/scaffold-templates/get-or-create-default-namespace"},{"openapi":"get /v1/templates/scaffolds/{template_id}/manifest","title":"Get Scaffold Template Manifest","href":"/api-reference/scaffold-templates/get-scaffold-template-manifest"}]}]},{"group":"Manifest","pages":[{"openapi":"post /v1/manifest/apply","title":"Apply Manifest","href":"/api-reference/manifest/apply-manifest"},{"openapi":"post /v1/manifest/validate","title":"Validate Manifest","href":"/api-reference/manifest/validate-manifest"},{"openapi":"get /v1/manifest/export","title":"Export Manifest Get","href":"/api-reference/manifest/export-manifest"},{"openapi":"post /v1/manifest/diff","title":"Diff Manifest","href":"/api-reference/manifest/diff-manifest"},{"openapi":"post /v1/manifest/generate","title":"Generate Manifest","href":"/api-reference/manifest/generate-manifest"},{"openapi":"post /v1/manifest/lint","title":"Lint Manifest","href":"/api-reference/manifest/lint-manifest"}]},{"group":"Analytics","pages":[{"group":"Namespace","pages":[{"openapi":"get /v1/analytics/namespaces/summary","title":"Get Namespace Summary","href":"/api-reference/analytics/get-namespace-summary"},{"openapi":"get /v1/analytics/indexes/usage","title":"Get Field Usage","href":"/api-reference/analytics/get-field-usage"},{"openapi":"get /v1/analytics/namespaces/fields/performance","title":"Get Field Performance","href":"/api-reference/analytics/get-field-performance"},{"openapi":"get /v1/analytics/namespaces/fields/most-queried","title":"Get Most Queried Fields","href":"/api-reference/analytics/get-most-queried-fields"},{"openapi":"get /v1/analytics/namespaces/indexes/compound-patterns","title":"Get Compound Index Patterns","href":"/api-reference/analytics/get-compound-index-patterns"},{"openapi":"get /v1/analytics/namespaces/indexes/recommendations","title":"Get Index Recommendations","href":"/api-reference/analytics/get-index-recommendations"},{"openapi":"get /v1/analytics/indexes/suggestions","title":"Get Index Suggestions","href":"/api-reference/analytics/get-index-suggestions"},{"openapi":"get /v1/analytics/namespaces/errors","title":"Get Namespace Errors","href":"/api-reference/analytics/get-namespace-errors"}]},{"group":"Buckets","pages":[{"openapi":"get /v1/analytics/buckets/{bucket_id}/health","title":"Get Bucket Health","href":"/api-reference/analytics/get-bucket-health"},{"openapi":"get /v1/analytics/buckets/health","title":"Get Namespace Buckets Health","href":"/api-reference/analytics/get-namespace-buckets-health"},{"openapi":"get /v1/analytics/buckets/{bucket_id}/storage","title":"Get Bucket Storage","href":"/api-reference/analytics/get-bucket-storage"},{"openapi":"get /v1/analytics/buckets/{bucket_id}/usage","title":"Get Bucket Usage","href":"/api-reference/analytics/get-bucket-usage"},{"openapi":"get /v1/analytics/buckets/{bucket_id}/sync-performance","title":"Get Sync Performance","href":"/api-reference/analytics/get-sync-performance"},{"openapi":"get /v1/analytics/buckets/{bucket_id}/sync-comparison","title":"Get Sync Comparison","href":"/api-reference/analytics/get-sync-comparison"},{"openapi":"get /v1/analytics/buckets/{bucket_id}/upload-performance","title":"Get Upload Performance","href":"/api-reference/analytics/get-upload-performance"}]},{"group":"Collections","pages":[{"openapi":"get /v1/analytics/collections/{collection_id}/overview","title":"Get Collection Overview","href":"/api-reference/analytics/get-collection-overview"},{"openapi":"get /v1/analytics/collections/{collection_id}/growth","title":"Get Document Growth","href":"/api-reference/analytics/get-document-growth"},{"openapi":"get /v1/analytics/collections/{collection_id}/latency","title":"Get Latency Metrics","href":"/api-reference/analytics/get-latency-metrics"},{"openapi":"get /v1/analytics/collections/{collection_id}/failures","title":"Get Failure Analysis","href":"/api-reference/analytics/get-failure-analysis"},{"openapi":"get /v1/analytics/extractors/performance","title":"Get Extractor Performance","href":"/api-reference/analytics/get-extractor-performance"}]},{"group":"Retrievers","pages":[{"openapi":"get /v1/analytics/retrievers/{retriever_id}/performance","title":"Get Retriever Performance","href":"/api-reference/analytics/get-retriever-performance"},{"openapi":"get /v1/analytics/retrievers/{retriever_id}/signals","title":"Get Retriever Signals","href":"/api-reference/analytics/get-retriever-signals"},{"openapi":"get /v1/analytics/namespaces/queries/slow","title":"Get Slow Queries","href":"/api-reference/analytics/get-slow-queries"},{"openapi":"get /v1/analytics/retrievers/{retriever_id}/stages","title":"Get Stage Breakdown","href":"/api-reference/analytics/get-stage-breakdown"},{"openapi":"get /v1/analytics/retrievers/{retriever_id}/cache-performance","title":"Get Cache Performance","href":"/api-reference/analytics/get-cache-performance"},{"openapi":"post /v1/analytics/retrievers/{retriever_id}/analyze-tuning","title":"Analyze For Tuning","href":"/api-reference/analytics/analyze-for-tuning"}]},{"group":"Engine","pages":[{"openapi":"get /v1/analytics/performance/engine","title":"Get Engine Performance","href":"/api-reference/analytics/get-engine-performance"},{"openapi":"get /v1/analytics/performance/engine/bottlenecks","title":"Analyze Bottlenecks","href":"/api-reference/analytics/analyze-bottlenecks"},{"openapi":"get /v1/analytics/performance/engine/extractors","title":"Get Extractor Breakdown","href":"/api-reference/analytics/get-extractor-breakdown"},{"openapi":"get /v1/analytics/performance/engine/stages","title":"Get Engine Stage Breakdown","href":"/api-reference/analytics/get-engine-stage-breakdown"},{"openapi":"get /v1/analytics/performance/engine/slow-operations","title":"Get Slowest Operations","href":"/api-reference/analytics/get-slowest-operations"}]},{"group":"Taxonomies","pages":[{"openapi":"get /v1/analytics/taxonomies/{taxonomy_id}/assignments","title":"Get Assignment Metrics","href":"/api-reference/analytics/get-assignment-metrics"},{"openapi":"get /v1/analytics/taxonomies/{taxonomy_id}/confidence","title":"Get Confidence Distribution","href":"/api-reference/analytics/get-confidence-distribution"},{"openapi":"get /v1/analytics/taxonomies/{taxonomy_id}/labels","title":"Get Label Distribution","href":"/api-reference/analytics/get-label-distribution"},{"openapi":"get /v1/analytics/taxonomies/{taxonomy_id}/enrichments","title":"Get Enrichment History","href":"/api-reference/analytics/get-enrichment-history"}]},{"group":"Clusters","pages":[{"openapi":"get /v1/analytics/clusters/{cluster_id}/execution-history","title":"Get Execution History","href":"/api-reference/analytics/get-execution-history"},{"openapi":"get /v1/analytics/clusters/{cluster_id}/failures","title":"Get Failure Analysis","href":"/api-reference/analytics/get-failure-analysis-1"}]},{"group":"Inference \u0026 Extractors","pages":[{"openapi":"get /v1/analytics/collections/{collection_id}/extractors","title":"Get Extractor Performance","href":"/api-reference/analytics/get-extractor-performance-1"}]},{"group":"Diagnostics","pages":[{"openapi":"get /v1/analytics/performance/batches/{batch_id}/diagnostics","title":"Get Batch Diagnostics","href":"/api-reference/analytics/get-batch-diagnostics"},{"openapi":"get /v1/analytics/usage/summary","title":"Get Usage Summary","href":"/api-reference/analytics/get-usage-summary"},{"openapi":"get /v1/analytics/retrievers/{retriever_id}/slow-queries","title":"Get Slowest Queries","href":"/api-reference/analytics/get-slowest-queries"}]}]},{"group":"Discovery","pages":[{"openapi":"get /v1/discovery","title":"Get All Discovery Information","href":"/api-reference/discovery/get-all-discovery-information"},{"openapi":"get /v1/discovery/extractors","title":"List Available Feature Extractors","href":"/api-reference/discovery/list-available-feature-extractors"},{"openapi":"get /v1/discovery/stages","title":"List Available Retriever Stages with Examples","href":"/api-reference/discovery/list-available-retriever-stages-with-examples"},{"openapi":"get /v1/discovery/schema","title":"Get Manifest Schema for Agent Configuration","href":"/api-reference/discovery/get-manifest-schema-for-agent-configuration"}]},{"group":"Inference","pages":[{"openapi":"post /v1/inference","title":"Execute Raw Inference","href":"/api-reference/inference/execute-raw-inference"}]},{"group":"Events","pages":[{"openapi":"get /v1/events","title":"List change feed events","href":"/api-reference/events/list-events"}]},{"group":"Resource Search","pages":[{"openapi":"post /v1/resources/search","title":"Search Resources","href":"/api-reference/resource-search/search-resources"}]},{"group":"Onboarding","pages":[{"openapi":"post /v1/onboarding/infer-config","title":"Infer onboarding configuration","href":"/api-reference/onboarding/infer-onboarding-configuration"},{"openapi":"get /v1/onboarding/suggested-config","title":"Get the org's signup-time suggested namespace","href":"/api-reference/onboarding/get-suggested-config"},{"openapi":"post /v1/signup/agent","title":"Sign up as an agent","href":"/api-reference/onboarding/agent-signup"}]}]}]}]}]},"basePath":"","title":"Partially Update Batch - Mixpeek","buildId":"dpl_BSPUBxEoKhhK7HzXVDPxdPvWx4xc","deploymentId":"dpl_BSPUBxEoKhhK7HzXVDPxdPvWx4xc","dynamics":["c2aa737661","2b790eda92","b0f84f53ff","a9bb46feb3"],"chunks":["OpenApiReferenceProvider-Do6EtGSl.js","utils-id4tQ6My.js","cloneDeep-izs3Hzxb.js","_Uint8Array-BVIqJ90l.js","isObjectLike-C8vmzPEN.js","isArray-Dcu1dQ5t.js","utils-Byrixac4.js","toString-BX-iyh8G.js","isSymbol-CSH9U2Gm.js","addUniqueKeysToSchema-nakYa3Zl.js","snippetPresets-Bq-pUS7h.js","style-to-object-BY3JEAdk.js","v4-DIgz8zy6.js","conditional-requirements-CFM0id0Q.js","ApiExamples-CujTC_9K.js","isEqual-BfaiBbcl.js","usePlaygroundInputsStore-FneVUsVu.js","shallow-DY9WpUCz.js","getApiPlaygroundDisplay-Bz8WpPfA.js","CodeSnippet-DgCuJg4G.js","CodeSnippets-Dub9c4T0.js","Page-CePXnjd1.js","getPastedJson-xdI8y2XB.js","icon--yVw5vkr.js","MethodPill-C47t7MdS.js","DuplicateIcon-BzwJ2LeC.js","createIcon-BzggGy84.js","DynamicMarkdown-AlxJK8VR.js","SectionHeading-CMlwFxdK.js","axios-vUo_bRtp.js","Markdown-BNgNrPsi.js","clsx-apD-Q0Oo.js","lib-DuNMoZ05.js","lib-B8HbPgcC.js","html-void-elements-mCSPHezU.js","dist-qmLwQyWY.js","katex-YMM-FMeD.js","marked.esm-CZWythHk.js","AssistantPageLink-Dc4qTkWx.js"]}