> ## Documentation Index > Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Export Collection > Export collection documents to JSON, CSV, Parquet or WebDataset format. **Export Formats:** - **JSON**: Line-delimited JSON (JSONL) format. Good for streaming. - **CSV**: Comma-separated values. Best for spreadsheets. - **PARQUET**: Columnar format (default). Best for data pipelines. - **WEBDATASET**: Tar shards for training loaders. One sample per document, carrying `.json` and, with `include_media=True`, the source object's bytes. The response returns every shard plus the brace pattern loaders take. **Vector Export:** Vectors are large and exported separately. When `include_vectors=True`, a separate file is created for vectors with document_id mapping. **Field Selection:** Use `select_fields` to export only specific fields, reducing file size. **Lineage:** Set `include_lineage=True` to add `lineage_chain`, `source_content_hash` and `document_created_at` to every row, so each exported document traces back to its source object and processing steps. **Filtering:** Apply filters to export a subset of documents. **Destination (your own storage):** Set `destination` to a storage connection you own plus a prefix, and the files and a `manifest.json` are written there as well as to the download copy. The connection needs `write_enabled` set, which is off by default, and its credentials need the write permission on your side (`s3:PutObject`, or `storage.objects.create` for GCS). A connection without it is refused with the permission named. **Response:** Returns presigned download URLs valid for 1 hour. **Limits:** - Large exports may take time. Consider using `sample_size` for testing. - Vector exports significantly increase processing time. ## OpenAPI ````yaml post /v1/collections/{collection_identifier}/export openapi: 3.1.0 info: title: Mixpeek API description: >- This is the Mixpeek API, providing access to various endpoints for data processing and retrieval. termsOfService: https://mixpeek.com/terms contact: name: Mixpeek Support url: https://mixpeek.com/contact email: info@mixpeek.com version: '0.82' servers: - url: https://api.mixpeek.com description: Production security: - BearerAuth: [] paths: /v1/collections/{collection_identifier}/export: post: tags: - Collections summary: Export Collection description: |- Export collection documents to JSON, CSV, Parquet or WebDataset format. **Export Formats:** - **JSON**: Line-delimited JSON (JSONL) format. Good for streaming. - **CSV**: Comma-separated values. Best for spreadsheets. - **PARQUET**: Columnar format (default). Best for data pipelines. - **WEBDATASET**: Tar shards for training loaders. One sample per document, carrying `.json` and, with `include_media=True`, the source object's bytes. The response returns every shard plus the brace pattern loaders take. **Vector Export:** Vectors are large and exported separately. When `include_vectors=True`, a separate file is created for vectors with document_id mapping. **Field Selection:** Use `select_fields` to export only specific fields, reducing file size. **Lineage:** Set `include_lineage=True` to add `lineage_chain`, `source_content_hash` and `document_created_at` to every row, so each exported document traces back to its source object and processing steps. **Filtering:** Apply filters to export a subset of documents. **Destination (your own storage):** Set `destination` to a storage connection you own plus a prefix, and the files and a `manifest.json` are written there as well as to the download copy. The connection needs `write_enabled` set, which is off by default, and its credentials need the write permission on your side (`s3:PutObject`, or `storage.objects.create` for GCS). A connection without it is refused with the permission named. **Response:** Returns presigned download URLs valid for 1 hour. **Limits:** - Large exports may take time. Consider using `sample_size` for testing. - Vector exports significantly increase processing time. operationId: export_collection_v1_collections__collection_identifier__export_post parameters: - name: collection_identifier in: path required: true schema: type: string description: The ID or name of the collection to export title: Collection Identifier description: The ID or name of the collection to export requestBody: content: application/json: schema: $ref: '#/components/schemas/CollectionExportRequest' description: >- Keys this body does not declare are refused with 422 extra_forbidden naming the accepted keys (since 2026-11-02). x-unknown-keys: until: '2026-11-02' then: 422 extra_forbidden responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/CollectionExportResponse' '400': description: Bad Request content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '401': description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '403': description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '404': description: Not Found content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '422': description: Unprocessable Entity content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '429': description: Too Many Requests content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '500': description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '503': description: Service Unavailable content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' security: - BearerAuth: [] NamespaceHeader: [] components: schemas: CollectionExportRequest: properties: format: $ref: '#/components/schemas/ExportFormat' description: 'Export format: json (line-delimited), csv, or parquet (default).' default: parquet include_vectors: type: boolean title: Include Vectors description: >- Whether to include vectors in the export. Vectors are exported to a separate file due to their large size. This significantly increases export time and file size. default: false include_lineage: type: boolean title: Include Lineage description: >- Add lineage columns to every row: lineage_chain (the processing steps from source object to document), source_content_hash (SHA-256 of the source content) and document_created_at (ISO 8601). Kept even when select_fields is set. In CSV and Parquet, lineage_chain is a JSON-encoded string. default: false select_fields: anyOf: - items: type: string type: array - type: 'null' title: Select Fields description: >- Specific fields to include in the export. If not provided, all fields are exported. Supports dot notation for nested fields (e.g., 'metadata.title', 'metadata.author'). examples: - - document_id - metadata.title - metadata.category filters: anyOf: - $ref: '#/components/schemas/LogicalOperator-Input' - type: 'null' description: >- Filter conditions to export only matching documents. Uses LogicalOperator format (AND/OR/NOT) same as document listing. sample_size: anyOf: - type: integer maximum: 1000000 minimum: 1 - type: 'null' title: Sample Size description: >- Maximum number of documents to export. If not provided, exports all documents. Useful for testing exports or creating sample datasets. include_media: type: boolean title: Include Media description: >- WebDataset only. Put the source object's bytes in each sample alongside its row, resolved through the document's root object. A document whose object is gone, or which never had one, ships with its row alone and is counted in the response's media summary. This moves the full media set, so expect a much larger export and a longer run. default: false samples_per_shard: type: integer maximum: 100000 minimum: 1 title: Samples Per Shard description: >- WebDataset only. Documents per tar shard. Smaller shards parallelize better across training workers; larger shards mean fewer files to move. default: 1000 destination: anyOf: - $ref: '#/components/schemas/ExportDestination' - type: 'null' description: >- Write the export into your own storage through a connection you own. The files and a manifest land under the prefix you give. The 7-day download copy is still written too, so the presigned URLs in the response keep working and nothing existing changes. type: object title: CollectionExportRequest description: >- Request model for exporting collection data. **Export Formats:** - **JSON**: Line-delimited JSON (JSONL) format, one document per line. Good for streaming and large files. - **CSV**: Comma-separated values. Best for tabular data analysis in spreadsheets. - **PARQUET**: Columnar format optimized for analytics. Best for large datasets and data pipelines. - **WEBDATASET**: Tar shards in the WebDataset layout, one sample per document. Each sample carries `.json` (the same row the other formats emit) and, with `include_media=True`, the source object's bytes as `.`. Read it with any WebDataset loader. `samples_per_shard` sets the shard size. **Vector Export:** Vectors are stored separately from document metadata due to their large size. When `include_vectors=True`, vectors are exported to a separate file with the naming convention: `{collection_name}_vectors.{format}` **Lineage:** When `include_lineage=True`, every row also carries `lineage_chain` (each processing step from the source object to this document), `source_content_hash` (SHA-256 of the source content) and `document_created_at` (ISO 8601). In JSON the chain is a list; in CSV and Parquet it is a JSON-encoded string so every row keeps one column type. **Field Selection:** Use `select_fields` to export only specific fields, reducing file size for large collections. Supports dot notation for nested fields (e.g., "metadata.title"). **Filtering:** Apply filters to export a subset of documents. Uses the same LogicalOperator format as the documents list endpoint. CollectionExportResponse: properties: download_url: type: string title: Download Url description: Presigned URL for downloading the exported file. Valid for 1 hour. s3_path: type: string title: S3 Path description: >- Full S3 path where the export is stored (for internal reference). For webdataset this is the prefix holding the shards, not a single file. format: $ref: '#/components/schemas/ExportFormat' description: The format of the exported file. document_count: type: integer minimum: 0 title: Document Count description: Number of documents included in the export. file_size_bytes: type: integer minimum: 0 title: File Size Bytes description: >- Size of the exported file in bytes. For webdataset this is every shard added together. exported_at: type: string format: date-time title: Exported At description: Timestamp when the export was completed. vectors_download_url: anyOf: - type: string - type: 'null' title: Vectors Download Url description: >- Presigned URL for downloading the vectors file (if include_vectors=True). Vectors are exported separately due to their large size. vectors_s3_path: anyOf: - type: string - type: 'null' title: Vectors S3 Path description: Full S3 path for the vectors file (if include_vectors=True). shard_count: anyOf: - type: integer minimum: 0 - type: 'null' title: Shard Count description: >- WebDataset only. Number of tar shards written. Always exact, even when the shards list below is capped. shard_pattern: anyOf: - type: string - type: 'null' title: Shard Pattern description: >- WebDataset only. Brace pattern naming every shard, e.g. documents-{000000..000007}.tar. Most loaders take this directly. shards: anyOf: - items: $ref: '#/components/schemas/ExportShard' type: array - type: 'null' title: Shards description: >- WebDataset only. Per-shard paths and presigned URLs, capped at 200 entries. When shard_count exceeds that, address the rest through s3_path and shard_pattern. media: anyOf: - $ref: '#/components/schemas/ExportMediaSummary' - type: 'null' description: >- WebDataset only, present when include_media was set. Counts what media reached the shards and what did not. destination: anyOf: - $ref: '#/components/schemas/ExportDestinationResult' - type: 'null' description: >- Present when the request named a destination. Lists what was written into your storage and where the manifest is. type: object required: - download_url - s3_path - format - document_count - file_size_bytes - exported_at title: CollectionExportResponse description: |- Response model for collection export. Contains the presigned URL for downloading the exported file. The URL is valid for a limited time (typically 1 hour). examples: - document_count: 10000 download_url: >- https://s3.amazonaws.com/bucket/org_xxx/ns_xxx/api_collections_export/col_xxx/v1/documents.parquet?... exported_at: '2024-01-15T10:30:00Z' file_size_bytes: 5242880 format: parquet s3_path: >- s3://bucket/org_xxx/ns_xxx/api_collections_export/col_xxx/v1/documents.parquet ErrorResponse: properties: success: type: boolean title: Success description: Always false for error responses default: false status: type: integer title: Status description: HTTP status code for this error error: $ref: '#/components/schemas/ErrorDetail' description: Error details payload type: object required: - status - error title: ErrorResponse description: Error response model. examples: - error: details: id: ns_123 resource: namespace message: Namespace not found type: NotFoundError status: 404 success: false ExportFormat: type: string enum: - json - csv - parquet - webdataset title: ExportFormat description: Supported export formats for collection data. LogicalOperator-Input: properties: AND: anyOf: - items: anyOf: - $ref: '#/components/schemas/LogicalOperator-Input' - $ref: '#/components/schemas/FilterCondition' type: array - type: 'null' title: And description: Logical AND operation - all conditions must be true example: - field: name operator: eq value: John - field: age operator: gte value: 30 OR: anyOf: - items: anyOf: - $ref: '#/components/schemas/LogicalOperator-Input' - $ref: '#/components/schemas/FilterCondition' type: array - type: 'null' title: Or description: Logical OR operation - at least one condition must be true example: - field: status operator: eq value: active - field: role operator: eq value: admin NOT: anyOf: - items: anyOf: - $ref: '#/components/schemas/LogicalOperator-Input' - $ref: '#/components/schemas/FilterCondition' type: array - type: 'null' title: Not description: Logical NOT operation - all conditions must be false example: - field: department operator: eq value: HR - field: location operator: eq value: remote case_sensitive: anyOf: - type: boolean - type: 'null' title: Case Sensitive description: Whether to perform case-sensitive matching default: false example: true additionalProperties: true type: object title: LogicalOperator description: >- Represents a logical operation (AND, OR, NOT) on filter conditions. Allows nesting with a defined depth limit. Also supports shorthand syntax where field names can be passed directly as key-value pairs for equality filtering (e.g., {"metadata.title": "value"}). ExportDestination: properties: connection_id: type: string title: Connection Id description: >- A storage connection you own, with write access enabled. Enable it with PATCH /v1/organizations/connections/{connection_id} and {"write_enabled": true}. prefix: type: string title: Prefix description: >- Key prefix inside the connection's bucket to write under, for example 'releases/2026-09-20/'. Empty writes at the root of the bucket. A trailing slash is added if you omit it. default: '' type: object required: - connection_id title: ExportDestination description: >- Where to write an export, when the caller wants it in their own storage. The connection must have write access enabled. That is opt-in per connection and checked only when the connection is named here, so every read-only connection keeps working exactly as it does today. ExportShard: properties: filename: type: string title: Filename description: Shard file name, e.g. documents-000000.tar. s3_path: type: string title: S3 Path description: Full S3 path to this shard. download_url: type: string title: Download Url description: Presigned URL for this shard. Valid for 1 hour. sample_count: type: integer minimum: 0 title: Sample Count description: Documents in this shard. file_size_bytes: type: integer minimum: 0 title: File Size Bytes description: Size of this shard in bytes. type: object required: - filename - s3_path - download_url - sample_count - file_size_bytes title: ExportShard description: One tar shard of a WebDataset export. ExportMediaSummary: properties: samples_with_media: type: integer minimum: 0 title: Samples With Media samples_without_media: type: integer minimum: 0 title: Samples Without Media members_written: type: integer minimum: 0 title: Members Written members_skipped_too_large: type: integer minimum: 0 title: Members Skipped Too Large description: Blobs over the per-member size limit. objects_not_found: type: integer minimum: 0 title: Objects Not Found description: Documents whose source object no longer exists. download_failures: type: integer minimum: 0 title: Download Failures media_bytes: type: integer minimum: 0 title: Media Bytes type: object required: - samples_with_media - samples_without_media - members_written - members_skipped_too_large - objects_not_found - download_failures - media_bytes title: ExportMediaSummary description: >- What happened to media in a WebDataset export with include_media set. Every outcome is counted. An export that dropped media silently would read as a smaller dataset rather than an incomplete one. ExportDestinationResult: properties: connection_id: type: string title: Connection Id description: The connection written through. prefix: type: string title: Prefix description: The prefix the files were written under. files: items: type: string type: array title: Files description: Every key written, the manifest included. manifest_key: type: string title: Manifest Key description: >- The manifest's key. It describes the export: the files with their sizes and SHA-256 checksums, the document count, the format, and the selection that produced it, so a training run can pin this release and verify it. bytes_written: type: integer minimum: 0 title: Bytes Written description: Total bytes written. type: object required: - connection_id - prefix - files - manifest_key - bytes_written title: ExportDestinationResult description: What was written to the caller's own storage. ErrorDetail: properties: message: type: string title: Message description: Human-readable error message type: type: string title: Type description: Stable error type identifier (machine-readable) code: anyOf: - type: string - type: 'null' title: Code description: >- Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition. details: anyOf: - additionalProperties: true type: object - type: 'null' title: Details description: >- Optional structured details to help debugging (validation errors, IDs, etc.) type: object required: - message - type title: ErrorDetail description: Error detail model. FilterCondition: properties: field: type: string title: Field description: Field name to filter on operator: $ref: '#/components/schemas/FilterOperator' description: Comparison operator default: eq value: anyOf: - $ref: '#/components/schemas/DynamicValue' - {} title: Value description: Value to compare against type: object required: - field - value title: FilterCondition description: |- Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against FilterOperator: description: >- Supported filter operators across database implementations. `text` is a case-insensitive substring match on the MVS vector store, in the in-memory evaluator and in the collections list filter; it does not tokenize or rank. On the Mongo query translator it becomes a Mongo `$text` search, which needs a collection text index and searches every indexed field. For whole-word matching on one field use `words`: every word of the value must appear as a whole word, compared on its English stem. enum: - eq - ne - gt - lt - gte - lte - in - nin - contains - starts_with - ends_with - regex - exists - is_null - text - phrase - words - geo_radius - geo_bounding_box - geo_polygon title: FilterOperator type: string DynamicValue: description: A value that should be dynamically resolved from the query request. properties: type: type: string const: dynamic title: Type default: dynamic field: type: string title: Field description: >- The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id' examples: - inputs.query_text - filters.AND[0].value required: - field title: DynamicValue type: object securitySchemes: BearerAuth: type: http scheme: bearer description: >- Mixpeek API key, sent as `Authorization: Bearer mxp_sk_...`. Create one in Studio under Settings → API Keys, or with an admin key via `POST /v1/organizations/users/{user_email}/api-keys`. A missing, malformed, unknown, revoked, inactive or expired key returns 401; a key without the permission, scope or plan an operation needs returns 403. NamespaceHeader: type: apiKey in: header name: X-Namespace description: >- Namespace id (`ns_...`), not the namespace name. This scopes the request rather than authenticating it, and it is required on every operation marked `x-mixpeek-namespace-scoped`. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.