> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Connect Mixpeek to your cloud object storage to ingest multimodal data in place.

> [!NOTE]
>
> Integrate Mixpeek directly with your existing object storage solutions like AWS S3, Google Cloud Storage (GCS), and Azure Blob Storage to process and analyze your multimodal data where it lives.

Object storage is the warehouse's canonical store, where all extracted features persist across tiers and serve as the durable source of truth regardless of which serving layer is active. Object storage systems are a fundamental component for storing large amounts of unstructured data, making them ideal upstream sources for Mixpeek. By connecting your buckets, you enable Mixpeek to automatically discover, process, and index your files, making images, videos, audio, and PDFs searchable in place.

## Why Connect Object Storage?

> [card] Centralized Data Processing
>
> Process diverse file types stored in your buckets without needing to move data. Mixpeek accesses files directly from your provider.

> [card] Scalable Ingestion
>
> Use the scalability of cloud object storage. Mixpeek can handle growing volumes of data as your needs evolve.

> [card] Automated Workflows
>
> Set up automated pipelines. New files added to connected buckets can be automatically indexed and enriched by Mixpeek.

> [card] Secure Access
>
> Utilize secure authentication methods (like IAM roles or access keys) to grant Mixpeek the necessary permissions to access your data.

## Supported Providers

Mixpeek supports direct integration with the major cloud object storage providers:

> [card] [AWS S3](https://docs.mixpeek.com/docs/integrations/object-storage/s3.md)
>
> Connect your Amazon Simple Storage Service (S3) buckets.

> [card] [Google Drive](https://docs.mixpeek.com/docs/integrations/object-storage/google-drive.md)
>
> Google Drive and Workspace shared drives, via service account or OAuth.

> [card] [Supabase Storage](https://docs.mixpeek.com/docs/integrations/object-storage/supabase.md)
>
> S3-compatible storage on Supabase, via project S3 access keys.

> [card] [Cloudflare R2](https://docs.mixpeek.com/docs/integrations/object-storage/r2.md)
>
> S3-compatible egress-free storage on Cloudflare's edge.

> [card] [Backblaze B2](https://docs.mixpeek.com/docs/integrations/object-storage/backblaze.md)
>
> Low-cost S3-compatible object storage.

> [card] [Wasabi](https://docs.mixpeek.com/docs/integrations/object-storage/wasabi.md)
>
> S3-compatible hot object storage.

> [card] [Tigris](https://docs.mixpeek.com/docs/integrations/object-storage/tigris.md)
>
> Globally distributed S3-compatible storage.

> [card] [Box](https://docs.mixpeek.com/docs/integrations/object-storage/box.md)
>
> Connect Box workspaces with OAuth or CCG.

> [card] [Mux](https://docs.mixpeek.com/docs/integrations/object-storage/mux.md)
>
> Sync video assets from the Mux video infrastructure platform — playback metadata flows through to retriever results.

> [card] [Iconik](https://docs.mixpeek.com/docs/integrations/object-storage/iconik.md)
>
> Sync media assets from Iconik DAM — proxy files, metadata, and real-time webhook sync.

Choose your provider above to find specific setup instructions.

> [!NOTE]
>
> **Google Cloud Storage** and **Azure Blob Storage** use the provider's native SDK, so
> neither needs S3-compatible interop keys. The provider types are `gcs` and `azure_blob`.
> A bare `azure` returns HTTP 422.

The full connector list is the provider picker in Studio under **Settings > Connections > New Connection**. It covers the object stores above plus [Snowflake](https://docs.mixpeek.com/docs/integrations/snowflake-warehouse.md), [PostgreSQL](https://docs.mixpeek.com/docs/integrations/databases/postgresql.md), [SharePoint / OneDrive](https://docs.mixpeek.com/docs/integrations/object-storage/sharepoint.md), [RSS](https://docs.mixpeek.com/docs/integrations/web-data/rss.md), [HTTP API](https://docs.mixpeek.com/docs/integrations/web-data/http-api.md), [BrightData](https://docs.mixpeek.com/docs/integrations/web-data/brightdata.md), [Instagram](https://docs.mixpeek.com/docs/integrations/social-media/instagram.md), [TikTok](https://docs.mixpeek.com/docs/integrations/social-media/tiktok.md), and [inbound email](https://docs.mixpeek.com/docs/integrations/email.md).

## Best Practices for Data Structuring

While Mixpeek can process complex, nested data structures within a single bucket connection, a more scalable strategy often involves structuring your data upfront in your object storage, then relying on taxonomy joins to put them together.

We rely on the post processing joins to intelligently combine related content after ingestion. This approach offers several advantages over trying to group files during the initial upload:

> [card] Explicit Structure
>
> By organizing content into separate buckets or prefixes, you make the relationships between files clear and programmatically accessible. This is more reliable than trying to infer relationships from file names or metadata.

> [card] Flexible Joining
>
> Mixpeek's taxonomy and clustering features allow you to join related content based on multiple criteria (IDs, metadata, semantic similarity) after ingestion, giving you more control over how content is combined.

> [card] Scalable Processing
>
> Processing pipelines become simpler and more focused when handling one type of content structure at a time, making it easier to scale and maintain.

> [card] Reliable Updates
>
> When new content is added to a structured bucket, Mixpeek can process it independently and then join it with related content, avoiding issues with partial uploads or timing dependencies.

**Recommended Approach: Pre-Structured Pipelines**

1. **Separate Buckets or Prefixes:** Organize related but distinct types of content into separate S3 buckets or dedicated prefixes (folders) within a single bucket.
   * *Example:* For analyzing video content, you might store raw videos in `s3://my-videos/raw/`, extracted transcripts in `s3://my-videos/transcripts/`, and associated metadata JSON files in `s3://my-videos/metadata/`.
2. **Multiple Mixpeek Connections:** Set up distinct Mixpeek buckets & collections pointing to each specific bucket or prefix.
3. **Join in Mixpeek:** After ingestion, use Mixpeek's enrichment features like **Clustering** or **Taxonomies** (joining based on matching IDs or other rules) to link the related pieces of content (e.g., connecting a transcript to its corresponding video and metadata).

This approach is more reliable than alternatives like:

* Relying on file naming conventions or metadata (which can be inconsistent)
* Using "agent mode" with LLMs to infer relationships (which is experimental and less predictable)
* Trying to group files during upload (which can be fragile due to timing issues)

Consider this approach if you are dealing with complex multimodal data where different components (like video, audio, text transcripts, metadata) need to be linked and analyzed together.

## Turn On Versioning in Your Bucket

Mixpeek reads your objects by reference: the files stay in your bucket, and Mixpeek stores the features it extracts from them. A namespace snapshot records which version of each object it saw (on Google Cloud Storage, the highest object generation at the time of the snapshot). A restore can point back at those versions only if your bucket still keeps them. If an object is overwritten or deleted after the snapshot and the bucket keeps no old versions, the restored documents point at the new file, or at nothing.

We recommend turning on object versioning, with old versions kept at least as long as your snapshots. Namespace snapshots are kept for 30 days by default.

* **Google Cloud Storage:** enable [Object Versioning](https://cloud.google.com/storage/docs/object-versioning), then add a lifecycle rule that deletes noncurrent versions after 30 days (`daysSinceNoncurrentTime: 30`).
* **AWS S3 and S3-compatible stores:** enable [bucket versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html), then add a lifecycle rule with `NoncurrentVersionExpiration` set to 30 days.
* **Azure Blob Storage:** enable [blob versioning](https://learn.microsoft.com/azure/storage/blobs/versioning-overview), then add a lifecycle policy that deletes previous versions after 30 days.

Versioning stores a copy of every overwritten object for the retention you choose, so its cost is your overwrite volume multiplied by the retention days. Buckets whose objects are written once and never replaced pay almost nothing for it.

## Getting Started

1. **Choose your Provider:** Select the object storage provider you use (AWS S3, GCS, Azure Blob Storage).
2. **Configure Access:** Follow the provider-specific guide to grant Mixpeek secure access to your desired bucket(s). This typically involves setting up appropriate permissions (e.g., read access).
3. **Add Connection in Mixpeek:** Use the Mixpeek dashboard or API to add the connection details for your object storage bucket.

   > [!WARNING]
   >
   > Creating, updating, or deleting a storage connection requires an API key with the **`admin`** permission. Connections hold credentials, so they sit behind organization administration alongside members, billing, secrets, and audit.
   >
   > The presets in **Settings → API Keys → New key** top out at read + write + delete, which is enough to create buckets, collections, and syncs but **not** connections. To grant `admin`, open the key's **Edit** dialog and tick **Admin**. A key without it gets:
   >
   > ```json theme={null}
   > {"error": {"type": "UnauthorizedError",
   >            "details": {"required_permissions": ["admin"],
   >                        "your_permissions": ["read", "write", "delete"]}}}
   > ```
   >
   > Reading and listing connections only needs `read`, so a scripted sync that references an existing `connection_id` does not need an admin key.
4. **Start Processing:** Once connected, Mixpeek can begin discovering and processing files according to your configured pipelines.

Ready to connect your data? Select a provider guide above to begin.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.