> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deployment

> Run Mixpeek in Kubernetes or on managed Ray

Mixpeek is split into two deployable components:

* **API Layer** – FastAPI and a Redis connection (HTTP endpoints, task orchestration, webhooks). Background work is published to the work runtime, whose ledger is in MongoDB and whose units run on Ray.
* **Engine Layer** – Ray cluster + Ray Serve (extractors, inference, clustering, taxonomy runs).

Shared dependencies: MongoDB, [MVS](https://mixpeek.com/mvs), Redis, and S3-compatible object storage.

## Production Topology (Kubernetes)

```
Namespace: mixpeek-api
  └─ api-deployment (ReplicaSet + HPA; serves HTTP and publishes background work)

Namespace: kuberay-system
  └─ kuberay-operator (one per CLUSTER, watches every namespace)

Namespace: mixpeek-engine
  └─ RayService/mixpeek-engine-svc   (you apply this; KubeRay renders the rest)
       ├─ head pod          rayStartParams num-cpus: 0, schedules only
       ├─ cpu-workers       autoscaled group
       ├─ batch-workers     autoscaled group
       └─ gpu-workers       autoscaled group

Namespace: mixpeek-data
  ├─ mongodb (StatefulSet, replica set)
  ├─ mvs (StatefulSet or distributed cluster)
  └─ redis (Deployment or Redis Cluster)
```

Recommended node pools:

* **API nodes** – general purpose (e.g., `t3.xlarge`), scale the API pods horizontally.
* **CPU workers** – compute-optimized (e.g., `c5.4xlarge`) for text extraction, clustering.
* **GPU workers** – GPU instances (e.g., `p3.2xlarge`) for embeddings, rerankers, video processing.

Expose the API via an ingress or load balancer; keep Ray Serve internal unless exposing custom inference endpoints.

## Ray on Kubernetes (KubeRay)

> [!NOTE]
>
> This page describes Mixpeek operating a Ray cluster it deploys. If you already
> run Ray or KubeRay and want Mixpeek to use **yours** instead, see
> [Bring Your Own Ray](https://docs.mixpeek.com/docs/customer-hosted/overview.md) — it covers the three contracts, [every
> permission we ask for](https://docs.mixpeek.com/docs/customer-hosted/security-rbac.md), and [what we check before
> deploying anything](https://docs.mixpeek.com/docs/customer-hosted/qualification.md).

The Engine layer runs as a **RayService**, a custom resource the KubeRay operator
reconciles. You apply one YAML; KubeRay creates the head pod, the worker groups, and the
services in front of them, and replaces them on config change.

You do not create Ray pods yourself, and there is nothing to scale by hand. Each worker
group carries its own `minReplicas` and `maxReplicas`, and Ray's autoscaler moves within
that range on pending-task pressure.

The API layer reaches the cluster over `ENGINE_API_URL`, a plain in-cluster HTTP address
such as `http://mixpeek-engine-svc-head-svc.mixpeek-engine:8265`.

> [!WARNING]
>
> Job submission carries **no credential**. The API builds a bare `http://host:port` to
> the Ray dashboard and submits against it, so reachability is the whole access control.
> Keep the Ray dashboard port off any public ingress and restrict it with a
> NetworkPolicy. Anything that can reach the port can submit a job.

### Namespace-scoped or cluster-wide

Two different scopes, and only one of them is per-deployment:

| Component | Scope | Installed |
| - | - | - |
| KubeRay operator | Cluster | Once, in `kuberay-system`. It watches all namespaces. |
| RayService | Namespace | Once per deployment, in that deployment's namespace. |

So a second Mixpeek deployment on the same cluster means a second namespace with its own
RayService, not a second operator. The operator needs cluster-level permission to create
pods and services in the namespaces it watches, which is the one privilege you cannot
scope away.

Isolate deployments by namespace, one per tenant, with a `LimitRange` and a
`NetworkPolicy` on each. Separate node pools with taints keep GPU work off shared
capacity, since namespaces alone do not partition nodes.

## Core Environment Variables

| Service | Key | Description |
| - | - | - |
| API | `MONGO_URI`, `MVS_URL`, `REDIS_URL`, `S3_BUCKET`, `ENGINE_API_URL` | Connectivity |
| Engine | `MONGO_URI`, `MVS_URL`, `S3_BUCKET`, `RAY_memory`, `RAY_num_gpus` | Runtime config |
| Shared | `ENABLE_ANALYTICS`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc. | Optional providers |

Secrets should be injected via Kubernetes secrets, environment managers, or cloud secret stores.

## Health & Verification

* **Endpoint**: `GET /v1/health/deep` – checks Redis, MongoDB, MVS, object storage, the task queue and the engine. `GET /v1/health` is a liveness constant.
* **Smoke test**: create namespace → bucket → collection → upload object → submit batch → execute retriever.
* **Tasks**: `GET /v1/health/beat` confirms the work scheduler fired within the last 5 minutes, so webhook events, cache invalidations and maintenance tasks run.

## Scaling Guidelines

| Component | Scaling Strategy | Notes |
| - | - | - |
| FastAPI | Horizontal autoscale on CPU/utilization | Stateless, use HPA |
| Work runtime | Units run on the Ray worker pools; the ledger is in MongoDB | A unit outlives the pod that published it. Cancel a task with `POST /v1/tasks/{task_id}/kill` |
| Ray workers | Autoscale CPU/GPU pools | Use Ray autoscaler or managed Ray policies |
| MVS | Scales automatically with object storage backend | Monitor vector count and query latency |
| MongoDB | Use managed cluster (Atlas, DocumentDB) | Ensure indexes on `namespace_id`, `internal_id` |
| Redis | Scale vertically or cluster | Used for task queue + caching |

Monitor Ray dashboard (port 8265) for job status, resource utilization, and Serve deployments.

## Deployment Checklist

1. Provision MongoDB, MVS, Redis, and S3/GCS buckets (with IAM roles).
2. Deploy Ray cluster (head + workers) and confirm job submission works.
3. Deploy the API; configure environment variables to point to Ray + data stores.
4. Configure ingress/HTTPS, secrets, and network policies.
5. Run health checks and quickstart workflow to verify end-to-end functionality.
6. Set up observability (logs, metrics, webhooks) and configure backups for MongoDB.

## References

* [Architecture](https://docs.mixpeek.com/docs/overview/architecture.md) – full system design
* [Observability](https://docs.mixpeek.com/docs/operations/observability.md) – metrics, logs, dashboards
* [Security](https://docs.mixpeek.com/docs/operations/security.md) – tenancy, auth, secret management
* [Webhooks](https://docs.mixpeek.com/docs/operations/webhooks.md) – event processing pipeline


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.