> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Moment Group

> Merge contiguous temporal intervals into consolidated video moments, grouped by parent object

![Moment Group stage merging frame-level intervals into consolidated video moments](https://mintcdn.com/mixpeek/Bj-7wkJ91nGXYgQD/assets/retrievers/moment-group.svg?fit=max\&auto=format\&n=Bj-7wkJ91nGXYgQD\&q=85\&s=0391797d858a7a01e09d04318bffa886)

The Moment Group stage takes frame-level or chunk-level documents with temporal metadata and merges contiguous intervals into consolidated video moments (time ranges), grouped by parent object. Instead of returning individual frames that matched a query, you get precise start/end time ranges pinpointing where in a video the match occurs.

> [!NOTE]
>
> **Stage Category**: REDUCE (Merges intervals into moments)
>
> **Transformation**: N frame/chunk documents → M consolidated moments per parent video

## When to Use

| Use Case | Description |
| - | - |
| **Sub-scene localization** | Pinpoint exactly where in a video a query matches |
| **Moment extraction** | Return time-range clips instead of individual frames |
| **Video search results** | Provide start/end timestamps for player seek |
| **Highlight reels** | Extract the top-scoring moments across a video library |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Grouping by non-temporal fields | `group_by` |
| Time-window aggregations (hour/day/week) | `temporal` |
| Results without temporal metadata | `aggregate` |
| Semantic clustering of results | `cluster` |

## Requirements

* Input documents must have temporal intervals — either `query_chunks` with `start_ms`/`end_ms` (attached by `feature_search` preprocessing) or document-level time fields.
* The `parent_field` must exist on the documents so results can be grouped by source video/object.

> [!WARNING]
>
> **Text query vs file query — pick the right `time_field`.** The default `time_field: "query_chunks"` is only populated when you search with a **file** input (via [query preprocessing](https://docs.mixpeek.com/docs/retrieval/stages/feature-search.md#query-preprocessing)). For a plain **text** query, the matched video segments carry top-level `start_time`/`end_time` in **seconds**. Set `time_field: "start_time"`; the stage derives the end field by swapping `start`→`end` and scales seconds to milliseconds by the field name, so `merge_tolerance_ms` and the returned moments stay in milliseconds. Leaving `query_chunks` with a text query yields **empty moments** — the most common silent failure here. See the [Video Moment Localization recipe](https://docs.mixpeek.com/docs/retrieval/cookbook.md#video-moment-localization).

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `parent_field` | string | `"source_object_id"` | Field to group results by parent video/object. Common values: `source_object_id` (standard ingest lineage), `root_object_id` (top-level ancestor), or any custom field. |
| `time_field` | string | `"query_chunks"` | Source of temporal intervals. `"query_chunks"` reads the array `feature_search` attaches (each entry has `start_ms`, `end_ms`, `score`). For document-level timestamps, use a dot-path like `"metadata.start_ms"` — the stage derives the end field by replacing `"start"` with `"end"`. The unit is keyed off the field name: `*_time`, `*_s`, `*_sec`, `*_seconds` are read as seconds and scaled to milliseconds; `*_ms` and anything else pass through as milliseconds. |
| `merge_tolerance_ms` | integer | `2000` | Maximum gap in milliseconds between intervals before they are split into separate moments. `0` = exact-overlap only. Range: 0–60000. |
| `max_moments_per_parent` | integer | `10` | Maximum moments returned per parent, sorted by score (highest first). Range: 1–100. |
| `score_strategy` | string | `"max"` | How to aggregate scores across merged intervals: `"max"`, `"avg"`, or `"sum"`. |
| `min_score` | float | `null` | Drop moments scoring below this threshold. Range: 0.0–1.0. |
| `output_mode` | string | `"annotated"` | `"annotated"` keeps the best-scoring document per parent and attaches a `moments` array (preserves all original document fields). `"moments_only"` emits standalone moment documents with IDs like `moment_{parent}_{start_ms}`. |

## Configuration Examples

```json Basic Moment Grouping theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 2000,
      "max_moments_per_parent": 5,
      "score_strategy": "max",
      "output_mode": "annotated"
    }
  }
}
```

```json Tight Scene Localization theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 1500,
      "max_moments_per_parent": 3,
      "score_strategy": "max",
      "min_score": 0.6,
      "output_mode": "annotated"
    }
  }
}
```

```json Standalone Moment Documents theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "root_object_id",
      "time_field": "query_chunks",
      "merge_tolerance_ms": 3000,
      "max_moments_per_parent": 10,
      "score_strategy": "avg",
      "output_mode": "moments_only"
    }
  }
}
```

```json Document-Level Timestamps theme={null}
{
  "stage_name": "moment_group",
  "stage_type": "reduce",
  "config": {
    "stage_id": "moment_group",
    "parameters": {
      "parent_field": "source_object_id",
      "time_field": "metadata.start_ms",
      "merge_tolerance_ms": 2000,
      "max_moments_per_parent": 5,
      "score_strategy": "max",
      "output_mode": "annotated"
    }
  }
}
```

## Output Schema

### Annotated Mode (default)

In `annotated` mode, the best-scoring document per parent is preserved with a `moments` array attached:

```json theme={null}
{
  "document_id": "doc_abc123",
  "source_object_id": "video_001",
  "score": 0.92,
  "content": "...",
  "metadata": { "..." : "..." },
  "moments": [
    {
      "start_ms": 12000,
      "end_ms": 18500,
      "score": 0.92,
      "match_count": 4,
      "document_ids": ["doc_abc123", "doc_abc124", "doc_abc125", "doc_abc126"]
    },
    {
      "start_ms": 45000,
      "end_ms": 51000,
      "score": 0.78,
      "match_count": 2,
      "document_ids": ["doc_abc130", "doc_abc131"]
    }
  ]
}
```

### Moments-Only Mode

In `moments_only` mode, standalone moment documents are emitted:

```json theme={null}
{
  "document_id": "moment_video_001_12000",
  "start_ms": 12000,
  "end_ms": 18500,
  "score": 0.92,
  "match_count": 4,
  "document_ids": ["doc_abc123", "doc_abc124", "doc_abc125", "doc_abc126"]
}
```

### Moment Object Fields

| Field | Type | Description |
| - | - | - |
| `start_ms` | integer | Start of the moment in milliseconds |
| `end_ms` | integer | End of the moment in milliseconds |
| `score` | float | Aggregated score (per `score_strategy`) |
| `match_count` | integer | Number of source intervals merged into this moment |
| `document_ids` | array | IDs of the original documents that contributed to this moment |

## Performance

| Metric | Value |
| - | - |
| **Latency** | 5-50ms |
| **Memory** | O(N) where N = input documents |
| **Cost** | Free |
| **Scalability** | Efficient — runs in the API layer, no engine call |

## Common Pipeline Patterns

### Video Search with Moment Localization

The canonical use case: search across video frames, then consolidate matches into seekable moments.

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
            "top_k": 200
          }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "moment_group",
    "stage_type": "reduce",
    "config": {
      "stage_id": "moment_group",
      "parameters": {
        "parent_field": "source_object_id",
        "time_field": "start_time",
        "merge_tolerance_ms": 1500,
        "max_moments_per_parent": 3,
        "score_strategy": "max",
        "min_score": 0.6,
        "output_mode": "annotated"
      }
    }
  }
]
```

### Moment Extraction with Reranking

Search, rerank for precision, then group into moments:

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
            "top_k": 500
          }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "top_k": 100
      }
    }
  },
  {
    "stage_name": "moment_group",
    "stage_type": "reduce",
    "config": {
      "stage_id": "moment_group",
      "parameters": {
        "parent_field": "source_object_id",
        "time_field": "start_time",
        "merge_tolerance_ms": 2000,
        "max_moments_per_parent": 5,
        "score_strategy": "max",
        "output_mode": "annotated"
      }
    }
  }
]
```

## Error Handling

| Error | Behavior |
| - | - |
| Missing `parent_field` on document | Document skipped |
| Missing temporal metadata | Document skipped |
| No intervals pass `min_score` | Parent omitted from output |
| Empty input | Empty results returned |

## Related

* [Temporal](https://docs.mixpeek.com/docs/retrieval/stages/temporal.md) - Time-window aggregations with drift detection
* [Group By](https://docs.mixpeek.com/docs/retrieval/stages/group-by.md) - Group documents by any field value
* [Aggregate](https://docs.mixpeek.com/docs/retrieval/stages/aggregate.md) - Statistical aggregations
* [Deduplicate](https://docs.mixpeek.com/docs/retrieval/stages/deduplicate.md) - Remove duplicate documents


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.