Aggregate

Compute statistical aggregations and metrics across document results

The Aggregate stage computes statistical aggregations across your search results, including counts, sums, averages, min/max values, and custom metrics. This is useful for analytics, faceted search, and understanding result distributions.

Stage Category: REDUCE (Aggregates results)

Transformation: N documents → aggregation results + optional documents

Aggregating a whole collection? Use POST /v1/collections/{collection_identifier}/documents/aggregate. It runs the same functions over every matching document, not just the current pipeline’s results. It returns exact counts at any collection size, and count, count_distinct, sum, avg, min and max stream without a row cap. Pair it with the is_null filter operator for field-presence checks, such as counting documents missing a from_collection ancestor across 180k+ assets.

The two request bodies share group_by (an array of objects), function and alias. The endpoint adds filters, having, unwind and range_buckets, and calls its sort direction sort_direction where this stage uses sort_order.

aggregations is required, with at least one entry. Every entry needs function and alias. field is required for every function except count.

Read the results from the stage statistics. This stage consumes the documents and, unless include_documents is true, emits none. The groups and metrics are in stage_statistics.stages.<your stage_name>.metadata.aggregations. A response with an empty documents array and a null top-level facets is what a working aggregate stage returns. Top-level facets is filled only by the feature_search stage’s own facets option. See Output.

When to Use

Use Case	Description
Faceted search	Count documents by category
Analytics	Compute metrics across results
Price ranges	Min/max/avg calculations
Distributions	Understand result characteristics

When NOT to Use

Scenario	Recommended Alternative
Grouping with full docs	group_by
Simple counting	Use search facets
Per-document stats	code_execution
LLM-based analysis	summarize

Parameters

Parameter	Type	Default	Description
aggregations	array	Required	Aggregation operations to compute. See below.
group_by	array	none	Fields to group by, as {"field": "metadata.category", "alias": "category"}. Omit alias and the last path segment names the group key. Omit group_by for one global group.
sort_by	string	none	An alias from aggregations to sort groups by. Applies only with group_by.
sort_order	string	desc	asc or desc.
limit	integer	none	Keep the first N groups after sorting (1 to 1000). Applies only with group_by.
include_documents	boolean	false	Pass the input documents through to the next stage.

Each entry in aggregations:

Field	Type	Description
function	string	Required. One of the functions below.
alias	string	Required. The key for this metric in the output. Unique within the stage.
field	string	The field path to aggregate, in dot notation. Required for every function except count.
field_b	string	Second field for co_occurrence and correlation.
percentile_value	number	Percentile to compute, 0 to 100. Only for percentile; defaults to 50.
top_k	integer	Values or pairs to keep for frequency and co_occurrence, 1 to 100. Defaults to 10.

A field path resolves against the document the way the Group By stage does. score and document_id are read directly. A bare name such as client_id is looked up on the document, then in metadata. A dotted path such as metadata.price walks the document.

Aggregation Functions

Function	Description	Example
count	Number of documents in the group	Total results
count_distinct	Number of unique values of field	Unique authors
sum	Sum of a numeric field	Total revenue
avg	Average of a numeric field	Average price
min	Smallest value of a numeric field	Lowest price
max	Largest value of a numeric field	Highest price
first	Value from the first document in the group	Top-ranked title
last	Value from the last document in the group	Lowest-ranked title
collect	List of all values, duplicates kept	All tags
collect_distinct	List of unique values	Distinct tags
percentile	One percentile of a numeric field	P50, P95
stddev	Standard deviation (sample)	Score spread
variance	Variance (sample)	Price volatility
frequency	Top-K values with counts and percent	Top categories
co_occurrence	Top-K pairs of two fields with counts	Brand and category pairs
correlation	Pearson correlation of two numeric fields	Price vs. rating

Configuration Examples

Basic Aggregations

{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "count", "alias": "total"},
        {"function": "avg", "field": "metadata.price", "alias": "avg_price"},
        {"function": "min", "field": "metadata.price", "alias": "min_price"},
        {"function": "max", "field": "metadata.price", "alias": "max_price"}
      ]
    }
  }
}

Output

The groups and metrics come back in the stage’s statistics:

{
  "documents": [],
  "facets": null,
  "stage_statistics": {
    "stages": {
      "aggregate": {
        "input_count": 25,
        "output_count": 0,
        "metadata": {
          "num_documents_in": 25,
          "num_groups": 2,
          "aggregations": [
            {"group": {"category": "electronics"}, "metrics": {"count": 15, "avg_rating": 4.2}},
            {"group": {"category": "clothing"}, "metrics": {"count": 10, "avg_rating": 4.5}}
          ]
        }
      }
    }
  }
}

stages is keyed by the stage_name you gave the stage. output_count is 0 when include_documents is false, because the stage consumes the documents. With include_documents: true it equals input_count, and the documents continue to the next stage unchanged.

Without group_by there is one entry and group is null:

{"group": null, "metrics": {"total": 150, "avg_price": 49.99}}

Performance

Metric	Value
Latency	5-50ms
Memory	O(groups × aggregations)
Cost	Free
Scalability	Efficient for large result sets

Common Pipeline Patterns

Search with Facets

[
  {
    "stage_name": "semantic_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 500 }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "count", "alias": "total"},
          {"function": "frequency", "field": "metadata.category", "alias": "categories", "top_k": 10},
          {"function": "percentile", "field": "metadata.price", "percentile_value": 50, "alias": "median_price"}
        ],
        "include_documents": true
      }
    }
  }
]

Analytics Pipeline

[
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "conditions": {
          "AND": [
            {"field": "metadata.date", "operator": "gte", "value": "2024-01-01"},
            {"field": "metadata.status", "operator": "eq", "value": "published"}
          ]
        }
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "count", "alias": "total_published"},
          {"function": "sum", "field": "metadata.views", "alias": "total_views"},
          {"function": "avg", "field": "metadata.engagement", "alias": "avg_engagement"},
          {"function": "percentile", "field": "metadata.views", "percentile_value": 90, "alias": "p90_views"}
        ],
        "group_by": [{"field": "metadata.author", "alias": "author"}]
      }
    }
  }
]

E-Commerce Product Analytics

[
  {
    "stage_name": "hybrid_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "query": { "input_mode": "text", "value": "{{INPUT.query}}" }, "top_k": 500 }
        ],
        "final_top_k": 500
      }
    }
  },
  {
    "stage_name": "structured_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.in_stock",
        "operator": "eq",
        "value": true
      }
    }
  },
  {
    "stage_name": "aggregate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "aggregate",
      "parameters": {
        "aggregations": [
          {"function": "count", "alias": "available_products"},
          {"function": "min", "field": "metadata.price", "alias": "lowest_price"},
          {"function": "max", "field": "metadata.price", "alias": "highest_price"},
          {"function": "avg", "field": "metadata.rating", "alias": "avg_rating"},
          {"function": "count_distinct", "field": "metadata.brand", "alias": "brand_count"}
        ],
        "group_by": [{"field": "metadata.category", "alias": "category"}],
        "include_documents": true
      }
    }
  }
]

Aggregation Details

Count

{"function": "count", "alias": "total"}

Counts documents in the group. No field required.

Sum / Avg / Min / Max

{"function": "avg", "field": "metadata.price", "alias": "average_price"}

Reads numeric values only. A document whose value is missing or not a number is skipped. With no numeric values the metric is null.

Count Distinct

{"function": "count_distinct", "field": "metadata.author", "alias": "unique_authors"}

Counts unique values exactly. A list-valued field contributes each element.

Percentile

{"function": "percentile", "field": "score", "percentile_value": 95, "alias": "p95_score"}

Returns one percentile per entry, interpolated between neighbouring values. Add one entry per percentile you want.

Statistical Aggregation Examples

Percentile Analysis

{
  "stage_name": "aggregate",
  "stage_type": "reduce",
  "config": {
    "stage_id": "aggregate",
    "parameters": {
      "aggregations": [
        {"function": "percentile", "field": "score", "alias": "median_score", "percentile_value": 50},
        {"function": "percentile", "field": "score", "alias": "p90_score", "percentile_value": 90},
        {"function": "stddev", "field": "score", "alias": "score_spread"},
        {"function": "variance", "field": "score", "alias": "score_variance"}
      ]
    }
  }
}

Statistical Output Examples

Frequency returns value counts with percentages:

{
  "top_categories": [
    {"value": "electronics", "count": 45, "percent": 30.0},
    {"value": "clothing", "count": 38, "percent": 25.3},
    {"value": "books", "count": 22, "percent": 14.7}
  ]
}

Co-occurrence returns field pair counts:

{
  "brand_category_pairs": [
    {"field_a": "Nike", "field_b": "footwear", "count": 12, "percent": 8.0},
    {"field_a": "Nike", "field_b": "apparel", "count": 8, "percent": 5.3}
  ]
}

Correlation returns the Pearson coefficient (-1 to 1):

{
  "price_rating_corr": 0.342156
}

Error Handling

Error	Behavior
Missing field	The document is skipped for that aggregation
Non-numeric value in a numeric aggregation	The value is skipped; with no numeric values the metric is null
Empty results	count is 0; other metrics are null or an empty list
Unknown function, or a missing alias	Rejected when the retriever is created
Insufficient data for stddev, variance, correlation	Returns null (needs 2 or more values)

Related

• Group By - Group with full documents
• Sample - Statistical sampling
• Summarize - LLM-powered analysis