> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Search

> LLM-driven multi-step retrieval with iterative reasoning and tool orchestration

![Agent Search stage showing LLM reasoning loop with retriever stages as tools](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/agent-search.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=4cee3285137fc96359b6d06b82ef243b)

The Agent Search stage uses an LLM reasoning loop to orchestrate other retriever stages as callable tools. Instead of executing a fixed sequence of stages, the LLM dynamically decides which stages to invoke, with what arguments, and how many iterations to perform based on the query and intermediate results.

> [!NOTE]
>
> **Stage Category**: FILTER (Adaptive retrieval)
>
> **Transformation**: Query → LLM reasoning (1-N iterations) → Refined documents

## When to Use

| Use Case | Description |
| - | - |
| **Multi-hop queries** | Questions requiring following references across documents |
| **Iterative refinement** | Broad search followed by intelligent narrowing |
| **Complex filtering logic** | When the right filter conditions depend on query semantics |
| **Tree/hierarchy navigation** | Navigating hierarchical document indexes top-down |
| **Exploratory search** | When the best search strategy isn't known upfront |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple keyword or vector search | `feature_search` |
| Known metadata filter conditions | `attribute_filter` |
| Fixed pipeline with known stages | Chain stages directly |
| Latency-critical applications (\< 1s) | Direct stage execution |
| Cost-sensitive high-volume queries | Pre-configured pipelines |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `strategy` | string | `iterative_refinement` | Reasoning strategy (see Strategies below) |
| `stages` | list\[string] | *from strategy* | Which retriever stages the agent can invoke as tools |
| `system_prompt` | string | *from strategy* | Custom system prompt for the LLM. Supports `{{INPUT.*}}` variables |
| `max_iterations` | integer | `5` | Maximum reasoning iterations (1-20) |
| `timeout_seconds` | float | `60.0` | Total timeout for the reasoning loop (5-300s) |
| `provider` | string | *auto* | LLM provider (openai, google, anthropic, typesafe). `typesafe` runs `cluster_navigation` only |
| `model_name` | string | *auto* | Specific model to use |
| `feedback` | string | `null` | Feedback from a prior execution to inject into the agent's prompt. Use to correct or refine behavior based on previous results |
| `min_confidence` | float | `0.0` | Minimum confidence (0.0–1.0) required to accept results. Below this threshold, the agent keeps searching even if it signals done |
| `auto_strategy` | boolean | `false` | When true, a lightweight LLM call selects the best strategy automatically before the main loop. Adds \~0.5–1s latency |
| `context_source` | object | `null` | Structured context for the first prompt. `{"type": "manifest"}` gives the agent a map of the retriever's collections (see Namespace Map below) |

## Strategies

| Strategy | Default Tools | Best For |
| - | - | - |
| `iterative_refinement` | feature\_search, attribute\_filter | Progressive narrowing of results |
| `multi_hop` | feature\_search, attribute\_filter, llm\_filter | Following cross-document references |
| `tree_navigation` | feature\_search, attribute\_filter | Hierarchical index traversal |
| `cluster_navigation` | attribute\_filter | Walking a cluster hierarchy top-down to the clusters that answer the query |
| `full_catalog` | *every registered stage* | Open-ended queries where the model should compose the pipeline |
| `custom` | *user-specified* | Full control over tools and prompt |

## Namespace Map

Set `context_source` to `{"type": "manifest"}` to show the agent what it can search before it calls a tool. The stage reads the namespace manifest, keeps only the retriever's collections, and adds what is in the data:

* each collection's id, document count, extractor and fields
* which collection it is derived from, and the lineage field that links a document to its parent
* the most common values of its indexed fields and taxonomy output fields
* the clusters over it, with each group's label and size

The agent uses the map to choose a collection, a filter value or a cluster. Collections outside the retriever never appear in it. Building the map costs a few database reads before the first LLM call; `context_source.built_ms` in the stage metadata reports the time.

```json theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "iterative_refinement",
      "context_source": {"type": "manifest"},
      "max_iterations": 5
    }
  }
}
```

The same map is available outside a retriever from [`GET /v1/manifest/export?detail=stats`](https://docs.mixpeek.com/docs/operations/manifests#describe-what-is-in-a-namespace).

## Cluster Navigation

`cluster_navigation` searches a cluster hierarchy the way a person browses one. A cluster run writes one centroid document per cluster, with its `label`, `summary` and `parent_cluster_id`. The stage reads those centroids, starts at the top-level clusters, and asks the model which child cluster most likely contains documents that answer the query. It repeats this until it reaches leaf clusters. It then returns the member documents of the best leaf clusters.

The walk is a beam search, and every decision goes to a [decision model](https://docs.mixpeek.com/docs/processing/jev.md#swap-the-decision-model). Each decision returns a probability for every child cluster. The stage keeps the best `beam_width` paths and ranks them by the geometric mean of their decision probabilities, so shallow and deep leaves compare fairly. All paths at one level are asked in one request.

`model_name` picks the decision model. Leave it unset to use the deployment default, which is `jev-latest`. Jev returns calibrated probabilities. A generative model such as `gemini-2.5-flash-lite` reports its own confidence for each option, and the stage marks those as uncalibrated in `probabilities_calibrated`.

A parent cluster is shown to the model as the leaves a descent through it can reach, such as `Sports: Skateboarding / Surfing`. Labeling picks one label per cluster, so a parent that holds species and landforms can come back labeled `Species`. Listing the leaves keeps a query about a mountain from skipping that branch.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `cluster_navigation.centroid_collection_ids` | list\[string] | retriever collections | Collections that hold the centroid documents, usually the cluster's output collection |
| `cluster_navigation.beam_width` | integer | `3` | Paths kept at each level, 1 to 8. `1` is greedy descent |
| `cluster_navigation.max_leaf_clusters` | integer | `3` | Leaf clusters whose members are returned, 1 to 20 |
| `cluster_navigation.membership_field` | string | `cluster_id` | Field on member documents that names their cluster |
| `cluster_navigation.label_field` | string | `label` | Centroid field shown to the model as the option name |
| `cluster_navigation.summary_field` | string | `summary` | Centroid field shown to the model as the option description |
| `cluster_navigation.parent_field` | string | `parent_cluster_id` | Centroid field that names the parent cluster |

The stage matches members to clusters by `membership_field`, so cluster ids must be unique within the collections it reads. Point `centroid_collection_ids` at one cluster's output collection. If several cluster runs write `cl_0` into the same collection, members of every `cl_0` come back.

When the stage receives documents from an earlier stage, it keeps the ones in the chosen clusters and preserves their order. When it runs first, it reads the members from the collections. Each returned document carries `cluster_path` and `cluster_path_score`. Stage metadata includes `chosen_clusters` (each leaf with its path and score), `navigation_trace` (the candidates and probabilities at every level), `model_used` and `probabilities_calibrated`.

```json theme={null}
{
  "stage_name": "browse_clusters",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "cluster_navigation",
      "model_name": "jev-latest",
      "cluster_navigation": {
        "centroid_collection_ids": ["col_cluster_output"],
        "beam_width": 3,
        "max_leaf_clusters": 2
      }
    }
  }
}
```

See [Hierarchical search over clusters](https://docs.mixpeek.com/docs/tutorials/hierarchical-cluster-search.md) for an end-to-end example.

## Available Tools (Stages)

The agent can invoke any registered retriever stage as a tool. Each stage is presented to the LLM with a simplified parameter schema:

| Tool | What the LLM Sees | Typical Use |
| - | - | - |
| `feature_search` | Semantic similarity search with query and top\_k | Vector search |
| `attribute_filter` | Filter by field, operator, and value | Metadata filtering |
| `llm_filter` | Filter by natural language criteria | Semantic filtering |
| `rerank` | Re-score documents by relevance to query | Result refinement |
| `llm_enrich` | Generate new fields using LLM analysis | Information extraction |
| `taxonomy_enrich` | Classify documents against a taxonomy | Categorization |

## Configuration Examples

```json Iterative Refinement theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "iterative_refinement",
      "max_iterations": 5,
      "timeout_seconds": 30.0
    }
  }
}
```

```json Custom Strategy with Specific Tools theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "custom",
      "stages": ["feature_search", "attribute_filter", "rerank"],
      "system_prompt": "You are a document retrieval agent. Start with feature_search to find initial matches, then use attribute_filter to narrow by metadata, and rerank for final ordering.",
      "max_iterations": 4,
      "timeout_seconds": 45.0
    }
  }
}
```

```json Multi-Hop with LLM Filter theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "multi_hop",
      "max_iterations": 7,
      "timeout_seconds": 60.0
    }
  }
}
```

```json Tree Navigation theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "tree_navigation",
      "max_iterations": 10,
      "timeout_seconds": 90.0
    }
  }
}
```

```json With Feedback + Confidence Threshold theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "iterative_refinement",
      "max_iterations": 5,
      "timeout_seconds": 45.0,
      "min_confidence": 0.7,
      "feedback": "Previous results included marketing documents — focus only on technical specs."
    }
  }
}
```

```json Auto-Strategy Selection theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "auto_strategy": true,
      "max_iterations": 5,
      "timeout_seconds": 60.0
    }
  }
}
```

```json Full Catalog (model composes pipeline) theme={null}
{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "full_catalog",
      "max_iterations": 8,
      "timeout_seconds": 90.0
    }
  }
}
```

> [!NOTE]
>
> `full_catalog` hands the LLM every registered filter/sort/rerank/enrich stage as a tool and lets it decide the pipeline shape per query. Higher cost and latency than the pinned strategies — use it when composition quality matters more than cost.

> [!TIP]
>
> Use the `custom` strategy when you need precise control over which tools the agent can access and how it should reason. The built-in strategies provide sensible defaults for common patterns.

> [!TIP]
>
> Use `feedback` to create a human-in-the-loop refinement cycle: execute a retriever, review the results, then re-execute with feedback describing what was wrong. The agent adjusts its strategy based on your corrections. Pair with `min_confidence` to ensure the agent keeps searching until results meet your quality bar.

## How It Works

1. **Strategy selection**: If `auto_strategy` is enabled, a lightweight LLM call picks the best strategy for the query
2. **Prompt assembly**: The system prompt is built from the strategy defaults, with `feedback` prepended if provided
3. **Reasoning loop**: Each iteration, the LLM receives the query, available tools, and a budget note showing remaining iterations and seconds
4. **Tool execution**: The LLM calls retriever stages as tools. Results are summarized and fed back
5. **Confidence gating**: The LLM calls `finish_search` to declare done with a confidence score (0.0–1.0). If confidence is below `min_confidence`, the loop continues
6. **Context compression**: When the conversation grows long, older messages are replaced with a compact working-memory summary to stay within context limits
7. **Return**: Accumulated results and metadata (confidence, summary, reasoning trace) are returned

Each tool call creates a sub-state execution of the target stage, inheriting namespace and collection context from the parent. Non-empty results from each iteration replace the previous working set. If a refinement query returns zero results, the previous results are preserved.

## Performance

| Metric | Value |
| - | - |
| **Latency** | 2-30s (depends on iterations and sub-stages) |
| **Memory** | O(N) per iteration result set |
| **Cost** | LLM API calls + sub-stage costs per iteration |
| **Complexity** | O(iterations \* sub-stage complexity) |

## Common Pipeline Patterns

### Agent as First Stage

```json theme={null}
[
  {
    "stage_name": "agent_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "agent_search",
      "parameters": {
        "strategy": "iterative_refinement",
        "stages": ["feature_search", "attribute_filter"],
        "max_iterations": 3
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 10
      }
    }
  }
]
```

### Complex Query Decomposition

```json theme={null}
[
  {
    "stage_name": "agent_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "agent_search",
      "parameters": {
        "strategy": "custom",
        "stages": ["feature_search", "attribute_filter", "llm_filter"],
        "system_prompt": "Break down the user's complex query into sub-queries. Use feature_search for each sub-query, attribute_filter to narrow by metadata, and llm_filter to validate relevance. Combine the best results.",
        "max_iterations": 5,
        "timeout_seconds": 45.0
      }
    }
  },
  {
    "stage_name": "deduplicate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "deduplicate",
      "parameters": {
        "strategy": "field",
        "fields": ["document_id"]
      }
    }
  }
]
```

## Response Metadata

The stage returns execution metadata in `stage_statistics`:

| Field | Description |
| - | - |
| `iterations_used` | Number of reasoning iterations completed |
| `max_iterations` | Maximum iterations that were configured |
| `timeout_hit` | Whether the timeout was reached |
| `strategy` | Strategy that was used |
| `auto_strategy_used` | Whether auto-strategy routing was active |
| `stages_invoked` | List of tool calls made (name + arguments + result count) |
| `total_llm_cost` | Total LLM API cost for the reasoning loop |
| `model_used` | LLM model that was used |
| `final_confidence` | Agent's self-reported confidence in results (0.0–1.0) |
| `final_summary` | Agent's explanation of what was found and why it stopped |
| `reasoning_trace` | Per-iteration trace: agent reasoning, tool calls, durations |
| `context_source` | With `context_source.type` `manifest`: collections and clusters in the map, its size in characters, and `built_ms`. On failure, `error` says why and the search runs without the map |

## Error Handling

| Error | Behavior |
| - | - |
| Sub-stage execution fails | Error message returned to LLM; it can retry or try a different approach |
| Timeout reached | Returns results accumulated so far (graceful degradation) |
| Max iterations reached | Runs a recovery call to capture confidence, then returns accumulated results |
| Unknown stage in tools list | Stage is skipped (logged as warning) |
| LLM returns no tool calls | Runs a recovery call to capture confidence, then returns current results |
| Empty collection | Returns empty result set (no error) |

## Related

* [Feature Search](https://docs.mixpeek.com/docs/retrieval/stages/feature-search.md) - Vector similarity search (commonly used as agent tool)
* [Attribute Filter](https://docs.mixpeek.com/docs/retrieval/stages/attribute-filter.md) - Metadata filtering (commonly used as agent tool)
* [LLM Filter](https://docs.mixpeek.com/docs/retrieval/stages/llm-filter.md) - Natural language filtering
* [Rerank](https://docs.mixpeek.com/docs/retrieval/stages/rerank.md) - Result re-scoring by relevance


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.