Agent Search

LLM-driven multi-step retrieval with iterative reasoning and tool orchestration

The Agent Search stage uses an LLM reasoning loop to orchestrate other retriever stages as callable tools. Instead of executing a fixed sequence of stages, the LLM dynamically decides which stages to invoke, with what arguments, and how many iterations to perform based on the query and intermediate results.

Stage Category: FILTER (Adaptive retrieval)

Transformation: Query → LLM reasoning (1-N iterations) → Refined documents

When to Use

Use Case	Description
Multi-hop queries	Questions requiring following references across documents
Iterative refinement	Broad search followed by intelligent narrowing
Complex filtering logic	When the right filter conditions depend on query semantics
Tree/hierarchy navigation	Navigating hierarchical document indexes top-down
Exploratory search	When the best search strategy isn’t known upfront

When NOT to Use

Scenario	Recommended Alternative
Simple keyword or vector search	feature_search
Known metadata filter conditions	attribute_filter
Fixed pipeline with known stages	Chain stages directly
Latency-critical applications (< 1s)	Direct stage execution
Cost-sensitive high-volume queries	Pre-configured pipelines

Parameters

Parameter	Type	Default	Description
strategy	string	iterative_refinement	Reasoning strategy (see Strategies below)
stages	list[string]	from strategy	Which retriever stages the agent can invoke as tools
system_prompt	string	from strategy	Custom system prompt for the LLM. Supports {{INPUT.*}} variables
max_iterations	integer	5	Maximum reasoning iterations (1-20)
timeout_seconds	float	60.0	Total timeout for the reasoning loop (5-300s)
provider	string	auto	LLM provider (openai, google, anthropic, typesafe). typesafe runs cluster_navigation only
model_name	string	auto	Specific model to use
feedback	string	null	Feedback from a prior execution to inject into the agent’s prompt. Use to correct or refine behavior based on previous results
min_confidence	float	0.0	Minimum confidence (0.0–1.0) required to accept results. Below this threshold, the agent keeps searching even if it signals done
auto_strategy	boolean	false	When true, a lightweight LLM call selects the best strategy automatically before the main loop. Adds ~0.5–1s latency

Strategies

Strategy	Default Tools	Best For
iterative_refinement	feature_search, attribute_filter	Progressive narrowing of results
multi_hop	feature_search, attribute_filter, llm_filter	Following cross-document references
tree_navigation	feature_search, attribute_filter	Hierarchical index traversal
cluster_navigation	attribute_filter	Walking a cluster hierarchy top-down to the clusters that answer the query
full_catalog	every registered stage	Open-ended queries where the model should compose the pipeline
custom	user-specified	Full control over tools and prompt

Cluster Navigation

cluster_navigation searches a cluster hierarchy the way a person browses one. A cluster run writes one centroid document per cluster, with its label, summary and parent_cluster_id. The stage reads those centroids, starts at the top-level clusters, and asks the model which child cluster most likely contains documents that answer the query. It repeats this until it reaches leaf clusters. It then returns the member documents of the best leaf clusters.

The walk is a beam search, and every decision goes to a decision model. Each decision returns a probability for every child cluster. The stage keeps the best beam_width paths and ranks them by the geometric mean of their decision probabilities, so shallow and deep leaves compare fairly. All paths at one level are asked in one request.

model_name picks the decision model. Leave it unset to use the deployment default, which is jev-latest. Jev returns calibrated probabilities. A generative model such as gemini-2.5-flash-lite reports its own confidence for each option, and the stage marks those as uncalibrated in probabilities_calibrated.

A parent cluster is shown to the model as the leaves a descent through it can reach, such as Sports: Skateboarding / Surfing. Labeling picks one label per cluster, so a parent that holds species and landforms can come back labeled Species. Listing the leaves keeps a query about a mountain from skipping that branch.

Parameter	Type	Default	Description
cluster_navigation.centroid_collection_ids	list[string]	retriever collections	Collections that hold the centroid documents, usually the cluster’s output collection
cluster_navigation.beam_width	integer	3	Paths kept at each level, 1 to 8. 1 is greedy descent
cluster_navigation.max_leaf_clusters	integer	3	Leaf clusters whose members are returned, 1 to 20
cluster_navigation.membership_field	string	cluster_id	Field on member documents that names their cluster
cluster_navigation.label_field	string	label	Centroid field shown to the model as the option name
cluster_navigation.summary_field	string	summary	Centroid field shown to the model as the option description
cluster_navigation.parent_field	string	parent_cluster_id	Centroid field that names the parent cluster

The stage matches members to clusters by membership_field, so cluster ids must be unique within the collections it reads. Point centroid_collection_ids at one cluster’s output collection. If several cluster runs write cl_0 into the same collection, members of every cl_0 come back.

When the stage receives documents from an earlier stage, it keeps the ones in the chosen clusters and preserves their order. When it runs first, it reads the members from the collections. Each returned document carries cluster_path and cluster_path_score. Stage metadata includes chosen_clusters (each leaf with its path and score), navigation_trace (the candidates and probabilities at every level), model_used and probabilities_calibrated.

{
  "stage_name": "browse_clusters",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "cluster_navigation",
      "model_name": "jev-latest",
      "cluster_navigation": {
        "centroid_collection_ids": ["col_cluster_output"],
        "beam_width": 3,
        "max_leaf_clusters": 2
      }
    }
  }
}

See Hierarchical search over clusters for an end-to-end example.

Available Tools (Stages)

The agent can invoke any registered retriever stage as a tool. Each stage is presented to the LLM with a simplified parameter schema:

Tool	What the LLM Sees	Typical Use
feature_search	Semantic similarity search with query and top_k	Vector search
attribute_filter	Filter by field, operator, and value	Metadata filtering
llm_filter	Filter by natural language criteria	Semantic filtering
rerank	Re-score documents by relevance to query	Result refinement
llm_enrich	Generate new fields using LLM analysis	Information extraction
taxonomy_enrich	Classify documents against a taxonomy	Categorization

Configuration Examples

Iterative Refinement

{
  "stage_name": "agent_search",
  "stage_type": "filter",
  "config": {
    "stage_id": "agent_search",
    "parameters": {
      "strategy": "iterative_refinement",
      "max_iterations": 5,
      "timeout_seconds": 30.0
    }
  }
}

full_catalog hands the LLM every registered filter/sort/rerank/enrich stage as a tool and lets it decide the pipeline shape per query. Higher cost and latency than the pinned strategies — use it when composition quality matters more than cost.

Use the custom strategy when you need precise control over which tools the agent can access and how it should reason. The built-in strategies provide sensible defaults for common patterns.

Use feedback to create a human-in-the-loop refinement cycle: execute a retriever, review the results, then re-execute with feedback describing what was wrong. The agent adjusts its strategy based on your corrections. Pair with min_confidence to ensure the agent keeps searching until results meet your quality bar.

How It Works

1. Strategy selection: If auto_strategy is enabled, a lightweight LLM call picks the best strategy for the query
2. Prompt assembly: The system prompt is built from the strategy defaults, with feedback prepended if provided
3. Reasoning loop: Each iteration, the LLM receives the query, available tools, and a budget note showing remaining iterations and seconds
4. Tool execution: The LLM calls retriever stages as tools. Results are summarized and fed back
5. Confidence gating: The LLM calls finish_search to declare done with a confidence score (0.0–1.0). If confidence is below min_confidence, the loop continues
6. Context compression: When the conversation grows long, older messages are replaced with a compact working-memory summary to stay within context limits
7. Return: Accumulated results and metadata (confidence, summary, reasoning trace) are returned

Each tool call creates a sub-state execution of the target stage, inheriting namespace and collection context from the parent. Non-empty results from each iteration replace the previous working set. If a refinement query returns zero results, the previous results are preserved.

Performance

Metric	Value
Latency	2-30s (depends on iterations and sub-stages)
Memory	O(N) per iteration result set
Cost	LLM API calls + sub-stage costs per iteration
Complexity	O(iterations * sub-stage complexity)

Common Pipeline Patterns

Agent as First Stage

[
  {
    "stage_name": "agent_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "agent_search",
      "parameters": {
        "strategy": "iterative_refinement",
        "stages": ["feature_search", "attribute_filter"],
        "max_iterations": 3
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "query": "{{INPUT.query}}",
        "document_field": "content"
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 10
      }
    }
  }
]

Complex Query Decomposition

[
  {
    "stage_name": "agent_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "agent_search",
      "parameters": {
        "strategy": "custom",
        "stages": ["feature_search", "attribute_filter", "llm_filter"],
        "system_prompt": "Break down the user's complex query into sub-queries. Use feature_search for each sub-query, attribute_filter to narrow by metadata, and llm_filter to validate relevance. Combine the best results.",
        "max_iterations": 5,
        "timeout_seconds": 45.0
      }
    }
  },
  {
    "stage_name": "deduplicate",
    "stage_type": "reduce",
    "config": {
      "stage_id": "deduplicate",
      "parameters": {
        "strategy": "field",
        "fields": ["document_id"]
      }
    }
  }
]

Response Metadata

The stage returns execution metadata in stage_statistics:

Field	Description
iterations_used	Number of reasoning iterations completed
max_iterations	Maximum iterations that were configured
timeout_hit	Whether the timeout was reached
strategy	Strategy that was used
auto_strategy_used	Whether auto-strategy routing was active
stages_invoked	List of tool calls made (name + arguments + result count)
total_llm_cost	Total LLM API cost for the reasoning loop
model_used	LLM model that was used
final_confidence	Agent’s self-reported confidence in results (0.0–1.0)
final_summary	Agent’s explanation of what was found and why it stopped
reasoning_trace	Per-iteration trace: agent reasoning, tool calls, durations

Error Handling

Error	Behavior
Sub-stage execution fails	Error message returned to LLM; it can retry or try a different approach
Timeout reached	Returns results accumulated so far (graceful degradation)
Max iterations reached	Runs a recovery call to capture confidence, then returns accumulated results
Unknown stage in tools list	Stage is skipped (logged as warning)
LLM returns no tool calls	Runs a recovery call to capture confidence, then returns current results
Empty collection	Returns empty result set (no error)

Related

• Feature Search - Vector similarity search (commonly used as agent tool)
• Attribute Filter - Metadata filtering (commonly used as agent tool)
• LLM Filter - Natural language filtering
• Rerank - Result re-scoring by relevance