Rerank

Re-score and reorder search results using cross-encoder models for higher precision

The Rerank stage uses cross-encoder models to re-score and reorder search results. Unlike bi-encoder models (used in semantic search), cross-encoders process the query and document together, enabling more accurate relevance scoring at the cost of higher latency.

Stage Category: SORT (Reorders documents)

Transformation: N documents → top_k documents (re-ranked by relevance)

When to Use

Use Case	Description
Two-stage retrieval	Fast recall (search) + precise ranking (rerank)
High-precision requirements	When ranking quality is critical
Top-N optimization	Improve quality of final displayed results
RAG applications	Better context selection for LLM generation

When NOT to Use

Scenario	Recommended Alternative
Large result sets (1000+)	Too slow; use sort_by_field
Real-time requirements (< 20ms)	Use search scores directly
Simple attribute sorting	sort_by_field

Budget for this stage before you add it. A cross-encoder scores every candidate against the query, so its cost grows with the number of candidates you hand it, not with top_k.

Measured on a production pipeline, BAAI__bge_reranker_v2_m3 took p50 3.26s on its own, against a retriever whose search, filter and index stages together took ~38ms. The rerank was 99% of a 4.4s p50.

Set top_k on the stage feeding it. Reranking 20 candidates and keeping 10 costs a fraction of reranking 200.

Parameters

Parameter	Type	Default	Description
inference_name	string	BAAI__bge_reranker_v2_m3	Reranking inference service. List options with GET /engine/inference. Ignored when feature_uri is set.
feature_uri	string	null	Custom reranker plugin (mixpeek://...). Overrides inference_name.
top_k	integer	null	Number of results to keep after reranking (omit to keep all, reordered).
query	string	{{INPUT.query}}	Query for relevance scoring
document_field	string	content	Document field to rerank against
relevance_criteria	string	null	What counts as relevant, stated as a condition. Used by typesafe__jev and mixpeek__decision; cross-encoders ignore it.
decision_model	string	null	Model that answers for mixpeek__decision or typesafe__jev. jev-latest is Jev; a generative model such as gemini-2.5-flash-lite answers the same question. Unset means the deployment default for mixpeek__decision and Jev for typesafe__jev.

Available Models

inference_name	Model	Score
BAAI__bge_reranker_v2_m3	Multilingual cross-encoder	Normalized relevance score
typesafe__jev	Jev, TypeSafe AI’s evaluation model	Probability (0 to 1) that the document meets relevance_criteria
mixpeek__decision	The decision model named by decision_model	Probability (0 to 1) that the document meets relevance_criteria, calibrated only when Jev answers

Jev scores each query and document pair in its own request, so no document affects another document’s score. Pairs run concurrently. Set relevance_criteria to the exact condition you rank on. Jev reads it word for word.

With a decision model, stage metadata names the model that scored the documents in model_used and says whether its probabilities are calibrated in probabilities_calibrated.

On 100 SciFact claims, reranking the top 30 vector results with Jev raised nDCG@10 from 0.817 to 0.880 at 669 ms per query. The same rerank through mixpeek__decision with gemini-2.5-flash-lite scored 0.806 at 2.1 s.

For other models, deploy your own via a custom extractor and reference it with feature_uri.

Configuration Examples

Rerank with Jev

{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "typesafe__jev",
      "relevance_criteria": "The text is a drug advertisement that discloses side effects",
      "document_field": "text",
      "top_k": 10
    }
  }
}

How Cross-Encoders Work

Bi-Encoder (Search)	Cross-Encoder (Rerank)
Query and doc encoded separately	Query + doc encoded together
Pre-compute doc embeddings	Must process each pair
Fast (< 10ms for millions)	Slower (50-100ms for 100 docs)
Good approximate ranking	Precise relevance scoring

Cross-encoders see the full context of both query and document together, enabling better understanding of semantic relationships.

Two-Stage Retrieval Pattern

The recommended pattern is fast recall followed by precise reranking:

[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]

Why this works:

1. Search stage: Fast, retrieves 100 candidates (< 20ms)
2. Rerank stage: Slower but precise, picks best 10 (50-100ms)
3. Total: High-quality results in 70-120ms

Performance

Metric	Value
Latency	50-100ms (depends on candidate count)
Optimal input size	50-200 documents
Maximum practical	~500 documents
Batching	Automatic

Reranking 1000+ documents is not recommended. Use top_k limits in the search stage to control candidate pool size.

Output

Each returned document includes:

Field	Type	Description
document_id	string	Unique document identifier
score	float	Reranker relevance score
original_score	float	Score from previous stage
rerank_position	integer	Position after reranking

Common Pipeline Patterns

Search + Rerank + Limit

[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 20
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 5
      }
    }
  }
]

Search + Filter + Rerank

[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 200
          }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "category_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.category",
        "operator": "eq",
        "value": "{{INPUT.category}}"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]

Custom Reranker (BYO Model)

Use your own reranker model deployed as a custom extractor instead of the built-in models. Set feature_uri to route reranking through your extractor’s inference endpoint.

Parameters

Parameter	Type	Default	Description
feature_uri	string	null	Feature URI of a custom reranker plugin. Overrides inference_name when set.

Your plugin must accept {pairs: [[query, doc], ...]} and return {scores: [float, ...]}.

Configuration Example

{
  "stage_name": "my_rerank",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "feature_uri": "mixpeek://my_reranker@1.0.0/rerank",
      "top_k": 10
    }
  }
}

Set inference_type: "rerank" in your plugin’s manifest to declare compatibility with the rerank stage.

Trade-offs

Aspect	Impact
Higher precision	Better relevance scoring
Higher latency	50-100ms per batch
Limited scale	Best for < 500 candidates
API costs	Per-document scoring

Related

• Feature Search - Initial candidate retrieval
• Feature Search - Combined vector + text search
• Sort Attribute - Simple attribute sorting