Rerank

Re-score and reorder search results using cross-encoder models for higher precision

The Rerank stage uses cross-encoder models to re-score and reorder search results. Unlike bi-encoder models (used in semantic search), cross-encoders process the query and document together, enabling more accurate relevance scoring at the cost of higher latency.

Stage Category: SORT (Reorders documents)

Transformation: N documents → top_k documents (re-ranked by relevance)

​

When to Use

Use Case	Description
Two-stage retrieval	Fast recall (search) + precise ranking (rerank)
High-precision requirements	When ranking quality is critical
Top-N optimization	Improve quality of final displayed results
RAG applications	Better context selection for LLM generation

​

When NOT to Use

Scenario	Recommended Alternative
Large result sets (1000+)	Too slow; use sort_by_field
Real-time requirements (< 20ms)	Use search scores directly
Simple attribute sorting	sort_by_field

Budget for this stage before you add it. A cross-encoder scores every candidate against the query, so its cost grows with the number of candidates you hand it, not with top_k.

Measured on a production pipeline, BAAI__bge_reranker_v2_m3 took p50 3.26s on its own, against a retriever whose search, filter and index stages together took ~38ms. The rerank was 99% of a 4.4s p50.

Set top_k on the stage feeding it. Reranking 20 candidates and keeping 10 costs a fraction of reranking 200.

​

Parameters

Parameter	Type	Default	Description
inference_name	string	BAAI__bge_reranker_v2_m3, if served	Reranking inference service. See Default model. Ignored when feature_uri is set.
feature_uri	string	null	Custom reranker plugin (mixpeek://...). Overrides inference_name.
top_k	integer	null	Number of results to keep after reranking (omit to keep all, reordered).
max_candidates	integer	10	Score at most this many incoming documents, in the order the previous stage returned them; the rest follow unscored in that order. Set a larger number (up to 1000), or null to score everything, when you need deeper reranking and can accept the latency.
query	string	{{INPUT.query}}	Query for relevance scoring
document_field	string	content	Document field holding the text to rerank against. See Text only.
relevance_criteria	string	null	What counts as relevant, stated as a condition. Used by typesafe__jev and mixpeek__decision; cross-encoders ignore it.
decision_model	string	null	Model that answers for mixpeek__decision or typesafe__jev. jev-latest is Jev; a generative model such as gemini-2.5-flash-lite answers the same question. Unset means the deployment default for mixpeek__decision and Jev for typesafe__jev.
learning_config	object	null	The stage’s learning block. evaluation_dataset_id names the evaluation a trained reranker is scored on: a trained set ships only when it scores higher there than the live configuration, and without it nothing ships. shipped_weight_set is read-only, written by the evaluation gate (weight_set_id, evaluation_id, baseline_evaluation_id, shipped_at); a request that sets it is refused.

​

Default model

The stage calls BAAI__bge_reranker_v2_m3 when you omit inference_name. A deployment serves only the models it has turned on, so that model may be absent.

To find what a deployment serves, save the retriever. If the model is not served, the save returns a 422. Its details.served_alternatives lists the served reranking models, and details.default says whether the refused model was the default. Set inference_name to one of the listed models.

With on_error: "skip" the save skips this check. Execution then skips the stage and names the skip in warnings.

​

Scores after rerank

The stage orders documents by scores.rerank (or your score_field) and leaves score as the retrieval score, so score is not in descending order after this stage. Read or threshold scores.rerank when you want the value that ordered the results. The stage’s metadata names that field as ordered_by, and a score_normalize stage needs it set as its score_field.

​

Available Models

Availability depends on the deployment, as described in Default model.

inference_name	Model	Score
BAAI__bge_reranker_v2_m3	Multilingual cross-encoder	Normalized relevance score
typesafe__jev	Jev, TypeSafe AI’s evaluation model	Probability (0 to 1) that the document meets relevance_criteria
mixpeek__decision	The decision model named by decision_model	Probability (0 to 1) that the document meets relevance_criteria, calibrated only when Jev answers

Jev scores each query and document pair in its own request, so no document affects another document’s score. Pairs run concurrently. Set relevance_criteria to the exact condition you rank on. Jev reads it word for word.

With a decision model, stage metadata names the model that scored the documents in model_used and says whether its probabilities are calibrated in probabilities_calibrated.

On 100 SciFact claims, reranking the top 30 vector results with Jev raised nDCG@10 from 0.817 to 0.880 at 669 ms per query. The same rerank through mixpeek__decision with gemini-2.5-flash-lite scored 0.806 at 2.1 s.

For other models, deploy your own via a custom extractor and reference it with feature_uri.

​

Text only

Rerank scores text. It reads document_field as a string and never opens an image, a video or a URL. No served reranker scores image content.

The field on the candidates	What the stage does
Missing on every candidate	Fails with a 400 naming the field. With on_error: "skip", the stage is skipped and warnings says so.
Missing on some candidates	Scores the rest and leaves those candidates out. warnings names them, and stage metadata reports documents_missing_field.
A URL or a non-text value	Scores that value as a string, which says nothing about the media. warnings says so, and stage metadata reports documents_non_text_field.

The exception is the default document_field: when no candidate has content, the stage uses text and says so in warnings.

For documents with only an image:

• Rank them with feature_search on an image embedding. A multimodal embedding takes a text query.
• Reorder them with maxsim_rerank if they hold token vectors from a multi-vector visual feature.
• Store a caption or OCR text at ingest, and set document_field to that field.

​

Configuration Examples

Rerank with Jev

{
  "stage_name": "rerank",
  "stage_type": "sort",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "inference_name": "typesafe__jev",
      "relevance_criteria": "The text is a drug advertisement that discloses side effects",
      "document_field": "text",
      "top_k": 10
    }
  }
}

​

How Cross-Encoders Work

Bi-Encoder (Search)	Cross-Encoder (Rerank)
Query and doc encoded separately	Query + doc encoded together
Pre-compute doc embeddings	Must process each pair
Fast (< 10ms for millions)	Slower (50-100ms for 100 docs)
Good approximate ranking	Precise relevance scoring

Cross-encoders see the full context of both query and document together, enabling better understanding of semantic relationships.

​

Two-Stage Retrieval Pattern

The recommended pattern is fast recall followed by precise reranking:

[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]

Why this works:

1. Search stage: Fast, retrieves 100 candidates (< 20ms)
2. Rerank stage: Slower but precise, picks best 10 (50-100ms)
3. Total: High-quality results in 70-120ms

​

Performance

Metric	Value
Latency	50-100ms (depends on candidate count)
Optimal input size	50-200 documents
Maximum practical	~500 documents
Batching	Automatic

Reranking 1000+ documents is not recommended. Use top_k limits in the search stage to control candidate pool size.

​

Output

Each returned document includes:

Field	Type	Description
document_id	string	Unique document identifier
score	float	Reranker relevance score
original_score	float	Score from previous stage
rerank_position	integer	Position after reranking

​

Common Pipeline Patterns

​

Search + Rerank + Limit

[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 100
          }
        ],
        "final_top_k": 100
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 20
      }
    }
  },
  {
    "stage_name": "limit",
    "stage_type": "reduce",
    "config": {
      "stage_id": "limit",
      "parameters": {
        "limit": 5
      }
    }
  }
]

​

Search + Filter + Rerank

[
  {
    "stage_name": "search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1",
            "query": { "input_mode": "text", "value": "{{INPUT.query}}" },
            "top_k": 200
          }
        ],
        "final_top_k": 200
      }
    }
  },
  {
    "stage_name": "category_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "metadata.category",
        "operator": "eq",
        "value": "{{INPUT.category}}"
      }
    }
  },
  {
    "stage_name": "rerank",
    "stage_type": "sort",
    "config": {
      "stage_id": "rerank",
      "parameters": {
        "inference_name": "BAAI__bge_reranker_v2_m3",
        "top_k": 10
      }
    }
  }
]

​

Custom Reranker (BYO Model)

Use your own reranker model deployed as a custom extractor instead of the built-in models. Set feature_uri to route reranking through your extractor’s inference endpoint.

​

Parameters

Parameter	Type	Default	Description
feature_uri	string	null	Feature URI of a custom reranker plugin. Overrides inference_name when set.

Your plugin must accept {pairs: [[query, doc], ...]} and return {scores: [float, ...]}.

​

Configuration Example

{
  "stage_name": "my_rerank",
  "config": {
    "stage_id": "rerank",
    "parameters": {
      "feature_uri": "mixpeek://my_reranker@1.0.0/rerank",
      "top_k": 10
    }
  }
}

Set inference_type: "rerank" in your plugin’s manifest to declare compatibility with the rerank stage.

​

Trade-offs

Aspect	Impact
Higher precision	Better relevance scoring
Higher latency	50-100ms per batch
Limited scale	Best for < 500 candidates
API costs	Per-document scoring

​

Related

• Feature Search - Initial candidate retrieval
• Feature Search - Combined vector + text search
• Sort Attribute - Simple attribute sorting