> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Enrich

> Extract structured data from documents using language model analysis

![LLM Enrich stage showing structured data extraction with language models](https://mintcdn.com/mixpeek/TwtTrae3Fi3EFJ72/assets/retrievers/llm-enrich.svg?fit=max\&auto=format\&n=TwtTrae3Fi3EFJ72\&q=85\&s=029ebd511f0d8a7b933a0168a6624a55)

The LLM Enrich stage uses language models to extract structured data from document content. It can identify entities, classify content, extract key information, and generate structured outputs.

> [!NOTE]
>
> **Stage Category**: ENRICH (Enriches documents)
>
> **Transformation**: N documents → N documents (with extracted data added)

## When to Use

| Use Case | Description |
| - | - |
| **Entity extraction** | Extract names, dates, amounts from text |
| **Content classification** | Categorize documents by topic/type |
| **Key information extraction** | Pull specific facts from unstructured text |
| **Structured output generation** | Convert prose to structured data |

## When NOT to Use

| Scenario | Recommended Alternative |
| - | - |
| Simple field transformation | `json_transform` |
| Predefined taxonomy classification | `taxonomy_enrich` |
| Large-scale processing | Pre-process during indexing |
| Real-time low-latency | Use cached extractions |

## Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `provider` | string | `google` | LLM provider: `openai`, `google`, `anthropic`, or `typesafe` (auto-inferred from `model_name` if omitted) |
| `model_name` | string | provider default | Specific model, e.g. `gpt-4o-mini`, `gemini-2.5-flash-lite`, `claude-3-5-haiku` |
| `prompt` | string | `Summarize the following content in 2-3 sentences: {{DOC.content}}` | Extraction instructions (template, supports `{{DOC.field}}` / `{{INPUT.field}}`) |
| `output_field` | string | `summary` | Dot-path of the field that receives the result |
| `output_schema` | object | `null` | JSON schema for structured output (`schema` is accepted as an alias) |
| `batch_size` | integer | `5` | Documents per batch, 1 to 25. Each document gets its own LLM call, and the calls in a batch run concurrently |
| `temperature` | float | `0.2` | Sampling temperature, 0.0 to 1.0 |
| `max_tokens` | integer | `null` | Maximum output tokens, 100 to 16000. `null` uses the provider default |
| `when` | object | `null` | Condition that selects which documents are enriched. Documents that do not match pass through unchanged |
| `use_vcache` | boolean | `null` | Turns semantic caching of LLM calls on or off for this stage. `null` follows the global setting |

The stage also accepts `multimodal_inputs`, `api_key` and `feature_uri`, described in the sections below.

Each document is sent to the model as content, with embedding vectors, blobs and internal bookkeeping fields removed, and `prompt` as the instruction. To point the model at a specific field, reference it in the prompt with `{{DOC.field_name}}`.

> [!WARNING]
>
> The stage runs on at most 10 documents. A larger working set fails with an error, so reduce it upstream with a filter or reduce stage, or set a smaller `final_top_k` on the search.

## Available Models

| Model | Speed | Quality | Best For |
| - | - | - | - |
| `gpt-4o-mini` | Fast | Good | Simple extractions |
| `gpt-4o` | Medium | Excellent | Complex analysis |
| `claude-3-haiku` | Fast | Good | Quick processing |
| `claude-3-sonnet` | Medium | Excellent | Nuanced extraction |
| `jev-latest` | Fast | Calibrated | Labels, flags and ratings from a fixed set |

`jev-latest` is [Jev](https://docs.mixpeek.com/docs/processing/jev.md), an evaluation model from TypeSafe AI. It requires an `output_schema`, and every field needs a closed set of values:

| Schema field | Jev answers it as |
| - | - |
| `boolean` | Yes/no, true when the probability is at least 0.5 |
| `number` with `minimum: 0`, `maximum: 1` | The yes probability itself |
| `string` with `enum` (2 to 255 values) | A choice from the enum |
| `integer` with `minimum` and `maximum` (at most 10 levels) | A score on those levels |
| `array` of `enum` strings | Every value whose probability is at least 0.5 |

A string field named `reason`, `reasoning`, `explanation`, `rationale` or `justification` is filled with the probabilities behind the answer. Any other free-text field is rejected when you create the retriever, and the error names the field.

## Configuration Examples

```json Classify with Jev theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "model_name": "jev-latest",
      "prompt": "Classify this document: {{DOC.content}}",
      "output_field": "classification",
      "output_schema": {
        "type": "object",
        "properties": {
          "channel": {"type": "string", "enum": ["tv_ad", "how_to", "news", "other"]},
          "urgency": {"type": "integer", "minimum": 1, "maximum": 3}
        }
      }
    }
  }
}
```

```json Basic Entity Extraction theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract all company names and person names mentioned in this document.",
      "output_field": "entities"
    }
  }
}
```

```json Structured Output with Schema theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Extract the following information from this product review.",
      "output_field": "review_analysis",
      "output_schema": {
        "type": "object",
        "properties": {
          "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
          "rating_mentioned": {"type": "number", "minimum": 1, "maximum": 5},
          "pros": {"type": "array", "items": {"type": "string"}},
          "cons": {"type": "array", "items": {"type": "string"}},
          "would_recommend": {"type": "boolean"}
        }
      }
    }
  }
}
```

```json Key Facts Extraction theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract key facts from this news article: main event, date, location, people involved, and outcome.",
      "output_field": "key_facts",
      "output_schema": {
        "type": "object",
        "properties": {
          "main_event": {"type": "string"},
          "date": {"type": "string"},
          "location": {"type": "string"},
          "people": {"type": "array", "items": {"type": "string"}},
          "outcome": {"type": "string"}
        }
      }
    }
  }
}
```

```json Topic Classification theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-3-haiku",
      "prompt": "Classify this document into primary and secondary topics. Be specific.",
      "output_field": "topics",
      "output_schema": {
        "type": "object",
        "properties": {
          "primary_topic": {"type": "string"},
          "secondary_topics": {"type": "array", "items": {"type": "string"}},
          "confidence": {"type": "number", "minimum": 0, "maximum": 1}
        }
      }
    }
  }
}
```

```json Contact Information Extraction theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract all contact information (emails, phone numbers, addresses) from this document.",
      "output_field": "contacts",
      "output_schema": {
        "type": "object",
        "properties": {
          "emails": {"type": "array", "items": {"type": "string", "format": "email"}},
          "phones": {"type": "array", "items": {"type": "string"}},
          "addresses": {"type": "array", "items": {"type": "string"}}
        }
      }
    }
  }
}
```

## Output Schema

Define structured output using JSON Schema:

```json theme={null}
{
  "output_schema": {
    "type": "object",
    "properties": {
      "field_name": {"type": "string"},
      "numeric_field": {"type": "number"},
      "boolean_field": {"type": "boolean"},
      "array_field": {"type": "array", "items": {"type": "string"}},
      "enum_field": {"type": "string", "enum": ["option1", "option2", "option3"]}
    },
    "required": ["field_name"]
  }
}
```

### Supported Types

| Type | Description |
| - | - |
| `string` | Text values |
| `number` | Numeric values |
| `boolean` | True/false |
| `array` | Lists of items |
| `object` | Nested objects |

## Output Examples

### Without Schema

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Apple Inc. announced...",
  "entities": "Companies: Apple Inc., Microsoft\nPeople: Tim Cook, Satya Nadella"
}
```

### With Schema

```json theme={null}
{
  "document_id": "doc_123",
  "content": "Great product, 5 stars!...",
  "review_analysis": {
    "sentiment": "positive",
    "rating_mentioned": 5,
    "pros": ["easy to use", "great value", "fast shipping"],
    "cons": ["packaging could be better"],
    "would_recommend": true
  }
}
```

## Performance

| Metric | Value |
| - | - |
| **Latency** | 300-800ms per batch |
| **Batch size** | 5 documents default |
| **Token usage** | \~200 tokens per document |
| **Parallel** | Documents in a batch are sent concurrently, and batches run one after another |

> [!WARNING]
>
> LLM enrichment is expensive. Consider pre-computing extractions during indexing for frequently accessed data.

## Common Pipeline Patterns

### Search + Extract + Filter

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
            "top_k": 10
          }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Extract the main topic and sentiment.",
        "output_field": "analysis",
        "output_schema": {
          "type": "object",
          "properties": {
            "topic": {"type": "string"},
            "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]}
          }
        }
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "analysis.sentiment",
        "operator": "eq",
        "value": "positive"
      }
    }
  }
]
```

### Entity Extraction Pipeline

```json theme={null}
[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": {"input_mode": "text", "value": "{{INPUT.query}}"},
            "top_k": 10
          }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "prompt": "Extract all entities with their types and relationships.",
        "output_field": "entities",
        "output_schema": {
          "type": "object",
          "properties": {
            "people": {"type": "array", "items": {"type": "object", "properties": {"name": {"type": "string"}, "role": {"type": "string"}}}},
            "organizations": {"type": "array", "items": {"type": "string"}},
            "locations": {"type": "array", "items": {"type": "string"}},
            "dates": {"type": "array", "items": {"type": "string"}}
          }
        }
      }
    }
  }
]
```

## Writing Effective Prompts

### Good Prompts

```
✓ "Extract the product name, price, and key features from this product listing."
✓ "Identify all dates mentioned and their associated events."
✓ "Classify the sentiment as positive, neutral, or negative, and explain why."
```

### Poor Prompts

```
✗ "Analyze this document" (too vague)
✗ "Get the data" (not specific)
✗ "Tell me about it" (unclear output)
```

> [!TIP]
>
> Be specific about what to extract and in what format. When using `output_schema`, the LLM will conform to the structure.

## Multimodal Query Inputs

Pass images from your query inputs to the LLM alongside document content. This enables visual comparison, brand matching, and cross-modal analysis.

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `multimodal_inputs` | object | `null` | Map of INPUT field names to media types (`"image"`, `"video"`) |

### How It Works

1. Declare which INPUT fields carry multimodal content via `multimodal_inputs`
2. At runtime, the stage extracts URLs from `{{INPUT.field_name}}`
3. Images are sent to the LLM alongside the prompt and document content
4. Works with providers that support vision (Google Gemini, OpenAI GPT-4o, Anthropic Claude)

### Configuration Examples

```json Brand Logo Comparison theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-lite",
      "prompt": "Compare this brand logo to the document. Rate alignment 1-10: {{DOC.brand_name}}",
      "output_field": "brand_alignment",
      "multimodal_inputs": {
        "query_image": "image"
      }
    }
  }
}
```

```json Visual Product Match theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Does this product image match the description? {{DOC.description}}",
      "output_field": "visual_match",
      "output_schema": {
        "type": "object",
        "properties": {
          "matches": {"type": "boolean"},
          "confidence": {"type": "number"},
          "differences": {"type": "array", "items": {"type": "string"}}
        }
      },
      "multimodal_inputs": {
        "product_photo": "image"
      }
    }
  }
}
```

```json Multi-Image Reference theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-lite",
      "prompt": "Compare the reference image against this document's content: {{DOC.title}}",
      "output_field": "comparison",
      "multimodal_inputs": {
        "reference_image": "image",
        "style_guide": "image"
      }
    }
  }
}
```

### Calling with Multimodal Inputs

When using `multimodal_inputs`, pass the image URLs in the retriever's `inputs`:

```json theme={null}
{
  "inputs": {
    "query": "Find products similar to this",
    "query_image": "https://storage.example.com/brand-logo.png"
  },
  "stages": [
    {
      "stage_name": "feature_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": { "..." : "..." }
      }
    },
    {
      "stage_name": "llm_enrich",
      "stage_type": "enrich",
      "config": {
        "stage_id": "llm_enrich",
        "parameters": {
          "prompt": "Compare this image to: {{DOC.product_name}}",
          "output_field": "match_score",
          "multimodal_inputs": {"query_image": "image"}
        }
      }
    }
  ]
}
```

> [!NOTE]
>
> If a declared multimodal input field is missing from the query inputs at runtime, the stage proceeds without it (text-only mode). No error is raised.

***

## Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek's default keys. This gives you control over costs, rate limits, and API usage.

### Why Use BYOK?

| Benefit | Description |
| - | - |
| **Cost Control** | Use your own LLM provider account and billing |
| **Rate Limits** | Use your own rate limits instead of shared |
| **Compliance** | Keep API calls under your own account |
| **Key Rotation** | Rotate keys without changing retrievers |

### Setup

**1. Store your API key as a secret**

Store your LLM provider API key in the organization secrets vault:

```bash theme={null}
curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'
```

**2. Reference the secret in your stage**

Use the `api_key` parameter with template syntax:

```json theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract key entities from this document.",
      "output_field": "entities",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}
```

### BYOK Configuration Example

```json OpenAI BYOK theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Summarize this document in 2-3 sentences.",
      "output_field": "summary",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}
```

```json Anthropic BYOK theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "anthropic",
      "provider": "anthropic",
      "model_name": "claude-3-haiku-20240307",
      "prompt": "Extract the main topics from this content.",
      "output_field": "topics",
      "api_key": "{{secrets.anthropic_api_key}}"
    }
  }
}
```

```json Google BYOK theme={null}
{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "provider": "google",
      "model_name": "gemini-3.1-flash-lite",
      "prompt": "Classify the sentiment of this text.",
      "output_field": "sentiment",
      "api_key": "{{secrets.google_api_key}}"
    }
  }
}
```

### Supported Providers

| Provider | Secret Name Example | Models |
| - | - | - |
| OpenAI | `openai_api_key` | gpt-4o, gpt-4o-mini |
| Anthropic | `anthropic_api_key` | claude-3-haiku, claude-3-sonnet, claude-3-opus |
| Google | `google_api_key` | gemini-3.1-flash-lite, gemini-2.5-pro |

> [!NOTE]
>
> When `api_key` is not specified, the stage uses Mixpeek's default API keys and usage is charged to your Mixpeek account.

## Custom Enrichment Model (BYO Plugin)

Use your own enrichment model deployed as a [custom extractor](https://docs.mixpeek.com/docs/processing/custom-extractors.md) instead of a hosted LLM provider. Set `feature_uri` to route enrichment through your extractor's inference endpoint.

### Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `feature_uri` | string | `null` | Feature URI of a custom enrichment plugin. Overrides `provider`/`model` when set. |

Your plugin must accept `{prompt: str, document: dict}` and return `{text: str}`.

### Configuration Example

```json theme={null}
{
  "stage_name": "my_enricher",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "feature_uri": "mixpeek://my_summarizer@1.0.0/summarize",
      "prompt": "Summarize: {{DOC.content}}",
      "output_field": "summary"
    }
  }
}
```

> [!TIP]
>
> Set `inference_type: "generate"` in your plugin's manifest to declare compatibility with LLM stages.

## Error Handling

| Error | Behavior |
| - | - |
| LLM call fails (timeout, rate limit, provider error) | Retried once when the error is retryable, then `output_field` holds an object with an `error` message |
| Response is not valid JSON | Raw text in `output_field` |
| More than 10 documents reach the stage | Stage fails; reduce the working set first |
| Invalid API key | `output_field` holds an `error` object with the provider's auth failure |

## Related

* [LLM Filter](https://docs.mixpeek.com/docs/retrieval/stages/llm-filter.md) - Filter using LLM evaluation
* [Taxonomy Enrich](https://docs.mixpeek.com/docs/retrieval/stages/taxonomy-enrich.md) - Predefined classification
* [JSON Transform](https://docs.mixpeek.com/docs/retrieval/stages/json-transform.md) - Template-based transformation


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.