{% extends "base.html" %} {% from "components/help_macros.html" import tooltip, help_panel, help_step, help_tip, glossary_term with context %} {% block title %}Embedding Settings{% endblock %} {% block extra_head %}{% endblock %} {% block content %}

Default Embedding Settings

Configure default embedding models for new collections and semantic search

{% call help_panel('embedding-how', 'Understanding Embeddings', icon='brain', collapsed=true, dismissible=true) %}
{{ help_step(1, "What are Embeddings?", "Embeddings convert text into numbers (vectors) that capture meaning. Similar concepts have similar vectors.") }} {{ help_step(2, "Document Chunk Size", "Maximum size of each chunk — characters for the recursive and semantic splitters, tokens for the token and sentence splitters. Larger chunks keep more context per search result but less precision; smaller chunks are more granular but may lose context.") }} {{ help_step(3, "Text Splitter", "Recursive: uses natural breaks. Semantic: AI-powered grouping (slower). Token-based: respects LLM limits.") }} {{ help_step(4, "Why Lock Settings?", "Mixing different embedding models causes inconsistent search results. Each collection uses one model.") }}
{{ help_tip("Recommended: Start with defaults. Only change if you understand the trade-offs.") }} {% endcall %}

Important: Existing Collections

Changing these settings will only affect NEW collections. Existing collections keep their original embedding settings to ensure data integrity and search consistency.

Once a collection is indexed, its embedding settings are locked. To change settings for an existing collection, use Re-Index All Documents to build a replacement index. The current index stays available until the replacement is ready.

Supported Sentence Transformers models

These are the models available directly through the Sentence Transformers provider. Indexing a large collection can take a while; keeping the default is fine if the resource use works for you. Choose a smaller model if CPU load or memory use interferes with other work. Each link opens the model's Hugging Face page with more details.

Models have different input limits; long chunks may be truncated. Review chunk size when switching models. For more locally managed model choices, use Ollama. Embedding performance guide ↗

Test Configuration

Test the currently selected embedding model to ensure it's working correctly.

Using another Sentence Transformers model? Sentence Transformers models require a one-time download before they can be used offline. Temporarily set Egress Scope to Public only, or use Adaptive while it resolves to public (a public primary search engine with no private collection selected). If you enabled Require local embeddings, turn it off for this download. Select the model under Configuration, then click Test Embedding Model. After the test passes, LDR can reuse the local cache when you return to a more restrictive scope. Direct arbitrary Hugging Face model IDs are not downloaded. For a broader choice of locally managed embedding models, we recommend using Ollama. Change this in Settings → Search Engines → Egress Scope.

Configuration

Saved Default Settings

These are your saved defaults for new collections. Change them using the form below.

Default Embedding Configuration

Characters for recursive/semantic; tokens for token and sentence (sentence is capped by the tokenizer's max length).
Ignored by the semantic splitter (always 0).
Recursive and semantic respect text separators; token and sentence ignore them.
Cosine similarity recommended for text embeddings
HNSW recommended for >10K documents (50-60× faster)
Recommended: Ensures fair comparison regardless of document length
JSON array of separators. Default works for 80% of use cases. Used by recursive and semantic splitters only — token and sentence ignore this. For code, add language-specific separators like "\nclass", "\ndef".
Changes save automatically. These settings are used as defaults for new collections; existing collections continue using their current settings unless re-indexed.
{% include "components/embedding_performance_hint.html" %}

Provider Information

{% endblock %} {% block page_scripts %} {% endblock %}