{% extends "base.html" %} {% block title %}Semantic dedup · mneme{% endblock %} {% block content %}

Semantic deduplication

Stream a list of items through the cache and let cosine similarity flag near-duplicates. No LLM is involved here — the cache plus an embedder is enough. Threshold is stricter (0.85) than the LLM-cache default because a false positive in dedup means dropping real content.

Backed by: cache.get(content, namespace="dedup") + the cache's Hit.similarity field.

One item per line. Re-running an item produces an exact-match drop; near-paraphrases produce a semantic drop with the similarity score.

Result

#DecisionLayerSimilarityLatencyContent

{% endblock %} {% block scripts %} {% endblock %}