Metadata-Version: 2.4
Name: pyvectorhound
Version: 1.3.3
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: Other/Proprietary License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Dist: numpy>=1.21,<3
Requires-Dist: requests>=2.28,<3
Requires-Dist: python-frontmatter>=1.0.0
Requires-Dist: pydantic>=2.0,<3
Requires-Dist: pyvectorhound[qdrant,chroma,milvus,weaviate,postgres,pgvector,dev] ; extra == 'all'
Requires-Dist: chromadb>=0.4,<1 ; extra == 'chroma'
Requires-Dist: pytest>=7.0,<9 ; extra == 'dev'
Requires-Dist: pytest-cov>=4.0,<7 ; extra == 'dev'
Requires-Dist: mypy>=1.0,<2 ; extra == 'dev'
Requires-Dist: black>=23.0,<26 ; extra == 'dev'
Requires-Dist: ruff>=0.1,<1 ; extra == 'dev'
Requires-Dist: maturin>=1.7,<2 ; extra == 'dev'
Requires-Dist: pymilvus>=2.3,<3 ; extra == 'milvus'
Requires-Dist: psycopg2-binary>=2.9,<3 ; extra == 'pgvector'
Requires-Dist: psycopg2-binary>=2.9,<3 ; extra == 'postgres'
Requires-Dist: qdrant-client>=1.7,<3 ; extra == 'qdrant'
Requires-Dist: weaviate-client>=3.24,<4 ; extra == 'weaviate'
Provides-Extra: all
Provides-Extra: chroma
Provides-Extra: dev
Provides-Extra: milvus
Provides-Extra: pgvector
Provides-Extra: postgres
Provides-Extra: qdrant
Provides-Extra: weaviate
License-File: LICENSE
Summary: Diagnostic tool for vector search failures in RAG and LLM systems. Evaluate embedding quality, analyze vector search rank correlation, benchmark BM25 vs semantic search, profile reranker performance. Identify root cause of retrieval failures.
Keywords: rag,retrieval-augmented-generation,vector-search,embedding,semantic-search,llm,large-language-models,information-retrieval,search-ranking,evaluation-metrics,bm25,reranking,debugging,diagnostics,nlp,natural-language-processing,similarity-search,faiss,elasticsearch,pinecone,milvus,ai-debugging
Author-email: Georgi Mammen Mullassery <mullassery@gmail.com>
Maintainer-email: Georgi Mammen Mullassery <mullassery@gmail.com>
License: Proprietary
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Bug Tracker, https://github.com/Mullassery/Pyvectorhound/issues
Project-URL: Changelog, https://github.com/Mullassery/Pyvectorhound/releases
Project-URL: Discussions, https://github.com/Mullassery/Pyvectorhound/discussions
Project-URL: Documentation, https://github.com/Mullassery/Pyvectorhound#readme
Project-URL: Homepage, https://github.com/Mullassery/Pyvectorhound
Project-URL: Repository, https://github.com/Mullassery/Pyvectorhound
Project-URL: Source Code, https://github.com/Mullassery/Pyvectorhound/tree/main

# PyVectorHound

**Diagnose why your RAG retrieval is returning the wrong documents.**

PyVectorHound is a component-level diagnostic engine for retrieval-augmented
generation (RAG) pipelines. Point it at a set of search results (from your
own pipeline, or from a live Qdrant/Chroma/Milvus/pgvector/Weaviate
instance) and it isolates *which stage* is failing — embedding quality or
vector search ranking — and gives you plain-English, ranked recommendations.

[![PyPI](https://img.shields.io/pypi/v/pyvectorhound)](https://pypi.org/project/pyvectorhound)
[![Python 3.8+](https://img.shields.io/badge/Python-3.8%2B-blue)](https://www.python.org)
[![Tests](https://github.com/Mullassery/PyVectorHound/actions/workflows/ci.yml/badge.svg)](https://github.com/Mullassery/PyVectorHound/actions/workflows/ci.yml)
[![License: Proprietary](https://img.shields.io/badge/License-Proprietary-blue.svg)](./LICENSE)

---

## What this actually does today

PyVectorHound does **not** run an embedding model, a reranker, or BM25 for
you, and it is **not** a vector database. It's a diagnostic layer that sits
on top of retrieval results you already have (or that it fetches from your
vector DB) and tells you, with real computed metrics, what's wrong:

- **Embedding-space diagnostics** (isotropy, coverage, distinctiveness) —
  computed by a Rust extension (`pyvectorhound._core`, built via PyO3) from
  the real per-document embeddings your database adapter's `get_embeddings()`
  returns.
- **Vector search accuracy** (precision, recall, MRR) — computed against
  `expected_docs` you supply as ground truth.
- **Root cause + ranked recommendations** — plain-English output combining
  the above.

BM25 (keyword search) and reranker diagnostics are reported as `"UNKNOWN"`:
PyVectorHound doesn't run a keyword-search index or a reranker itself, and
`Diagnosis` doesn't yet accept external BM25/reranker scores as input, so
rather than fabricate a number for a component it can't measure, it says so.

If a component doesn't have enough input to measure honestly (no `adapter`,
fewer than 2 documents with embeddings, or no `expected_docs`), it's
reported as `"UNKNOWN"` with an explanation of what to supply — never a
made-up number.

**PyVectorHound does not bundle an embedding model.** If you want
`Hound.diagnose()` to embed your query text for you, pass it an `embed_fn`
(a thin wrapper around whatever you already use — OpenAI, Cohere,
sentence-transformers, etc.). Without one, pass a precomputed
`query_embedding` per call. It will not silently generate a random vector
and pretend the resulting diagnosis means something.

### New: advanced retrieval ranking (Rust core)

`src/retrieval_ranking.rs` adds a `RetrievalRanker` that combines BM25,
semantic, recency, and diversity signals into a single multi-criteria
ranking, plus cross-encoder-style reranking support. It's compiled into the
native `_core` extension but not yet exposed as a Python-callable function —
if you need it from Python today, treat it as in-progress internal
infrastructure rather than a public API.

### Not yet real (known limitations)

Being upfront about what's still a stub, rather than leaving it to look
finished:

- `ModelComparison` / `Hound.compare_models()` reports real, published
  cost/latency metadata for known models, but has no way to measure quality
  (F1/NDCG) on its own — pass `quality_fn` for real numbers, or it reports
  quality as unmeasured.
- `Hound.compare_metrics()`, `Hound.detect_drift()`, and
  `QualityScorer.trend_analysis()` raise `NotImplementedError` — they have
  no historical data store. Use `Hound.track_metric()` +
  `Hound.get_trend_report()` (backed by the real, tested `TrendAnalyzer`)
  instead.
- Prebuilt wheels on PyPI currently cover macOS (arm64) only. Other
  platforms install from the source distribution, which needs a Rust
  toolchain to build the native extension (`maturin` handles this
  automatically via `pip install`, but it does mean `cargo` must be
  available).
- OpenTelemetry / LangChain / LlamaIndex / MCP integrations, the CLI, and
  the REST server exist and have passing tests but have seen far less
  real-world use than the core `Hound`/`Diagnosis` path above.

---

## Installation

```bash
pip install pyvectorhound
```

Optional vector database clients (only install the one(s) you use):

```bash
pip install pyvectorhound[qdrant]     # Qdrant
pip install pyvectorhound[chroma]     # Chroma
pip install pyvectorhound[milvus]     # Milvus
pip install pyvectorhound[weaviate]   # Weaviate
pip install pyvectorhound[pgvector]   # PostgreSQL + pgvector
```

Requires Python 3.8+.

---

## Quick start: diagnose results you already have

This is the fastest way to try it — no live database or embedding model
needed. `Diagnosis` fetches per-document embeddings for you via a small
adapter object (anything with a `get_embeddings(doc_ids) -> dict` method);
without one, the embedding component honestly reports `"UNKNOWN"` instead
of a fabricated score.

```python
from pyvectorhound import Diagnosis

class InMemoryAdapter:
    """Anything with get_embeddings(doc_ids) works -- swap in your own
    QdrantAdapter/ChromaAdapter/etc., or a wrapper around your pipeline."""
    def __init__(self, embeddings_by_id):
        self._embeddings_by_id = embeddings_by_id

    def get_embeddings(self, doc_ids):
        return {d: self._embeddings_by_id[d] for d in doc_ids if d in self._embeddings_by_id}

results = [
    {"id": "pricing.pdf", "score": 0.91},
    {"id": "onboarding.md", "score": 0.84},
    {"id": "faq.md", "score": 0.79},
]

diagnosis = Diagnosis(
    query="What's your return policy?",
    results=results,
    expected_docs=["returns.pdf", "policy.md"],  # ground truth
    adapter=InMemoryAdapter(my_document_embeddings),
)
diagnosis.analyze()

print(diagnosis.root_cause())
for rec in diagnosis.recommendations():
    print(f"[{rec['priority']}] {rec['action']}")

print(diagnosis.hunt())  # full plain-English report
```

A runnable version (with synthetic embeddings so it works with no setup) is
in [`examples/retrieval_debug.py`](examples/retrieval_debug.py).

## Quick start: diagnose against a live vector database

```python
from pyvectorhound import Hound

hound = Hound(
    db="qdrant",                      # qdrant | chroma | milvus | weaviate | postgres
    endpoint="localhost:6333",
    index_name="documents",
    # PyVectorHound doesn't ship an embedding model -- wrap whatever you use:
    embed_fn=lambda text: my_embedding_client.embed(text),
)

diagnosis = hound.diagnose(
    query="What's your return policy?",
    expected_docs=["returns.pdf", "policy.md"],
    top_k=5,
)
print(diagnosis.hunt())
```

`Hound` connects lazily — constructing it doesn't require a live server,
only calling `diagnose()` (or another querying method) does. `diagnose()`
already passes `self.adapter` into `Diagnosis`, so embedding-space
diagnostics work out of the box against your real database.

---

## Diagnostics it runs

| Component | What it measures | Requires |
|---|---|---|
| **Embedding** | Isotropy, coverage, distinctiveness of the retrieved documents' real embeddings | An `adapter` with `get_embeddings()`, and ≥2 retrieved documents |
| **Vector search** | Precision, recall, MRR | `expected_docs` (ground truth) |
| **BM25 (keyword)** | Not implemented — reports `UNKNOWN` | n/a |
| **Reranker** | Not implemented — reports `UNKNOWN` | n/a |

Every measured component is computed for real from the input you give it;
nothing is guessed when the input isn't there.

---

## Other tools

- `hound.quality_scorer()` — `QualityScorer` for scoring an embedding's
  validity, and (given corpus neighbors via the adapter) real
  isotropy/coverage/distinctiveness against the corpus.
- `hound.benchmark()` — `PerformanceBenchmark` for latency percentiles and
  database/embedding-model comparisons.
- `hound.analyze_trends()` — `TrendAnalyzer` for tracking metrics over time
  and detecting drift, regressions, and anomalies from real tracked values.
- `hound.tracer()` / `hound.replayer()` — capture a retrieval pipeline run
  and replay it under different configurations to compare recall/latency.

See [`examples/`](examples/) for runnable scripts, and
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) / [`docs/GUIDE.md`](docs/GUIDE.md)
for more detail.

---

## Development

```bash
git clone https://github.com/Mullassery/PyVectorHound.git
cd PyVectorHound
pip install maturin
maturin develop --release   # builds the Rust extension in place
pip install -e ".[dev]"
pytest tests/ -v
```

---

## License

Proprietary License — free to use with explicit attribution. See
[LICENSE](LICENSE).

