Metadata-Version: 2.4
Name: qdrant-lens
Version: 0.1.8
Summary: A knowledge browser and audit layer for Qdrant vector databases
Project-URL: Homepage, https://github.com/Shaik-Hamzah123/qdrant-lens
Project-URL: Repository, https://github.com/Shaik-Hamzah123/qdrant-lens
Project-URL: Bug Tracker, https://github.com/Shaik-Hamzah123/qdrant-lens/issues
License: MIT
Requires-Python: >=3.12
Requires-Dist: aiosqlite>=0.22.1
Requires-Dist: apscheduler<4,>=3.10
Requires-Dist: fastapi>=0.139.0
Requires-Dist: pydantic>=2.13.4
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: qdrant-client>=1.18.0
Requires-Dist: rich>=15.0.0
Requires-Dist: sqlalchemy>=2.0.51
Requires-Dist: typer>=0.26.8
Requires-Dist: uvicorn>=0.51.0
Requires-Dist: whenever>=0.10.1
Provides-Extra: all
Requires-Dist: asyncpg>=0.31.0; extra == 'all'
Requires-Dist: chonkie[all]>=1.7.0; extra == 'all'
Requires-Dist: deepagents>=0.6.12; extra == 'all'
Requires-Dist: fastembed>=0.8.0; extra == 'all'
Requires-Dist: langchain-litellm>=0.7.0; extra == 'all'
Requires-Dist: langchain-openai>=1.3.4; extra == 'all'
Requires-Dist: langchain>=1.3.12; extra == 'all'
Requires-Dist: langgraph>=1.2.8; extra == 'all'
Requires-Dist: liteparse>=2.5.0; extra == 'all'
Requires-Dist: neo4j>=6.2.0; extra == 'all'
Requires-Dist: pathspec>=1.1.1; extra == 'all'
Requires-Dist: pdfitdown>=4.2.0; extra == 'all'
Requires-Dist: pdfplumber>=0.11.10; extra == 'all'
Requires-Dist: pillow>=10.0.0; extra == 'all'
Requires-Dist: pymupdf>=1.28.0; extra == 'all'
Requires-Dist: pytesseract>=0.3.13; extra == 'all'
Provides-Extra: chat
Requires-Dist: deepagents>=0.6.12; extra == 'chat'
Requires-Dist: langchain-litellm>=0.7.0; extra == 'chat'
Requires-Dist: langchain-openai>=1.3.4; extra == 'chat'
Requires-Dist: langchain>=1.3.12; extra == 'chat'
Requires-Dist: langgraph>=1.2.8; extra == 'chat'
Provides-Extra: dev
Requires-Dist: httpx; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-asyncio; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: embed
Requires-Dist: fastembed>=0.8.0; extra == 'embed'
Provides-Extra: graph
Requires-Dist: neo4j>=6.2.0; extra == 'graph'
Provides-Extra: ingest
Requires-Dist: chonkie[all]>=1.7.0; extra == 'ingest'
Requires-Dist: fastembed>=0.8.0; extra == 'ingest'
Requires-Dist: liteparse>=2.5.0; extra == 'ingest'
Requires-Dist: pathspec>=1.1.1; extra == 'ingest'
Requires-Dist: pdfitdown>=4.2.0; extra == 'ingest'
Requires-Dist: pdfplumber>=0.11.10; extra == 'ingest'
Requires-Dist: pillow>=10.0.0; extra == 'ingest'
Requires-Dist: pymupdf>=1.28.0; extra == 'ingest'
Requires-Dist: pytesseract>=0.3.13; extra == 'ingest'
Provides-Extra: liteparse
Requires-Dist: liteparse>=2.5.0; extra == 'liteparse'
Provides-Extra: postgres
Requires-Dist: asyncpg>=0.31.0; extra == 'postgres'
Description-Content-Type: text/markdown

# qdrant-lens

![qdrant-lens](assets/asset_1.png)

**A knowledge browser for Qdrant vector databases.**

Qdrant's own dashboard is a point browser — it shows raw vectors and payloads. qdrant-lens is the layer above that: it shows developers *what the team has actually ingested*, organised the way a human thinks about it.

- **Knowledge tree** — browse ingested content by source → document type → document → chunk, not by vector ID
- **Accountability layer** — every upsert and delete is intercepted, signed with the actor identity and timestamp, and written to an append-only audit log; raw Qdrant has no concept of who wrote a point or why — qdrant-lens ensures nothing in the cluster changes without a trace
- **`_lens` schema enforcement** — every point carries a validated metadata namespace; violations are surfaced before bad data reaches the index
- **Sync-aware pipeline** — chunk-hash dedup means re-ingesting a folder only embeds what actually changed; a scroll-diff pattern keeps Qdrant in sync with your filesystem the same way git tracks your working tree
- **Structural graph layer** — document relationships (same source, shared tags, same author) derived purely from payload fields, no Neo4j required
- **Parser-agnostic ingest** — LiteParse (default) handles PDFs, Office docs, images, and Jupyter notebooks natively; falls back to a pdfplumber/pytesseract stack automatically if needed
- **Chat agent** — ask questions about a collection in plain English, get answers backed by live tree/history/search tools; destructive operations require explicit typed confirmation before executing
- **Export & migrate** — dump any collection to portable JSONL/JSON (vectors + metadata) with `lens export`; ready-made recipes for Pinecone, Weaviate, Chroma, and OpenSearch

---

## Documentation

| Guide | Description |
|-------|-------------|
| [Quickstart](docs/quickstart.md) | Install, first run, Day 1/Day 2, env setup |
| [Ingest Pipeline](docs/ingest.md) | File ingestion, parsers, chunking, dedup, hybrid search |
| [Client Reference](docs/client.md) | QdrantLensClient API, audit log, schema, core concepts |
| [Dashboard](docs/dashboard.md) | `lens serve`, UI views, FastAPI routes, CLI commands |
| [Export & Migration](docs/export.md) | Export collections to JSONL/JSON; migration recipes |
| [Parsers](docs/parsers.md) | `liteparse` vs `auto`, system deps, troubleshooting |
| [Testing](docs/testing.md) | Test suite overview, per-module test descriptions |
| [Contributing](docs/contributing.md) | Project structure, module map, adding parsers/routes/fields |

---

## Prerequisites

| Requirement | Version | Notes |
|-------------|---------|-------|
| Python | >= 3.12 | |
| Qdrant | >= 1.18.0 | Self-hosted, Docker, or [Qdrant Cloud](https://cloud.qdrant.io) |
| Docker | any | For `docker compose up` (optional). **Docker Hub login required** for image pulls — run `docker login` first, or download the Qdrant binary from [github.com/qdrant/qdrant/releases](https://github.com/qdrant/qdrant/releases) |
| Tesseract | any | OCR only — `sudo apt install tesseract-ocr` / `brew install tesseract`. Set `TESSDATA_PREFIX` to your tessdata dir (e.g. `/opt/homebrew/share/tessdata`) if OCR fails at runtime |
| LibreOffice | any | Office → PDF conversion (bundled in `[ingest]`) |
| Neo4j | >= 6.0 | Graph RAG only — `docker compose --profile graph up` |

---

## Install

```bash
# Core: client wrapper + dashboard API only
pip install qdrant-lens

# + full ingestion pipeline (PDF parsing, OCR, chunking) — includes LiteParse by default
pip install "qdrant-lens[ingest]"

# + FastEmbed for CLI embeddings and sparse vectors (bm25_enable=True)
pip install "qdrant-lens[embed]"

# Everything
pip install "qdrant-lens[all]"
```

See [docs/quickstart.md](docs/quickstart.md) for `uv`, Docker, and `.env` setup.

---

## Quickstart

### Day 1 — 5 minutes, zero code changes

```bash
pip install qdrant-lens
lens serve --url http://localhost:6333
# Dashboard opens at http://localhost:7367
# Tree renders from your existing collection immediately
```

### Day 2 — two line changes, audit log starts

```python
from qdrant_lens import QdrantLensClient

# Drop-in replacement for QdrantClient — all existing calls unchanged
client = QdrantLensClient(url="http://localhost:6333", api_key="...")

# All upsert/delete calls now validate _lens metadata and write audit events
client.upsert(collection_name="docs", points=[...])

# New lens-only methods
doc_id = client.make_doc_id("github", "repo/auth/login.py")
tree   = client.tree("docs")
events = client.history("docs", last_n=20)
violations = client.validate("docs")
```

### Async

```python
from qdrant_lens import AsyncQdrantLensClient

client = AsyncQdrantLensClient(url="...", api_key="...")
tree = await client.tree("docs")
```

---

See [docs/client.md](docs/client.md) for the full API reference and [docs/ingest.md](docs/ingest.md) for the ingest pipeline.

---

## Compatibility

- **qdrant-client >= 1.18.0 required.** Earlier versions are not supported.
- **Python >= 3.12** (3.10 and 3.11 may work but are untested).

See [COMPATIBILITY.md](COMPATIBILITY.md) for the full version matrix and system dependency notes.

---

## Built with

qdrant-lens stands on the shoulders of several excellent open-source projects:

| Project | Role |
|---------|------|
| [Chonkie](https://www.chonkie.ai/) | Chunking backbone — token-aware `RecursiveChunker`, `SemanticChunker`, and `CodeChunker` strategies |
| [PdfItDown](https://github.com/AstraBert/PdfItDown) | Office and image → PDF conversion in the `auto` parser stack (`.docx`, `.pptx`, `.xlsx`, `.png`, `.jpg`) |
| [LiteParse](https://github.com/run-llama/liteparse) | Default parser — single-library extraction for PDFs, Office docs, images, and more without a LibreOffice dependency |
