Metadata-Version: 2.2
Name: ragkit-vectorlite
Version: 0.1.0
Summary: A tiny, dependency-free in-memory vector store for prototyping RAG / semantic search.
Author-email: Meet2147 <meetjethwa3@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Meet2147/pythonLibraries/tree/main/vectorlite
Project-URL: Repository, https://github.com/Meet2147/pythonLibraries
Project-URL: Issues, https://github.com/Meet2147/pythonLibraries/issues
Keywords: vector,embeddings,rag,semantic-search,cosine,mmr,genai,llm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

<p align="center">
  <img src="https://raw.githubusercontent.com/Meet2147/pythonLibraries/main/vectorlite/assets/logo.png" alt="vectorlite" width="460">
</p>

<p align="center">
  <a href="https://pypi.org/project/ragkit-vectorlite/"><img src="https://img.shields.io/pypi/v/ragkit-vectorlite.svg" alt="PyPI"></a>
  <img src="https://img.shields.io/pypi/pyversions/ragkit-vectorlite.svg" alt="Python versions">
  <img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License: MIT">
</p>

# vectorlite

A tiny, dependency-free in-memory vector store for prototyping RAG and semantic search — no numpy, no FAISS, no Pinecone.

> Part of the **ragkit** suite. Install with `pip install ragkit-vectorlite`, then `import vectorlite`.

Every prototype seems to start by re-implementing cosine similarity and a little vector store from scratch. `vectorlite` is that little store, done once, correctly. It's pure standard library (Python 3.8+), so you can drop it into a notebook or a script and start querying embeddings in seconds.

## Install

```bash
pip install ragkit-vectorlite
```

Local development (from `vectorlite/`):

```bash
pip install -e .
```

## Quick Start

```python
from vectorlite import VectorStore

# metric defaults to "cosine"; dim is inferred from the first vector
store = VectorStore(metric="cosine")

store.add("doc1", [0.1, 0.2, 0.9], metadata={"topic": "space"}, document="Rockets and orbits.")
store.add("doc2", [0.9, 0.1, 0.0], metadata={"topic": "cooking"}, document="How to sear a steak.")
store.add("doc3", [0.15, 0.25, 0.85], metadata={"topic": "space"}, document="Satellites and telescopes.")

results = store.query([0.12, 0.2, 0.88], top_k=2)
for r in results:
    print(r.id, round(r.score, 4), r.document)
```

Each result is a `SearchResult` dataclass:

```python
SearchResult(id, score, vector, metadata, document)
```

Results are always sorted best-first.

## Metrics

Pass `metric=` when constructing the store:

| Metric        | Meaning                                  | Ranking                                   |
|---------------|------------------------------------------|-------------------------------------------|
| `"cosine"`    | Cosine similarity in `[-1, 1]` (default) | Higher is better                          |
| `"dot"`       | Raw dot product                          | Higher is better                          |
| `"euclidean"` | L2 distance                              | Closer is better (ranked internally by negative distance) |

For `euclidean`, "higher score means closer" — the store handles the sign for you, so results still come back best-first. The `score` on each result reflects the negative distance in that mode.

The standalone functions are available too, operating on plain lists of floats:

```python
from vectorlite import cosine_similarity, dot, euclidean_distance

cosine_similarity([1, 0], [1, 0])   # 1.0
cosine_similarity([1, 0], [0, 1])   # 0.0  (orthogonal)
cosine_similarity([0, 0], [1, 1])   # 0.0  (zero vector handled gracefully)
```

Mismatched dimensions raise `ValueError`.

## Metadata filtering

Pass a `filter` callable to restrict candidates *before* scoring. It receives each item's metadata dict and returns `True` to keep it:

```python
space_only = store.query(
    [0.12, 0.2, 0.88],
    top_k=5,
    filter=lambda md: md is not None and md.get("topic") == "space",
)
```

Only items whose metadata passes the filter are scored and ranked.

## MMR: diversity-aware results

Plain top-k similarity can return several near-duplicates of the same best match. **Maximal Marginal Relevance (MMR)** re-ranks results to balance relevance to your query against diversity among the results themselves.

```python
results = store.query_mmr(
    query_vector,
    top_k=3,
    fetch_k=20,       # pull this many by raw similarity first
    lambda_mult=0.5,  # 1.0 = pure relevance, 0.0 = pure diversity
)
```

How it works: `vectorlite` fetches `fetch_k` candidates by similarity, then greedily builds the result set. At each step it picks the candidate maximizing

```
lambda_mult * relevance(query, candidate)
    - (1 - lambda_mult) * max_similarity(candidate, already_selected)
```

So if you've already selected item **A**, a near-duplicate **A'** gets penalized for being too similar to **A**, and a different-but-still-relevant item **B** can win instead. Lower `lambda_mult` favors diversity; `lambda_mult=1.0` reduces to ordinary relevance ranking. Diversity is always measured with cosine similarity between candidate vectors.

## Save and load

The whole store — items, metric, and dim — serializes to plain JSON:

```python
store.save("mystore.json")

from vectorlite import VectorStore
store = VectorStore.load("mystore.json")
```

## Other operations

```python
len(store)              # number of items
"doc1" in store         # membership test
store.get("doc1")       # SearchResult (score 0.0) or None
store.delete("doc1")    # True if it existed, else False
store.ids()             # list of all ids
store.add_many([
    {"id": "x", "vector": [0.1, 0.2, 0.3], "metadata": {"k": "v"}},
    ("y", [0.4, 0.5, 0.6]),                       # (id, vector)
    ("z", [0.7, 0.8, 0.9], {"k": "v"}, "a doc"),  # (id, vector, metadata, document)
])
```

Adding an existing id overwrites the previous item.

## Prototype scale — and swapping in FAISS later

`vectorlite` does a brute-force O(n) scan on every query. That is genuinely fine for prototyping and small apps — think **up to ~10k–100k vectors**, where a full scan still returns in well under a second. There's no index, no approximate search, and no on-disk memory mapping.

When your corpus grows past that, or you need sub-millisecond latency at scale, graduate to a real vector database or ANN library — [FAISS](https://github.com/facebookresearch/faiss), [Chroma](https://www.trychroma.com/), [Qdrant](https://qdrant.tech/), or [Pinecone](https://www.pinecone.io/). The API here (`add`, `query`, metadata filtering, MMR) intentionally mirrors those tools, so porting your prototype is mostly a matter of swapping the store — your surrounding code stays the same.

## License

MIT
