Metadata-Version: 2.4
Name: numpy-vector-store
Version: 0.2.0
Summary: A fast, lightweight, and zero-setup in-memory vector store powered by NumPy
Project-URL: Homepage, https://github.com/tvanreenen/numpy-vector-store
Project-URL: Repository, https://github.com/tvanreenen/numpy-vector-store
Project-URL: Issues, https://github.com/tvanreenen/numpy-vector-store/issues
Author: Tim VanReenen
License: MIT
License-File: LICENSE
Keywords: embeddings,numpy,search,store,vector
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.10
Requires-Dist: numpy>=1.20.0
Description-Content-Type: text/markdown

# NumPy Vector Store

A fast, lightweight, zero-setup in-memory vector store powered by NumPy.

- **Tiny local vector search** for projects that do not need a vector database
- **Fast exact cosine search** using vectorized NumPy operations
- **Simple typed API** returning `VectorHit(index, value, metadata)`
- **Composable filtering** by passing prefiltered row indexes with `within_rows`
- **Portable persistence** as trusted local `.npz` files with `vectors` + `metadata`
- **No framework opinions**: bring your own embeddings, chunking, async, and metadata model

## Why?

This library is purpose-built for small to medium-scale vector search tasks and
offers a simple alternative to heavyweight vector databases when you do not need
network services, indexing infrastructure, ingestion pipelines, or domain-specific
metadata filtering.

## When/Where?

Below are benchmark results for cosine similarity search to help you assess its
suitability for your use case.

| Embedding Type | Dimensions | ~5ms | ~25ms | ~100ms | ~500ms |
|----------------|------------|------|--------|---------|---------|
| **Sentence Transformers** | 384 | 1K vectors<br/>1.5MB | 10K vectors<br/>15MB | 100K vectors<br/>147MB | 500K vectors<br/>732MB |
| **OpenAI Small** | 1536 | 500 vectors<br/>3MB | 5K vectors<br/>29MB | 25K vectors<br/>147MB | 100K vectors<br/>586MB |
| **OpenAI Large** | 3072 | 200 vectors<br/>2MB | 2.5K vectors<br/>29MB | 5K vectors<br/>59MB | 25K vectors<br/>293MB |

*Benchmarks performed on Apple M2 hardware.*

## Installation

```bash
uv add numpy-vector-store
```

## Quick Start

```python
import numpy as np
from numpy_vector_store import VectorStore

store = VectorStore[dict[str, str]](dimensions=3)

store.add(
    vectors=np.array([
        [1.0, 0.0, 0.0],
        [0.0, 1.0, 0.0],
        [0.0, 0.0, 1.0],
    ]),
    metadata=[
        {"title": "x-axis"},
        {"title": "y-axis"},
        {"title": "z-axis"},
    ],
)

hits = store.cosine_search(
    query=np.array([0.9, 0.1, 0.0]),
    top_k=2,
)

for hit in hits:
    print(f"{hit.metadata['title']}: {hit.value:.3f}")
```

`metadata` is an opaque row payload returned with hits. It can be a dict,
dataclass, string, integer row ID, or any other Python object that fits your
application.

## Prefiltering

The store does not implement a metadata query language. To filter by metadata,
produce row indexes first, then pass them with `within_rows`.

```python
rows = [
    i
    for i, metadata in enumerate(store.metadata)
    if metadata["title"].startswith("x")
]

hits = store.cosine_search(query, top_k=10, within_rows=rows)
```

For structured NumPy metadata, use NumPy to produce the row indexes:

```python
metadata_table = np.array(
    [
        ("intro", "A", 2024),
        ("setup", "A", 2023),
        ("guide", "B", 2024),
    ],
    dtype=[("title", "U20"), ("product", "U10"), ("year", "i4")],
)

store = VectorStore[int](dimensions=3)
store.add(vectors, metadata=np.arange(len(metadata_table)))

mask = (metadata_table["product"] == "A") & (metadata_table["year"] >= 2024)
rows = np.flatnonzero(mask)

hits = store.cosine_search(query, within_rows=rows)

for hit in hits:
    row = metadata_table[hit.metadata]
    print(row["title"], hit.value)
```

## Persistence

Pass a `file_path` and call `save()` / `load()` explicitly:

```python
store = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
store.add(embeddings, metadata)
store.save()

loaded = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
loaded.load()
```

Context manager usage auto-saves on exit:

```python
with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
    store.add(embeddings, metadata)
```

Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
arrays. Vectors are normalized when added or loaded, and similarity search only
normalizes the query vector. Loading validates shape, dimensions, row counts, and
zero-norm vectors. It also uses `allow_pickle=True` for flexible Python metadata
payloads, so only load files generated by your own application or another trusted
local process. Loading untrusted `.npz` files is not a supported security model.

## Migrating from 0.1

The preferred 0.2 API is `add(...)` and `cosine_search(...)`.

The 0.1 methods remain temporarily available, but emit `DeprecationWarning` and
will be removed in a future 0.x release:

```python
store.add_vectors(vectors_2d, metadata_array)
results = store.search(query, top_k=3, score_cutoff=0.5)
for index, value, metadata in results:
    ...
```

Legacy `search(...)` keeps returning tuples. New `cosine_search(...)` returns
`VectorHit` objects.

`metadata_schema` was removed. For vectorized metadata filtering, keep metadata
in a sidecar table and pass matching row indexes with `within_rows`.

## Contributing

```bash
git clone https://github.com/tvanreenen/numpy-vector-store.git
cd numpy-vector-store
uv sync --frozen --group dev
```

Before submitting a pull request:

1. Run `uv run ruff check`
2. Run `uv run ruff format --check`
3. Run `uv run mypy src/`
4. Run `uv run pytest`

## License

MIT License - see [LICENSE](LICENSE) file for details.
