Metadata-Version: 2.4
Name: rixdb
Version: 0.1.1.dev0
Summary: Fast, embeddable C++ vector database with Python bindings
Project-URL: Homepage, https://github.com/Pranav1173/RixDB
Project-URL: Repository, https://github.com/Pranav1173/RixDB
Author: Pranav1173
License: Apache-2.0
Keywords: ann,embeddings,hnsw,simd,vector-database
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: C++
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# RixDB

> Fast, embeddable vector database for Python — written in C++.

> ⚠️ **Pre-implementation.** RixDB is in active design — no C++ source exists yet. The code samples and API below reflect the planned design, not a working package. Follow the repo or watch releases to be notified when the first alpha is available.
 
RixDB is intended to be a zero-dependency, SIMD-accelerated vector index with memory-mapped persistence and a clean Python API. It is designed to be embedded directly inside your application — no server, no daemon, no configuration overhead.
 
```python
# Planned API — not yet functional
import rix
 
db = rix.Index.open_or_create(dim=768, metric="cosine", path="./index")
db.upsert(ids, vectors, metadata)
results = db.search(query_vector, top_k=8)
```
 
> RixDB is a ground-up implementation of HNSW-based vector search, built to understand the internals rather than wrap an existing library. It prioritizes correctness, clean API design, and the specific feature set needed for local embedding retrieval — not raw benchmark numbers.
 
---

## Why RixDB

Most local vector search options make you choose between simplicity and performance. ChromaDB is simple but slow on large corpora. FAISS is fast but has no metadata, no incremental updates, and a difficult API. hnswlib is close but general-purpose and lacks a metadata store.

RixDB is built for the specific case of embedding-based retrieval inside a single Python process:

- **Zero runtime dependencies.** Will ship as a single compiled `.so` / `.pyd`. Nothing to install beyond `pip install rix` once released.
- **Fast cold load (design target).** The index is planned to be memory-mapped so the OS page cache handles most I/O, with load time largely independent of index size.
- **SIMD-accelerated queries (planned).** Support for AVX-512, AVX2, SSE4.2, and NEON is intended, with code paths selected automatically at runtime based on CPU features.
- **Incremental updates (planned).** The index is intended to support inserts and deletes without full rebuilds, optimized for indices that change frequently.
- **Metadata-native (design goal).** The design calls for every vector to carry structured metadata stored in a flat binary layout alongside the index, avoiding a separate lookup table.

---

## Installation

> **Package name vs import name:** The PyPI distribution name is `rixdb`
> (what you type in `pip install`). The Python module name is `rix`
> (what you type in `import rix`). This follows the same convention as
> other libraries — `pip install Pillow` / `import PIL`,
> `pip install scikit-learn` / `import sklearn`.

### Planned install (Phase E — production release)

```
pip install rixdb
```

Pre-built wheels are planned for:

| Platform | Architecture |
|---|---|
| Linux | x86\_64, aarch64 |
| macOS | x86\_64, arm64 |
| Windows | x86\_64 |

To build from source (once C++ implementation exists), a C++17 compiler and CMake 3.21+ will be required.

---

## Quick Start

> **Not yet functional.** The following shows the planned API once Phase A is complete. The code samples reflect the final design, not the current state of the repository.

### Create and populate an index

```python
import rix
import numpy as np

# Open or create a persistent index
db = rix.Index.open_or_create(
    dim=768,
    metric="cosine",
    path="./my_index"
)

# Generate some example vectors (replace with real embeddings)
ids = ["doc_0", "doc_1", "doc_2"]
vectors = np.random.rand(3, 768).astype(np.float32)
metadata = [
    {"source": "readme.md", "section": "intro"},
    {"source": "api.md",    "section": "usage"},
    {"source": "guide.md",  "section": "install"},
]

db.upsert(ids, vectors, metadata)
```

### Search

```python
query = np.random.rand(768).astype(np.float32)
results = db.search(query, top_k=3)

for r in results:
    print(r.id, r.score, r.metadata)
```

### Update and delete

```python
# Upsert replaces existing IDs automatically
db.upsert(["doc_0"], new_vector, new_metadata)

# Delete by ID
db.delete(["doc_1"])
```

### Use as a context manager

```python
with rix.Index.open("./my_index") as db:
    results = db.search(query_vector, top_k=5)
```

---

## API Overview

### `rix.Index`

| Method | Description |
|---|---|
| `Index.create(dim, metric, path, params)` | Create a new empty index |
| `Index.open(path)` | Open an existing index |
| `Index.open_or_create(dim, metric, path, params)` | Open if exists, create if not |
| `upsert(ids, vectors, metadata)` | Insert or replace vectors by ID |
| `search(vector, top_k, ef_search, filters)` | Approximate nearest neighbor search |
| `search_batch(vectors, top_k, ef_search)` | Parallel batch search, one result list per query |
| `delete(ids)` | Remove vectors by ID |
| `get(ids)` | Fetch metadata without ANN search |
| `contains(id)` | Check if an ID exists |
| `list_ids(prefix)` | List all stored IDs |
| `compact()` | Force compaction of deleted entries |
| `stats()` | Index statistics and file sizes |
| `close()` | Flush and close (called automatically) |

### `rix.SearchResult`

| Field | Type | Description |
|---|---|---|
| `id` | `str` | External ID provided at insert time |
| `score` | `float` | Similarity score (higher = more similar) |
| `metadata` | `dict` | All metadata fields stored at insert time |

### Metrics

| Value | Distance Function | Use Case |
|---|---|---|
| `"cosine"` | Cosine similarity | Text and code embeddings (default) |
| `"l2"` | Euclidean distance | Image embeddings, some ML models |
| `"ip"` | Inner product | Recommendation systems |

---

## Tuning

RixDB works well at default settings for most use cases. The key parameters are:

| Parameter | Default | Effect |
|---|---|---|
| `M` | 16 | Connections per node. Higher = better recall, more memory |
| `ef_construction` | 200 | Build quality. Higher = better recall, slower build |
| `ef_search` | 50 | Query candidate pool. Higher = better recall, slower query |

Pass custom parameters at creation time:

```python
db = rix.Index.create(
    dim=768,
    metric="cosine",
    path="./index",
    params={"M": 32, "ef_construction": 400}
)
```

### Built-in profiles

```python
# High recall — best accuracy, higher memory
params = rix.profiles.high_recall

# Low memory — reduced RAM, slightly lower recall
params = rix.profiles.low_memory

# Fast build — quicker indexing, default query performance
params = rix.profiles.fast_build
```

---

## Benchmarks

> **Note — RixDB is not yet implemented.** The numbers below are conservative design targets, not measured results. RixDB is an initial, first-principles implementation — these targets reflect what a correct, well-optimized implementation of HNSW with AVX2 SIMD and mmap persistence *should* be able to achieve, not what has been verified. Benchmarks will be measured and published after Phase B is complete. Until then, treat all numbers here as aspirational.

Design targets on a mid-range developer laptop (Intel Core i7, AVX2, no GPU).
Index dimension: 768. Metric: cosine. Default parameters (M=16, ef\_search=50).

| Corpus Size | Query P95 (target) | Build Time (target) | Cold Load (target) | Recall\@10 (target) |
|---|---|---|---|---|
| 10K vectors | < 5ms | < 30s | < 100ms | ≥ 0.95 |
| 100K vectors | < 20ms | < 5min | < 100ms | ≥ 0.95 |
| 500K vectors | < 50ms | < 30min | < 100ms | ≥ 0.95 |

Feature comparison with alternatives (competitor numbers are from their published benchmarks — RixDB column is targets only):

| Library | Query P95 | Cold Load | Incremental Updates | Metadata |
|---|---|---|---|---|
| **RixDB** (target) | **< 20ms** | **< 100ms** | **✓** | **✓** |
| ChromaDB | ~25ms | ~400ms | ✓ | ✓ |
| hnswlib | ~4ms | ~180ms | ✗ | ✗ |
| FAISS (IVF) | ~3ms | ~200ms | ✗ | ✗ |

RixDB's query latency target is intentionally conservative. hnswlib is written by the authors of the HNSW paper and is extensively optimized. Matching it on the first implementation attempt is not a realistic expectation. The value RixDB adds over hnswlib is incremental updates and a built-in metadata store — not raw query speed.

---

## Architecture

RixDB is built on three core components:

**HNSW index** — Hierarchical Navigable Small World graph for O(log n) approximate nearest neighbor search. Parameters M, ef\_construction, and ef\_search control the recall/speed/memory tradeoff.

**SIMD distance kernel** — Runtime CPU detection selects the best available path: AVX-512, AVX2, SSE4.2, NEON, or scalar fallback. Cosine similarity reduces to a dot product after L2 normalization at insert time.

**mmap persistence** — The index and metadata live in memory-mapped binary files. The OS page cache manages physical memory. Cold load time is independent of index size.

Full architecture documentation is in [`docs/architecture.md`](https://github.com/Pranav1173/RixDB/blob/main/docs/architecture.md).

---

## Building From Source

> **Not yet possible.** No C++ source exists yet. The following describes the planned build process for when Phase A implementation begins.

Requirements (planned): C++17 compiler (GCC 10+, Clang 12+, MSVC 2019+), CMake 3.21+, Python 3.11+.

```
git clone https://github.com/Pranav1173/RixDB
cd RixDB
pip install -e ".[dev]"
```

To run the C++ unit tests:

```
cmake -B build -DRIX_BUILD_TESTS=ON
cmake --build build
./build/rix_test
```

To run the Python tests:

```
pytest tests/python/
```

---

## Contributing

Contributions are welcome. Please read [`CONTRIBUTING.md`](https://github.com/Pranav1173/RixDB/blob/main/CONTRIBUTING.md) before opening a pull request.

For significant changes — new features, API changes, algorithm modifications — open an issue first to discuss the design. The architecture document is the reference for all design decisions.

Areas where contributions are especially welcome:

- Additional SIMD kernels (AVX-512 path, improved NEON)
- Scalar quantization (int8 vector support)
- Windows ARM64 support
- Language bindings beyond Python

---

## License

Apache 2.0 — see [`LICENSE`](https://github.com/Pranav1173/RixDB/blob/main/LICENSE).

---

## Status

RixDB is currently in **pre-implementation planning**. The architecture is fully documented and the repository is set up, but no C++ code has been written yet. Development begins at Phase A — a correct scalar HNSW implementation with Python bindings.

Planned release milestones:

```
Phase A  →  0.1.0-alpha   Correct scalar HNSW + Python bindings
Phase B  →  0.2.0-alpha   SIMD + mmap + verified benchmarks
Phase C  →  0.3.0-alpha   Incremental updates + compaction
Phase D  →  0.4.0-beta    Parallel batch insert + thread safety
Phase E  →  1.0.0         PyPI publication + pre-built wheels
```

Follow the repository or watch releases to be notified when the first alpha is available.

Current version: `0.1.0.dev0` (name reservation only)