Metadata-Version: 2.4
Name: chassisdb
Version: 0.8.0
Summary: Python bindings for Chassis, an embedded vector index for local semantic search
Author: Chassis Contributors
License-Expression: MIT OR Apache-2.0
Project-URL: Homepage, https://github.com/tanvincible/chassis
Project-URL: Documentation, https://tanvincible.github.io/chassis/python/
Project-URL: Repository, https://github.com/tanvincible/chassis
Project-URL: Issues, https://github.com/tanvincible/chassis/issues
Keywords: vector,search,embedding,ann,hnsw
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE-MIT
License-File: LICENSE-APACHE
Requires-Dist: numpy>=1.26
Provides-Extra: dev
Requires-Dist: pytest>=9.0; extra == "dev"
Requires-Dist: pytest-cov>=7.0; extra == "dev"
Requires-Dist: black>=26.1; extra == "dev"
Requires-Dist: ruff>=0.14.14; extra == "dev"
Requires-Dist: mypy>=1.19.1; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs<2,>=1.6.0; extra == "docs"
Requires-Dist: mkdocs-material>=9.7.0; extra == "docs"
Requires-Dist: mkdocstrings[python]>=1.0; extra == "docs"
Dynamic: license-file

# chassisdb

Python bindings for Chassis, an embedded vector index for local semantic search: one file, in your process, no server.

## Features

- **Zero-copy operations**: Direct memory access via ctypes
- **NumPy integration**: Native support for NumPy arrays
- **Type hints**: Full type annotations for IDE support
- **Thread-safe**: Search from many threads while another adds or deletes; writes run one at a time
- **Pythonic API**: Context managers, properties, and familiar patterns

## Installation

```bash
pip install chassisdb
```

Import it as `chassisdb`. Wheels are built for Python 3.11 and later on Linux (x86-64 and ARM),
macOS on Apple silicon and Windows (x86-64). Elsewhere, or from a
checkout of the repository, `pip install .` in this directory builds it, which needs Rust.

### Development Installation

```bash
# Build the library, then install with dev dependencies
cargo build --release -p chassis-ffi
pip install -e ".[dev]"

# Run tests
pytest

# Type checking
mypy chassisdb

# Formatting
black chassisdb tests
ruff check chassisdb tests
```

## Quick Start

```python
import numpy as np
from chassisdb import VectorIndex

# Create or open an index
index = VectorIndex("embeddings.chassis", dimensions=768)

# Add vectors
vectors = np.random.rand(1000, 768).astype(np.float32)
for i, vec in enumerate(vectors):
    vector_id = index.add(vec)
    print(f"Added vector {vector_id}")

# Flush to disk
index.flush()

# Search
query = np.random.rand(768).astype(np.float32)
results = index.search(query, k=10)

for result in results:
    print(f"ID: {result.id}, Distance: {result.distance:.6f}")

# Index info
print(f"Total vectors: {len(index)}")
print(f"Dimensions: {index.dimensions}")
print(f"Empty: {index.is_empty()}")
```

## Advanced Usage

### Custom HNSW Parameters

```python
from chassisdb import VectorIndex, IndexOptions

# Configure HNSW parameters
options = IndexOptions(
    max_connections=32,      # Higher = better recall, more memory
    ef_construction=400,     # Higher = better index, slower build
    ef_search=100,           # Higher = better search, slower queries
    metric="cosine",         # "euclidean" (default) or "cosine"; fixed at creation
    precision="half",        # "full" (default) or "half": 16-bit floats, half the file; fixed at creation
)

index = VectorIndex("tuned.chassis", dimensions=768, options=options)
```

### Context Manager

```python
with VectorIndex("vectors.chassis", dimensions=128) as index:
    index.add([0.1] * 128)
# Flushed and closed here
```

Leaving the block flushes, unless an exception is leaving it, then closes the index. Closing
without a flush, with `close()` or by garbage collection, discards what was added or deleted since
the last `flush()`, as a crash would.

### Batch Operations

```python
import numpy as np

vectors = np.random.rand(10000, 768).astype(np.float32)
queries = np.random.rand(100, 768).astype(np.float32)

with VectorIndex("large.chassis", dimensions=768) as index:
    # Added and linked on every core
    index.add_batch(vectors)
    index.flush()

    # One call for every query: ids and distances are (100, 10) arrays
    ids, distances = index.search_batch(queries, k=10)
```

### NumPy Integration

```python
import numpy as np

# From NumPy array
vec = np.array([0.1, 0.2, 0.3], dtype=np.float32)
index.add(vec)

# From list
index.add([0.1, 0.2, 0.3])

# From tuple
index.add((0.1, 0.2, 0.3))

# All are automatically converted to float32
```

### Error Handling

```python
from chassisdb import VectorIndex, DimensionMismatchError, IndexLockedError

index = VectorIndex("vectors.chassis", dimensions=128)
try:
    index.add([0.1] * 64)
except DimensionMismatchError as e:
    print(e)
    # The vector has 64 components, but this index holds vectors of 128
    # help: make the vector with the same model as the index's vectors; this index was created for 128 dimensions

try:
    VectorIndex("vectors.chassis", dimensions=128)
except IndexLockedError:
    # One writer at a time; a reader searches while it writes.
    reader = VectorIndex("vectors.chassis", dimensions=128, read_only=True)
```

## API Reference

### `VectorIndex`

Main class for interacting with Chassis indexes.

#### Constructor

```python
VectorIndex(path: str | Path, dimensions: int, options: IndexOptions | None = None, read_only: bool = False)
```

To reopen an existing index, pass the metric, precision and `max_connections` it was created
with; others raise `OptionsMismatchError`, which names them. With `read_only=True` it only
searches, takes no lock, and takes the metric and precision from the file (`max_connections` has
to match); any number can search while one writer writes, in this process or others.

#### Methods

- **`add(vector: Sequence[float] | ndarray, id: int | None = None) -> int`**  
  Add a vector under `id` (0 to 2**64 - 2; `IdInUseError` if a live vector has it), or else under
  one past the largest id so far. Returns the id.

- **`add_batch(vectors: ndarray, ids: Iterable[int] | None = None) -> ndarray`**  
  Add a `(count, dimensions)` array at once, linking it on every core: much faster than `add` in a
  loop. Returns the ids. All or nothing: if any vector or id is rejected, none is added.

- **`delete(id: int) -> bool`**  
  Delete the vector with this id. Returns whether there was one. Search stops returning it at
  once; the delete is durable after the next `flush()`. To replace a vector, delete it and add
  the new one under the same id: the next flush saves both or neither.

- **`get(id: int) -> ndarray | None`**  
  The vector with this id, or `None` if no live vector has it. It comes back as the index keeps
  it: scaled to length 1 in a cosine index, and rounded in half precision.

- **`id in index`**  
  Whether a live vector has this id.

- **`ids() -> ndarray`**  
  The ids of the live vectors, in the order they were added.

- **`search(query: Sequence[float] | ndarray, k: int = 10, allowed: Iterable[int] | ndarray | None = None, excluded: Iterable[int] | ndarray | None = None) -> List[SearchResult]`**  
  Search for k nearest neighbors. Returns sorted list of results. With `allowed`, only those ids
  can be returned, e.g. `allowed=[row[0] for row in db.execute("SELECT id FROM docs WHERE owner = ?", (user,))]`;
  when walking the graph would cost more, as when few vectors match, every vector is checked
  instead and the results are exact. With `excluded`, those ids are never returned, as for "more
  like this": `index.search(index.get(42), k=10, excluded=[42])`.

- **`search_batch(queries: ndarray, k: int = 10) -> tuple[ndarray, ndarray]`**  
  Search for the k nearest neighbors of each row of a `(count, dimensions)` array, in one call:
  faster than `search` in a loop. Returns `(ids, distances)`, two `(count, k)` arrays, nearest
  first; a row with fewer than k results is filled out with id `2**64 - 1` and distance `inf`.

- **`flush() -> None`**  
  Flush changes to disk. Call after batch insertions.

- **`warm() -> None`**  
  Read the index into memory on another thread, so that searches stop waiting for the disk while
  its file is not in memory yet, as after a reboot. Returns at once; searches go on meanwhile.
  `IndexOptions(warm=True)` does the same from the moment the index is opened.

- **`compact() -> None`**  
  Rewrite the index without its deleted vectors and with a rebuilt graph, reclaiming their space.
  Ids don't change. Takes as long as building the index and needs disk for a second copy.
  Deletes cost speed, not recall: with half the vectors deleted, searches ran at about 70% of
  their speed, so compacting once `deleted_count()` nears `len()` is a good rule. The file
  shrinks by whole segments, which double as an index grows, so only when the live vectors fit in
  fewer of them.

- **`close() -> None`**  
  Close the index and free resources. Adds and deletes since the last `flush()` are discarded, as
  after a crash; leaving a `with` block flushes first, unless an exception is leaving it.

- **`__len__() -> int`**  
  Get number of vectors in the index.

- **`deleted_count() -> int`**  
  Deleted vectors the file still holds, until `compact()` removes them.

- **`is_empty() -> bool`**  
  Check if index is empty.

#### Properties

- **`dimensions: int`** - Number of dimensions per vector
- **`metric: str`** - `"euclidean"` or `"cosine"`, as the index was created; opening with
  `read_only=True` and explicit `options` naming another raises `OptionsMismatchError`
- **`precision: str`** - `"full"` or `"half"`, as the index was created
- **`path: Path`** - Path to the index file
- **`options: IndexOptions`** - HNSW configuration

### `IndexOptions`

Configuration for HNSW algorithm.

```python
@dataclass
class IndexOptions:
    max_connections: int = 16      # M parameter
    ef_construction: int = 200     # Build-time search quality
    ef_search: int = 50            # Query-time search quality
    metric: str = "euclidean"      # or "cosine": 1 - cosine similarity, unit-length storage
    precision: str = "full"        # or "half": vectors kept as 16-bit floats, half the file and memory
    huge_pages: bool = False       # Linux: keep the vectors on 2 MB pages (faster search on a large index)
    warm: bool = False             # read the index into memory in the background from the start
```

### `SearchResult`

Search result with ID and distance.

```python
@dataclass
class SearchResult:
    id: int          # Vector ID in index
    distance: float  # Distance to query (lower = closer)
```

### Exceptions

Each says what was wrong, with the value, and on a line starting `help:` what to do instead. Each
is a `ChassisError` and also the built-in exception it is a kind of, so `except ValueError`
catches a bad argument. The [errors reference](https://github.com/tanvincible/chassis/blob/main/docs/src/guide/errors.md)
lists their causes and fixes.

- **`InvalidArgumentError`** (`ValueError`) - an argument out of range: dimensions, an option, an
  id, a vector with NaN or infinity. A wrong type is a plain `TypeError`.
- **`DimensionMismatchError`** (`ValueError`) - a vector or query of other dimensions than the index's
- **`OptionsMismatchError`** (`ValueError`) - reopened with another metric, precision or
  `max_connections`
- **`IndexNotFoundError`** (`FileNotFoundError`) - no index at the path, for `read_only=True`
- **`NotAnIndexError`** (`ValueError`) - the file isn't a Chassis index
- **`IdInUseError`** (`ValueError`) - the id is taken; delete it first to replace it
- **`IndexLockedError`** - another writer has the index open
- **`ReadOnlyError`** - a `read_only=True` index asked to write
- **`CorruptIndexError`**, **`IndexFullError`**, **`InvalidPathError`** (`OSError`) - a damaged
  file, a full index, an operating-system error

## Thread Safety

Every method is safe to call from any thread. Searches run concurrently (ctypes releases the GIL
during each call); `add()`, `delete()` and `flush()` take the index's write lock, so they run one
at a time and searches wait for them. Don't call `close()` while other threads still use the index.

Other processes can search the file while one process writes it: open it with
`VectorIndex(path, dimensions, read_only=True)`. Such an index takes no lock, sees the writer's
newest `flush()` on every search, runs one search at a time, and raises `ReadOnlyError` on writes.
A flush becomes visible just before its final fsync, so one that then fails, or is lost to power
loss, may already have been seen.

```python
from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor() as pool:
    results = list(pool.map(lambda q: index.search(q, k=10), queries))
```

## Performance Tips

1. **Batch inserts before flushing:**
   ```python
   for vec in many_vectors:
       index.add(vec)
   index.flush()  # Once at the end
   ```

2. **Use NumPy arrays:**
   ```python
   # Faster (zero-copy)
   vec = np.array([0.1] * 768, dtype=np.float32)
   
   # Slower (creates copy)
   vec = [0.1] * 768
   ```

3. **Tune HNSW parameters:**
   - Increase `ef_construction` for better index quality
   - Increase `ef_search` for better search recall
   - Increase `max_connections` for better graph connectivity

4. **Use context managers:**
   ```python
   with VectorIndex(...) as index:
       # Automatic cleanup
   ```

## Library Location

chassisdb looks for `libchassis_ffi` in this order:

1. `CHASSIS_LIB_PATH` environment variable
2. Next to the Python package
3. `../target/release` (development)
4. System library paths

To set a custom location:
```bash
export CHASSIS_LIB_PATH=/path/to/lib
python your_script.py
```

## Examples

See the `examples/` directory for complete examples:
- `notes_search.py` - Notes in SQLite, their vectors in Chassis: search, "more like this", deletes
- `basic_usage.py` - Simple add and search
- `batch_insert.py` - Efficient bulk loading

## Testing

```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=chassisdb --cov-report=html

# Run specific tests
pytest tests/test_index.py -k delete
```

## License

Licensed under either of:

- Apache License, Version 2.0 ([LICENSE-APACHE](https://github.com/tanvincible/chassis?tab=Apache-2.0-1-ov-file))
- MIT License ([LICENSE-MIT](https://github.com/tanvincible/chassis?tab=MIT-2-ov-file))

at your option.

## Contributing

See [CONTRIBUTING.md](https://github.com/tanvincible/chassis?tab=contributing-ov-file) for development guidelines.
