Metadata-Version: 2.4
Name: lsmvec-client
Version: 0.2.0
Summary: Python client for the LSM-Vec vector database HTTP API
Author: LSM-Vec
License: Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Provides-Extra: numpy
Requires-Dist: numpy>=1.20; extra == "numpy"
Provides-Extra: embed
Requires-Dist: fastembed>=0.7.3; extra == "embed"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: fastembed>=0.7.3; extra == "dev"

# lsmvec-client — Python client for LSM-Vec

A thin, dependency-free Python client for the LSM-Vec vector database
HTTP API. Uses only the Python standard library; `numpy` is optional
(a convenience for `bulk_build`).

## Install

```bash
pip install lsmvec-client            # core, zero dependencies
pip install lsmvec-client[numpy]     # + numpy for bulk_build convenience
pip install lsmvec-client[embed]     # + FastEmbed for client-side text embedding
```

Or run straight from the repo without installing:

```python
import sys; sys.path.insert(0, "sdk/python")
from lsmvec_client import Client
```

## Quickstart

```python
from lsmvec_client import Client

client = Client(
    api_key="sk-live-...",                 # sent as Bearer token
    base_url="https://api.lsmvec.com",     # or http://localhost:8000 for local
)

# Insert with optional metadata
client.insert(1, [0.10, 0.20, 0.30, ...], metadata={"title": "intro"})

# Search
hits = client.search([0.10, 0.20, 0.30, ...], k=10)
for h in hits:
    print(h.id, h.distance)

# Filtered search (metadata predicate, same syntax as the HTTP API)
hits = client.search(
    [0.10, 0.20, ...], k=10,
    filter={"$and": [{"category": {"$eq": "docs"}}]},
)
```

## Bulk build (initial load)

The fastest way to populate a **new, empty** database. Builds the
whole index in memory (RNN-Descent) and writes it in one pass —
2-3× faster than per-vector inserts and higher recall. Initial-load
only; the DB must be empty.

```python
import numpy as np
from lsmvec_client import Client

client = Client(base_url="http://localhost:8000")

vectors = np.random.rand(100_000, 128).astype(np.float32)
report = client.bulk_build(vectors, threads=4)
print(report)   # {'n': 100000, 'elapsed_ms': ..., 'vectors_per_sec': ..., 'threads': 4}
```

`bulk_build` also accepts a plain list of equal-length float lists
(no numpy required):

```python
rows = [[0.1, 0.2, 0.3, 0.4], [0.5, 0.6, 0.7, 0.8], ...]
client.bulk_build(rows)
```

Pass `payloads=` (a list of `n` dicts/`None`s, positionally mapped to the
auto-assigned ids `0..n-1`) to attach metadata during the build; the report
then includes `payloads_written`. If the vectors build but a payload write
fails, the raised `LSMVecError.details` carries `rebuild_required: true`.

```python
report = client.bulk_build(vectors, payloads=[{"source": "docs"}] * len(vectors))
```

For incremental updates on an already-built index, use `insert()`,
`upsert()`, or `insert_batch()` instead — `bulk_build` rejects a non-empty DB.

## Batch insert (incremental)

`insert_batch` writes many already-embedded vectors over a non-empty index,
keeping your own ids and payloads. It consumes an iterable lazily
`chunk_size` at a time (a generator is fine) and POSTs each chunk in one
request:

```python
items = [(i, vec, {"source": "docs"}) for i, vec in enumerate(vectors)]
n = client.insert_batch(items, chunk_size=1000)   # -> total inserted
```

Items are `(id, vector)` or `(id, vector, metadata)`. On a mid-batch engine
error, `LSMVecError.details` carries `inserted` / `failed_index` so you can
resume.

## Text ingestion (client-side embedding)

With the `[embed]` extra, the client can chunk text, embed it, and store the
vectors — embedding runs **in the client**; the server only stores and
searches vectors.

```python
from lsmvec_client import Client
from lsmvec_client.ingest import LocalEmbedder, ingest_text, search_text

client   = Client(base_url="http://localhost:8000")
embedder = LocalEmbedder("BAAI/bge-small-en-v1.5")   # 384-d, runs locally via FastEmbed

ids  = ingest_text(client, "doc-1", "Refunds take up to 30 days.", embedder, start_id=0)
hits = search_text(client, "how long do refunds take?", embedder, k=3)
print(hits[0].text)
```

`LocalEmbedder` needs the `[embed]` extra; `ProviderEmbedder` calls an
OpenAI-compatible embeddings endpoint and is stdlib-only. The index dimension
must equal the embedder's output dimension.

## API

| Method | HTTP | Notes |
|---|---|---|
| `insert(id, vector, metadata=None)` | `POST /v1/vectors` | metadata is any JSON object |
| `upsert(id, vector)` | `PUT /v1/vectors/:id` | insert-or-replace vector |
| `get(id) -> dict` | `GET /v1/vectors/:id` | `{"id", "vector"}` |
| `delete(id)` | `DELETE /v1/vectors/:id` | |
| `get_payload(id) -> dict` | `GET /v1/vectors/:id/payload` | |
| `set_payload(id, payload)` | `PUT /v1/vectors/:id/payload` | replace |
| `merge_payload(id, partial)` | `PATCH /v1/vectors/:id/payload` | RFC 7396 merge |
| `search(vector, k=10, ef_search=None, filter=None) -> [SearchResult]` | `POST /v1/search` | |
| `insert_batch(items, chunk_size=1000) -> int` | `POST /v1/vectors/batch` | incremental; keeps your ids + payloads |
| `bulk_build(vectors, dim=None, threads=0, payloads=None) -> dict` | `POST /v1/build/bulk` | empty DB only; optional payloads |
| `stats() -> dict` | `GET /v1/stats` | tombstone / bloom counters |
| `health() -> bool` | `GET /health` | |
| `ready() -> bool` | `GET /ready` | DB open + responsive |

`search` returns a list of `SearchResult(id: int, distance: float)`.

## Errors

HTTP status codes map to typed exceptions (all subclass `LSMVecError`):

| Status | Exception |
|---|---|
| 400 | `InvalidArgument` |
| 401 | `Unauthorized` |
| 404 | `NotFound` |
| 413 | `PayloadTooLarge` |
| 429 | `RateLimited` |
| 5xx | `ServerError` |

```python
from lsmvec_client import NotFound

try:
    client.get(999999)
except NotFound:
    print("no such id")
```

Every exception keeps the full JSON error body on `.details` — e.g.
`inserted` / `failed_index` after a mid-batch `insert_batch` error, or
`rebuild_required` after a failed `bulk_build(payloads=...)`.

## Notes

- Vectors are stored with 8-bit scalar quantization (SQ8). `get()`
  returns the dequantized vector, which differs from the input by
  up to ~`range/255` per element. Distances and recall are computed
  on the quantized form.
- `id` is an integer in `[0, 2^63-1]`.
- On `insert` / `insert_batch`, a non-empty `metadata` replaces the
  payload, an explicit `{}` clears it, and omitting it leaves the
  existing payload unchanged.
- The client is synchronous and connection-per-request (stdlib
  `urllib`). For high-throughput ingestion, prefer `bulk_build` (empty
  index) or `insert_batch` (existing index) over a loop of `insert`.

## Testing

Against a running server:

```bash
LSMVEC_TEST_URL=http://localhost:8000 LSMVEC_TEST_DIM=8 \
    python3 sdk/python/tests/test_client.py
```
