Metadata-Version: 2.4
Name: hyperspacedb
Version: 3.1.3
Summary: Fastest Hyperbolic Vector DB Client
Author: YARlabs
Keywords: vector-database,ann,grpc,embeddings,hyperspace
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: grpcio>=1.50.0
Requires-Dist: protobuf>=4.21.0
Requires-Dist: numpy>=1.20.0
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: cohere
Requires-Dist: cohere>=4.0.0; extra == "cohere"
Provides-Extra: voyage
Requires-Dist: voyageai>=0.1.0; extra == "voyage"
Provides-Extra: google
Requires-Dist: google-generativeai>=0.3.0; extra == "google"
Provides-Extra: sentence-transformers
Requires-Dist: sentence-transformers>=2.2.0; extra == "sentence-transformers"
Provides-Extra: all
Requires-Dist: openai>=1.0.0; extra == "all"
Requires-Dist: cohere>=4.0.0; extra == "all"
Requires-Dist: voyageai>=0.1.0; extra == "all"
Requires-Dist: google-generativeai>=0.3.0; extra == "all"
Requires-Dist: sentence-transformers>=2.2.0; extra == "all"

# HyperspaceDB Python SDK

Official Python client for HyperspaceDB gRPC API v3.1.1

The SDK is designed for production services and benchmark tooling:
- collection management
- single and batch insert
- single and batch vector search
- recursive logical filters (`AND`, `OR`, `NOT`)
- bulk data management (`get_points`, `update_payload`, `scroll`, `count`)
- system health monitoring (`health_check`)
- graph traversal API methods
- optional embedder integrations
- multi-tenant metadata headers

## Requirements

- Python 3.8+
- Running HyperspaceDB server (default gRPC endpoint: `localhost:50051`)

## Installation

```bash
pip install hyperspacedb
```

Optional embedder extras:

```bash
pip install "hyperspacedb[openai]"
pip install "hyperspacedb[all]"
```

## Quick Start

```python
from hyperspace import HyperspaceClient

client = HyperspaceClient("localhost:50051", api_key="I_LOVE_HYPERSPACEDB")
collection = "docs_py"

client.delete_collection(collection)

# New Schema-driven API (Matryoshka + Multi-Vector support)
client.create_collection(
    collection,
    schema={
        "components": [
            {"name": "primary", "metric": "cosine", "full_dimension": 3, "weight": 1.0}
        ],
        "cascade_pipeline": []
    }
)

# id is now the first argument
client.insert(
    id=1,
    vector=[0.1, 0.2, 0.3],
    metadata={"source": "demo"},
    collection=collection,
)

results = client.search(
    vector=[0.1, 0.2, 0.3],
    top_k=5,
    collection=collection,
)
print(results)

client.close()
```

## Batch Search (Recommended for Throughput)

```python
queries = [
    [0.1, 0.2, 0.3],
    [0.3, 0.1, 0.4],
]

batch_results = client.search_batch(
    vectors=queries,
    top_k=10,
    collection="docs_py",
)
```

`search_batch` reduces per-request RPC overhead and should be preferred for high concurrency.

## Hybrid & Lexical Search (BM25)

HyperspaceDB supports advanced BM25 lexical ranking and hybrid fusion.

### 1. Pure Lexical Search (BM25)
Use `search_text` for full-text search. You can explicitly set BM25 scoring parameters:

```python
results = client.search_text(
    text="quantum leap",
    top_k=10,
    collection="docs",
    bm25_options={
        "method": "bm25plus",
        "k1": 1.2,
        "b": 0.75,
        "language": "english"
    }
)
```

### 2. Hybrid Search
Combine semantic vector results with lexical ranking. You can provide a pre-computed `vector` and a `hybrid_query` for lexical matching:

```python
results = client.search(
    vector=[0.1, 0.2, 0.3],
    hybrid_query="quantum computing",
    hybrid_alpha=0.7, # 70% vector weight, 30% lexical
    top_k=10,
    collection="docs"
)

results = client.search(
    query_text="quantum computing",
    hybrid_alpha=0.7,
    collection="docs"
)
```

### 3. Wave Search & Dynamic Restart Factor
Enable graph traversal-based Wave search using `use_wave` and control the return-to-seed coefficient with `restart_factor`:

```python
results = client.search(
    vector=[0.1, 0.2, 0.3],
    collection="docs",
    use_wave=True,
    restart_factor=0.6 # High factor (e.g., 0.7-0.8) keeps search close to seeds (factual QA); low factor (e.g., 0.2-0.3) allows deeper traversal (legal/citation exploration).
)
```

## Matryoshka Representation Learning (MRL) & Cascading

HyperspaceDB supports MRL through its **Cascade Pipeline**. This allows you to perform initial fast search on a truncated low-dimensional vector (e.g., 64D) and then rerank the results using the full vector (e.g., 1024D).

```python
client.create_collection(
    "mrl_collection",
    schema={
        "components": [
            {"name": "primary", "metric": "lorentz", "full_dimension": 1025, "weight": 1.0}
        ],
        "cascade_pipeline": [
            {
                "component_name": "primary",
                "cutoff_dimension": 129, # Initial search on 128D (+1)
                "store_in_ram": True,
                "rerank_top_k": 100
            }
        ]
    }
)
```

## Geometric Filters

HyperspaceDB introduces advanced spatial filters that run on the engine level:

```python
# 1. Proximity Search (Ball)
# Find vectors within radius 0.5 of the center
ball_f = client.filter_ball(center=[0.1, 0.2, 0.3], radius=0.5)

# 2. Workspace Constraints (Box)
# Find vectors within an N-dimensional bounding box
box_f = client.filter_box(min_bounds=[-1, -1, -1], max_bounds=[1, 1, 1])

# 3. Field of View / Angular Search (Cone)
# Based on ConE (Zhang & Wang, 2021)
cone_f = client.filter_cone(axes=[1.0, 0.0, 0.0], apertures=[0.5], cen=0.01)

results = client.search(
    vector=[0.1, 0.2, 0.3],
    filters=[ball_f, box_f] # Combine multiple filters
)

# 4. Recursive Logical Filters
and_f = client.filter_and([
    client.filter_match("status", "active"),
    client.filter_or([
        client.filter_range("score", gte=0.8),
        client.filter_match("priority", "high")
    ])
])
```

## Advanced Data Operations

### Bulk Retrieval (`get_points`)
```python
points = client.get_points(ids=[1, 2, 3], collection="docs")
```

### Metadata Updates (`update_payload`)
```python
client.update_payload(id=1, metadata={"status": "archived"}, collection="docs")
```

### Paginated Scanning (`scroll`)
```python
# Iteratively retrieve points with filters
for points in client.scroll(limit=100, filters=[and_f], collection="docs"):
    process(points)
```

### Point Counting (`count`)
```python
total = client.count(filters=[client.filter_match("category", "ai")], collection="docs")
```

### Health Check
```python
status = client.health_check() # Returns "ONLINE"
```

## API Summary

### Collection Operations

- `create_collection(name, schema: dict) -> bool`
- `delete_collection(name) -> bool`
- `list_collections() -> list[dict]`  # [{"name": str, "count": int, "schema": dict}]
- `get_collection_stats(name) -> dict`  # {"count": int, "indexing_queue": int, "schema": dict}


### Data Operations

- `insert(id, vector=None, document=None, metadata=None, typed_metadata=None, collection="", durability=Durability.DEFAULT) -> bool`
- `insert_text(id, text, metadata=None, collection="", durability=Durability.DEFAULT) -> bool`
- `vectorize(text, metric="l2") -> list[float]`
- `batch_insert(vectors, ids, metadatas=None, typed_metadatas=None, collection="", durability=Durability.DEFAULT) -> bool`
- `search(vector=None, query_text=None, top_k=10, filter=None, filters=None, hybrid_query=None, hybrid_alpha=None, bm25=None, collection="", options=None, use_wave=False, restart_factor=None) -> list[dict]`
- `search_text(text, top_k=10, filter=None, filters=None, hybrid_alpha=None, bm25=None, collection="") -> list[dict]`
- `search_batch(vectors, top_k=10, collection="") -> list[list[dict]]`
- `search_multi_collection(vector, collections, top_k=10) -> dict[str, list[dict]]`
- `search_multi_collection_text(text, collections, top_k=10) -> dict[str, list[dict]]`
- `delete(id, collection="") -> bool`
- `get_node(id, layer=0, collection="") -> dict`
- `get_neighbors(id, layer=0, limit=64, offset=0, collection="") -> list[dict]`
- `get_concept_parents(id, layer=0, limit=32, collection="") -> list[dict]`
- `get_subsumption_tree(root_id, max_depth=3, collection="") -> list[dict]`  # Lorentz hierarchy
- `traverse(start_id, max_depth=2, max_nodes=256, layer=0, traversal_mode=0, breadth_limit=10, filter=None, filters=None, collection="") -> list[dict]`
- `explore_graph(start_id, max_depth=2, max_nodes=256, collection="") -> dict` # Ego-Graph JSON
- `find_semantic_clusters(layer=0, min_cluster_size=3, max_clusters=32, max_nodes=10000, collection="") -> list[list[int]]`

For `filters` with `type="range"`, decimal thresholds are supported (`gte_f64/lte_f64` in gRPC payload are set automatically for non-integer values).

### Maintenance Operations

- `rebuild_index(collection, filter_query=None) -> bool`
- `trigger_vacuum() -> bool`
- `trigger_snapshot() -> bool`
- `configure(ef_search=None, ef_construction=None, collection="") -> bool`
- `trigger_reconsolidation(collection, target_vector, learning_rate) -> bool`
- `subscribe_to_events(types=None, collection=None) -> Iterator[dict]`
- `get_digest(collection="") -> dict`
- `sync_handshake(collection, client_buckets, client_logical_clock=0, client_count=0) -> dict`
- `sync_pull(collection, bucket_indices) -> Iterator[dict]`

`filter_query` example:
```python
client.rebuild_index(
    "docs_py",
    filter_query={"key": "energy", "op": "lt", "value": 0.1},
)
```

CDC subscription example:
```python
for event in client.subscribe_to_events(types=["insert", "delete"], collection="docs_py"):
    print(event)
```

### Hyperbolic Math Utilities

```python
from hyperspace.math import (
    mobius_add,
    exp_map,
    log_map,
    parallel_transport,
    riemannian_gradient,
    frechet_mean,
)
```

### Cognitive Math SDK (Spatial AI Engine)

Provides advanced tools for Agentic AI, running entirely on the client side:

```python
from hyperspace.math import (
    local_entropy,
    lyapunov_convergence,
    koopman_extrapolate,
    context_resonance,
)

# 1. Detect Hallucinations (Entropy approaches 1.0)
entropy = client.local_entropy(candidate=thought_vector, neighbors=neighbors, c=1.0)

# 2. Proof of Convergence (Negative derivative = convergence)
stability = client.get_trust_score(trajectory_ids=[1, 2, 3], collection="docs")

# 3. Extrapolate next thought (Koopman linearization)
next_thought = client.predict_momentum(trajectory_ids=[10, 11], steps=1.0)

# 4. Phase-Locked Loop for topic tracking
synced_thought = context_resonance(thought, global_context, resonance_factor=0.5, c=1.0)

# 5. Predict Semantic Relation (A + R ≈ B)
relation = client.predict_relation(id_a=1, id_b=2)
```

## Implicit Graph Engine (v3.1.1)

HyperspaceDB treats your vectors as nodes in a dynamic graph. Relationships are inferred from the geometry:
- **Lorentz / Poincare**: Hierarchy and subsumption (light cones).
- **L2 / Cosine**: Semantic similarity and adjacency.

### Subsumption Trees
Extract directed hierarchies from Lorentz-encoded data:
```python
tree = client.get_subsumption_tree(root_id=1, max_depth=5)
```

### Advanced Traversal
Navigate the graph using physical kernels:
```python
results = client.traverse(
    start_id=1,
    traversal_mode=2, # 0: GREEDY, 1: DIFFUSIVE, 2: MOMENTUM
    breadth_limit=5
)
```

## Durability Levels

Use `Durability` enum values:
- `Durability.DEFAULT`
- `Durability.ASYNC`
- `Durability.BATCH`
- `Durability.STRICT`

## Multi-Tenancy

Pass `user_id` to include `x-hyperspace-user-id` on all requests:

```python
client = HyperspaceClient(
    "localhost:50051",
    api_key="I_LOVE_HYPERSPACEDB",
    user_id="tenant_a",
)
```

## Embedding Pipeline (Optional)

HyperspaceDB supports **per-geometry embeddings** — each geometry (`l2`, `cosine`, `poincare`, `lorentz`, `hybrid`) can use its own backend independently.

### Quick Setup via Environment Variables

```bash
export HYPERSPACE_EMBED=true

# Cosine geometry → OpenAI API
export HS_EMBED_COSINE_PROVIDER=openai
export HS_EMBED_COSINE_EMBED_MODEL=text-embedding-3-small
export HS_EMBED_COSINE_API_KEY=sk-...

# Poincaré geometry → HuggingFace Hub (auto-downloads ONNX model)
export HS_EMBED_POINCARE_PROVIDER=huggingface
export HS_EMBED_POINCARE_HF_MODEL_ID=your-org/cde-spatial-poincare-128d
export HS_EMBED_POINCARE_DIM=128
export HF_TOKEN=hf_...  # Optional: for gated models

# Lorentz geometry → Local ONNX file
export HS_EMBED_LORENTZ_PROVIDER=local
export HS_EMBED_LORENTZ_MODEL_PATH=./models/lorentz_128d.onnx
export HS_EMBED_LORENTZ_TOKENIZER_PATH=./models/lorentz_128d_tokenizer.json
export HS_EMBED_LORENTZ_DIM=129
```

### Client-Side Embedder

The Python SDK also includes client-side embedders (no server config needed):

```python
from hyperspace.embedder import OpenAIEmbedder, LocalOnnxEmbedder, HuggingFaceEmbedder

# OpenAI
embedder = OpenAIEmbedder(api_key="sk-...", model="text-embedding-3-small")
vector = await embedder.encode("my text")

# Local ONNX — load from disk
embedder = LocalOnnxEmbedder(
    model_path="./models/bge-small.onnx",
    tokenizer_path="./models/bge-small-tokenizer.json",
    geometry="cosine",
)
vector = await embedder.encode("my text")

# HuggingFace Hub — auto-downloads on first use
# Cached at ~/.cache/huggingface/hub
embedder = HuggingFaceEmbedder(
    model_id="BAAI/bge-small-en-v1.5",
    geometry="cosine",
    hf_token=None,  # Set for gated/private models
)
vector = await embedder.encode("my text")
```

### Supported Geometries

| Geometry | Post-Processing | Typical Use Case |
|---|---|---|
| `cosine` | Unit normalize | Semantic similarity |
| `l2` | Unit normalize | Euclidean distance |
| `poincare` | Clamp to unit ball | Hierarchical data (ontologies) |
| `lorentz` | None (model handles it) | Mixed hierarchical + semantic |

## Zero-Knowledge Client-Side Encryption (ZK-Privacy)

HyperspaceDB v3.1.1 introduces Zero-Knowledge client-side encryption (ZK-Privacy). All private data (vectors, metadata, payloads) are encrypted/obfuscated *before* they leave the client. The database server never sees the raw vectors or plaintext data, ensuring maximum security even in public or untrusted DePIN environments.

### Key Features

1. **Vector Projection**: High-dimensional vectors are projected using a deterministic orthogonal matrix (or Lorentz boost matrix for hyperbolic spaces) generated from the collection key. This preserves distances (L2, Cosine, Lorentz) while hiding the vector coordinates.
2. **Anisotropic Noise Injection**: Injecting subtle deterministic noise into the vectors to prevent reconstruction attacks.
3. **Payload Encryption**: Sidecar payloads are encrypted client-side using AES-256-GCM before being sent to the database.
4. **Metadata Hashing**: Metadata keys and values are obfuscated using HMAC-SHA256.

### Usage Example

```python
from hyperspace import HyperspaceClient

client = HyperspaceClient("localhost:50051", api_key="I_LOVE_HYPERSPACEDB")

collection = "encrypted_docs"
secret_key = "my-super-secret-key"

# Register collection key to enable automatic client-side encryption/decryption
# noise_sigma defaults to 0.02 (2% anisotropic noise)
client.register_collection_key(collection, secret_key, metric="cosine", noise_sigma=0.02)

# 1. Insert vector (will be projected, noise injected, payload encrypted, metadata hashed)
client.insert(
    id=1,
    vector=[0.1, 0.2, 0.3],
    metadata={"category": "confidential"},
    collection=collection,
    payload=b"This is a highly secret document payload",
)

# 2. Search (search vector is projected and noise-injected; results are decrypted locally)
results = client.search(
    vector=[0.1, 0.2, 0.3],
    top_k=5,
    collection=collection,
    filter={"category": "confidential"},  # Filters are automatically hashed client-side
)

for res in results:
    print(f"ID: {res['id']}, Distance: {res['distance']}")
    if res.get("payload"):
        print(f"Decrypted Payload: {res['payload'].decode('utf-8')}")

client.close()
```

### Low-Level Crypto API

You can use the primitives directly for custom workflows (e.g., pre-encrypting payloads before batch insert):

```python
from hyperspace.crypto import derive_keys, encrypt_payload, decrypt_payload, hash_metadata_key, hash_metadata_value

# Derive deterministic AES + HMAC keys from your secret and collection name
aes_key, hmac_key = derive_keys(password="my-super-secret-key", collection_name="encrypted_docs")

# Encrypt a payload
plaintext = b"Confidential instructions for the AI agent"
ciphertext = encrypt_payload(plaintext, aes_key)

# Decrypt it back
recovered = decrypt_payload(ciphertext, aes_key)
assert recovered == plaintext

# Hash metadata for safe server-side storage
hashed_key   = hash_metadata_key("category", hmac_key)    # → "tag_3f8a9b..."
hashed_value = hash_metadata_value("confidential", hmac_key)  # → "val_7c2e1d..."
```

### ZK-Privacy for All Vector Geometries

ZK-Privacy is fully supported for **all geometry types** and **MRL (Matryoshka) vectors**:

| Geometry | Projection Method | Notes |
|---|---|---|
| `cosine` | Orthogonal matrix (Gram-Schmidt) | Unit-norm preserved |
| `l2` | Orthogonal matrix | Euclidean distance preserved |
| `poincare` | Orthogonal matrix + ball clamp | `‖x‖ < 1` boundary enforced post-projection |
| `lorentz` | Lorentz boost matrix `O(1,d)` | Time component x₀ recalculated after projection |
| `hybrid` | Per-component projection | Each sub-vector projected independently |
| `mrl` / cascade | Full vector projected, cutoffs applied after | Sub-vectors share the same projection matrix |

### ZK-Privacy API Reference

- `register_collection_key(collection_name, key, metric="l2", noise_sigma=0.02, schema=None)` — Register a secret key for a collection; all subsequent insert/search/filter calls on that collection are automatically ZK-encrypted.
- `create_encrypted_collection(name, key, schema, noise_sigma=0.02)` — Create collection and register its key in a single call.

### Security Model

```
Client machine (trusted)          Server / DePIN node (untrusted)
─────────────────────────         ──────────────────────────────────
plaintext vector                  projected + noisy vector  ← server stores this
    │                                       ▲
    ├── orthogonal projection ──────────────┘
    ├── anisotropic noise injection
    │
plaintext payload                 AES-256-GCM ciphertext    ← server stores this
    │                                       ▲
    └── AES-256-GCM encrypt ───────────────┘

plaintext metadata key/value      HMAC-SHA256 hash          ← server stores this
    │                                       ▲
    └── HMAC-SHA256 ──────────────────────-┘
```

The server **never** receives the secret key, the plaintext vectors, the plaintext payloads, or the plaintext metadata values. Decryption happens entirely on the client side.

---

## Best Practices

- Reuse one client instance per worker/process.
- Prefer `search_batch` for benchmark and high-QPS paths.
- Chunk large inserts instead of one huge request.
- Keep vector dimensionality aligned with collection configuration.
- For `lorentz` geometry, dimension = spatial_dim + 1 (the time component x₀).
- For `huggingface` provider, models are cached after first download.

## Error Handling

The SDK catches gRPC errors and returns `False` / `[]` in many methods.
For strict production observability, log return values and attach metrics around failed operations.

