Metadata-Version: 2.4
Name: garuddb
Version: 0.1.0
Summary: Hyperbolic Vector Database for Hierarchical Data — Poincaré geometry meets HNSW
Home-page: https://github.com/AryanRajendraDalvi/GarudDB
Author: GarudDB Contributors
License: Proprietary
Project-URL: Homepage, https://github.com/AryanRajendraDalvi/GarudDB
Keywords: vector-database,hyperbolic,poincare,hnsw,knowledge-graph
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: C++
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: huggingface-hub>=0.20.0
Provides-Extra: embed
Requires-Dist: torch>=2.0; extra == "embed"
Requires-Dist: geoopt>=0.5; extra == "embed"
Requires-Dist: sentence-transformers>=2.0; extra == "embed"
Provides-Extra: adapter
Requires-Dist: torch>=2.0; extra == "adapter"
Requires-Dist: transformers>=4.30; extra == "adapter"
Requires-Dist: geoopt>=0.5; extra == "adapter"
Requires-Dist: sentence-transformers>=2.0; extra == "adapter"
Provides-Extra: server
Requires-Dist: fastapi>=0.100; extra == "server"
Requires-Dist: uvicorn[standard]>=0.20; extra == "server"
Requires-Dist: boto3>=1.28; extra == "server"
Requires-Dist: python-multipart>=0.0.9; extra == "server"
Provides-Extra: all
Requires-Dist: garuddb[embed]; extra == "all"
Requires-Dist: garuddb[adapter]; extra == "all"
Requires-Dist: garuddb[server]; extra == "all"
Requires-Dist: onnxruntime>=1.15; extra == "all"
Requires-Dist: requests>=2.28; extra == "all"
Requires-Dist: tqdm>=4.60; extra == "all"
Dynamic: home-page
Dynamic: requires-python

# GarudDB: Hyperbolic Vector Database 🦅

**A topology-aware vector database built on Poincaré geometry.** Free and open-source, like ChromaDB — but for hierarchical data.

GarudDB natively encodes parent-child relationships, corporate ownership trees, ontologies, and knowledge graphs using **Riemannian Hyperbolic Geometry (Poincaré Ball Model)**. It answers complex tree queries that flat Euclidean databases like ChromaDB, Pinecone, and FAISS fundamentally cannot.

```bash
pip install garuddb
```

---

## ⚡ Quick Start

```python
import garuddb

# Create a persistent database
client = garuddb.PersistentClient(path="./my_database")

# Create a collection with your choice of embedding model
collection = client.create_collection(
    "companies",
    embedding_function=garuddb.DefaultEmbeddingFunction()  # or GarudEmbedFunction()
)

# Add documents — auto-embedded
collection.add(
    documents=["Apple Inc.", "Braeburn Capital Inc.", "Microsoft Corp."],
    metadatas=[{"country": "US"}, {"country": "US"}, {"country": "US"}],
    ids=["apple", "braeburn", "microsoft"]
)

# Standard query (ChromaDB-compatible)
results = collection.query(query_texts=["Apple subsidiary"], n_results=5)

# GarudDB-exclusive: Topology-aware search
subtree = collection.find_subtree("Apple Inc.", n_results=10)
ancestor = collection.find_common_ancestor("Apple Inc.", "Braeburn Capital Inc.")
neighbors = collection.search_neighbors("Apple Inc.", translation_mode="inward")
```

---

## 🏗️ What Makes GarudDB Different

### The Problem with Flat Vector Databases
ChromaDB, Pinecone, FAISS, and Weaviate all store vectors in **Euclidean or cosine space**. This works fine for semantic similarity, but completely breaks down for **hierarchical data**:

- "Find all subsidiaries of Apple" → **0% recall** in ChromaDB
- "What is the common parent company of these two entities?" → **Impossible** in flat databases
- "Traverse the ownership tree from leaf to root" → **Not a concept** that exists

### GarudDB's Solution: Poincaré Geometry
The Poincaré ball model has a crucial mathematical property: **exponentially more space near the boundary**. This perfectly mirrors how hierarchies work — one root, exponentially more leaves.

In GarudDB:
- **Root nodes** (parent companies, chapter headers) → positioned near the **origin** of the disk
- **Leaf nodes** (subsidiaries, paragraph text) → positioned near the **boundary** of the disk
- **Tree distance** = **geodesic distance** along the curved Poincaré manifold

This means tree traversal is just geometry. No graph database needed.

---

## 📊 Benchmark Summary

Built on the global GLEIF dataset (3.3M+ corporate entities):

| Metric | ChromaDB | GarudDB | Advantage |
|:---|:---|:---|:---|
| RAG Search Latency | 2.49 ms | **0.06 ms** | **~40x faster** |
| Max Scale (8GB RAM) | ~414k vectors | **2M+ vectors** | **5x more** |
| Recall@10 (Flat Text) | 10.2% | 11.2% | +1pp |
| Ownership Discovery R@10 | 0.0% | **26.1%** | **∞** |
| Deep Traversal R@10 | 0.0% | **14.0%** | **∞** |
| Metadata Filter Latency | 35–80 ms | **6–15 ms** | **~4x faster** |

---

## 🔌 Pluggable Embedding Functions

GarudDB is embedding-model agnostic. Use whatever model you want:

```python
# Option 1: Default — SentenceTransformer (384D, Euclidean, cosine metric)
collection = client.create_collection(
    "docs",
    embedding_function=garuddb.DefaultEmbeddingFunction()
)

# Option 2: Native — GarudEmbed V3 (64D, real Poincaré vectors)
# Trained for 60-75 epochs on hierarchical data — the gold standard
collection = client.create_collection(
    "companies",
    embedding_function=garuddb.GarudEmbedFunction()
)

# Option 3: Custom — Any model that returns vectors
class MyEmbedder:
    dimension = 768
    is_hyperbolic = False
    def __call__(self, texts):
        return my_model.encode(texts)  # Your model here

collection = client.create_collection("custom", embedding_function=MyEmbedder())

# Option 4: Pre-computed vectors — skip embedding entirely
collection.add(embeddings=[[0.1, 0.2, ...]], ids=["1"])
```

---

## 🧭 Topology Search API (The Moat)

These methods only exist in GarudDB. No other vector database can do this:

### `search_neighbors()` — Geodesic Translation
Navigate the Poincaré disk directionally:
```python
# Find parents/grandparents (move INWARD toward disk origin)
parents = collection.search_neighbors("Braeburn Capital", translation_mode="inward")

# Find children/subsidiaries (move OUTWARD toward disk boundary)
children = collection.search_neighbors("Apple Inc.", translation_mode="outward")
```

### `find_subtree()` — Deep Hierarchy Traversal
Find all entities within a topological branch:
```python
subtree = collection.find_subtree("Apple Inc.", n_results=20)
```

### `find_common_ancestor()` — Möbius Midpoint
Find the nearest common parent of two entities:
```python
ancestor = collection.find_common_ancestor("Entity A", "Entity B")
```

### `query_filtered()` — Native C++ Bitmask Filtering
Hardware-accelerated metadata filtering inside the HNSW graph:
```python
results = collection.query_filtered(
    "Apple Inc.",
    country="US",
    status="ACTIVE",
    n_results=10
)
```

---

## 🛠️ Installation & Setup

### 1. Install dependencies
```bash
pip install -r requirements.txt
```

### 2. Build the C++ engine
```bash
pip install -e .
```
This compiles `hyperbolic_index.cpp` into the `adk_core` Python extension with AVX2 SIMD acceleration.

### 3. Optional extras
```bash
pip install garuddb[embed]    # GarudEmbed V3 (PyTorch + Geoopt)
pip install garuddb[adapter]  # GarudAdapter migration (SentenceTransformers)
pip install garuddb[server]   # FastAPI server
pip install garuddb[all]      # Everything
```

---

## 🏛️ Architecture

### C++ Engine (`adk_core`)
The heart of GarudDB — a custom HNSW implementation written in C++ with:
- **Three distance kernels:** Poincaré geodesic, cosine, Euclidean — all SIMD-accelerated (AVX2)
- **Multi-layer HNSW:** Proper O(log N) ingest and search with geometric random level assignment
- **64-bit in-graph bitmask filtering:** Metadata filters evaluated at each HNSW node during traversal
- **Dual Index Architecture:** Dense HNSW (norm < 0.995) + flat scan for boundary singletons (norm ≥ 0.995)
- **GIL release:** Python threads run concurrently during C++ execution

### Python SDK (`garuddb/`)
```
garuddb/
├── __init__.py          # Public API exports
├── client.py            # EphemeralClient, PersistentClient
├── collection.py        # Collection API (add, query, search_neighbors, etc.)
├── embeddings.py        # Pluggable embedding functions
├── cli.py               # Command-line interface
├── engine/
│   ├── index.py         # TriIndex wrapper (dense + singleton + semantic)
│   ├── concurrency.py   # Read-Write lock for thread safety
│   └── weights.py       # GarudEmbed checkpoint downloader
└── server/              # FastAPI server (optional)
```

### Tri-Index Architecture
Every collection maintains three C++ indices:

| Index | Metric | Purpose |
|:---|:---|:---|
| `dense_index` | Poincaré | HNSW graph for multi-hop topology traversal |
| `singleton_index` | Poincaré | Flat scan for boundary nodes (no HNSW edges) |
| `semantic_index` | Cosine | Text-to-entity anchor resolution |

---

## 🧠 Mathematical Features

### 1. Geodesic Translation
Before searching, the query vector is shifted along the Poincaré geodesic:
- **Inward (× 0.92):** Traverse toward parent nodes near the disk origin
- **Outward (→ 0.995):** Traverse toward leaf subsidiaries near the disk boundary

### 2. Geometric Density Filter
Ported from the battle-tested `garud_search_engine.py`. Prevents results from bleeding across tree branches:
- **Gap threshold (0.6):** If consecutive results have a distance jump > 0.6, we've crossed a branch boundary — stop
- **Absolute cutoff (2.0):** Never return results beyond structural distance 2.0

### 3. 64-Bit In-Graph Bitmask Filtering
Metadata is encoded as `uint64_t` bitmasks at ingest time. Filters are evaluated with hardware `AND` inside the HNSW traversal — no post-filtering. Filter selectivity has zero impact on latency.

### 4. Multi-Hop Gradient Traversal
After initial HNSW search, the engine re-searches from the nearest non-trivial neighbor to expand the retrieval neighborhood. This walks along HNSW graph edges, which in Poincaré space correspond to actual tree topology links.

---

## 📁 Project Structure

| Directory | Purpose |
|:---|:---|
| `garuddb/` | **Core SDK** — the pip-installable package |
| `hyperbolic_index.cpp` | C++ HNSW engine source |
| `garud_embed_model.py` | GarudEmbed V3 model architecture |
| `train_garud_embed.py` | Training script for GarudEmbed |
| `garud_adapter.py` | Euclidean → GarudDB migration tool |
| `garud_search_engine.py` | Reference implementation (corporate graph) |
| `examples/` | Example applications |
| `garud_ingest/` | 6-layer ingestion pipeline |

---

## 🚀 Roadmap

- **v0.1.0:** Core SDK release (`pip install garuddb`) with pluggable embeddings
- **v0.2.0:** Built-in data connectors (CSV, JSON, PDF parsing into hierarchy)  
- **v0.3.0:** LLM Intent Routing for natural language queries
- **v1.0.0:** GPU-accelerated CUDA kernel for billion-scale traversal

---

## 📜 License

Proprietary — see LICENSE file for details.
