Metadata-Version: 2.4
Name: feather-db
Version: 0.16.0
Summary: Embedded vector database + living context engine — multimodal pockets, context graph, adaptive decay, MCP server
Author-email: "Hawky.ai Team" <hello@hawky.ai>
License: MIT
Project-URL: Homepage, https://www.getfeather.store/
Project-URL: Documentation, https://github.com/feather-store/feather/blob/main/README.md
Project-URL: Repository, https://github.com/feather-store/feather
Project-URL: Issues, https://github.com/feather-store/feather/issues
Project-URL: Changelog, https://github.com/feather-store/feather/blob/main/CHANGELOG.md
Project-URL: Hawky.ai, https://hawky.ai
Keywords: vector,database,similarity,search,embeddings,hnsw,ann,nearest-neighbor,context-graph,knowledge-graph,living-context,multimodal,ai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: C++
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.19.0
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.9; extra == "dev"
Provides-Extra: examples
Requires-Dist: sentence-transformers>=2.0; extra == "examples"
Provides-Extra: langchain
Requires-Dist: langchain>=0.1.0; extra == "langchain"
Requires-Dist: langchain-core>=0.1.0; extra == "langchain"
Provides-Extra: llamaindex
Requires-Dist: llama-index>=0.10.0; extra == "llamaindex"
Requires-Dist: llama-index-core>=0.10.0; extra == "llamaindex"
Provides-Extra: mcp
Requires-Dist: mcp>=0.9.0; extra == "mcp"
Provides-Extra: all
Requires-Dist: langchain>=0.1.0; extra == "all"
Requires-Dist: langchain-core>=0.1.0; extra == "all"
Requires-Dist: llama-index>=0.10.0; extra == "all"
Requires-Dist: llama-index-core>=0.10.0; extra == "all"
Requires-Dist: mcp>=0.9.0; extra == "all"
Requires-Dist: google-genai>=1.0.0; extra == "all"
Requires-Dist: anthropic>=0.20.0; extra == "all"
Requires-Dist: openai>=1.0.0; extra == "all"
Dynamic: license-file
Dynamic: requires-python

# Feather DB

**Embedded vector database + self-aligned context engine**

*Part of [Hawky.ai](https://hawky.ai) — AI-Native Development Tools*

[![PyPI](https://img.shields.io/pypi/v/feather-db)](https://pypi.org/project/feather-db/)
[![Crates.io](https://img.shields.io/crates/v/feather-db-cli)](https://crates.io/crates/feather-db-cli)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![Website](https://img.shields.io/badge/website-getfeather.store-blue)](https://www.getfeather.store/)
[![HuggingFace Space](https://img.shields.io/badge/demo-HuggingFace-yellow)](https://huggingface.co/spaces/Hawky-ai/feather-db)
[![Benchmark Results](https://img.shields.io/badge/dataset-benchmarks-orange)](https://huggingface.co/datasets/Hawky-ai/feather-db-benchmarks)

[![LongMemEval_S](https://img.shields.io/badge/LongMemEval__S-0.693%20%7C%20gpt--4o-success)](docs/benchmarks/longmemeval.md)
[![LongMemEval_S Cheap](https://img.shields.io/badge/LongMemEval__S-0.657%20%7C%20gemini--flash-success)](docs/benchmarks/longmemeval.md)
[![ANN p50](https://img.shields.io/badge/SIFT1M%20p50-0.19ms-blue)](docs/benchmarks/longmemeval.md)
[![ANN Recall](https://img.shields.io/badge/recall%4010-0.972-blue)](docs/benchmarks/longmemeval.md)

Feather DB is an embedded vector database and living context engine — zero-server, file-based, with a built-in knowledge graph, adaptive memory decay, LLM agent connectors, and a self-aligned ingestion engine that organises data automatically.

---

## What's New in v0.13–v0.15 — Ingestion, Memory & Claude (Phase 8)

| Capability | Version | Notes |
|---|---|---|
| **Persisted HNSW graph** | v0.16.0 | `save()` embeds the prebuilt graph (**file format v9**) so `load()` restores it instead of rebuilding — **5–25× faster cold load** (48 ms vs 2.7s/13.4s on 40k×128 clustered) and deterministic serial-build recall (0.988). Falls back to rebuild for dirty DBs / old files; ~25% larger files. |
| **Adaptive index capacity** | v0.15.3 | HNSW indices start at 4096 elements and grow via `resizeIndex()` on demand instead of preallocating 1M — **~7.7× less RAM** for many-namespace deployments (709→92 MB across 19 namespaces), no hard cap. |
| **Parallel HNSW load** | v0.13.0 | Graph rebuilt across a thread pool on open — **~4.7× faster load** (7.6s→1.7s for 40k×128), identical recall. `FEATHER_LOAD_THREADS` to cap. Still used for old files / DBs with pending deletions. |
| **Parallel batch ingest** | v0.13.0 | `DB.add_batch(ids, vecs, metas=None)` builds the graph in parallel with the GIL released — **~3.4× faster** bulk insert. |
| **SIMD on x86** | v0.13.0 | SSE/AVX L2 kernels (runtime-dispatched) compiled on x86_64; arm64 uses `-O3` NEON. `FEATHER_SIMD=none\|sse\|avx\|avx512`. |
| **In-RAM int8 quantization** | v0.15.0 | `set_int8_ram(modality, max_abs)` stores vectors as int8 **in memory** — **~1.7× less RAM** (227→129 MB at 60k×768), recall ~0.88. **File format v8.** |
| **MCP connector for Claude** | v0.14.0 | `feather-serve` exposes Feather as a persona context engine to **Claude Desktop / Code** — local `.feather` *or* remote Cloud API (`--api-url`). |
| **Real embedders** | v0.15.1 | `feather-serve --embed-provider gemini\|openai\|voyage\|cohere\|ollama` — semantic recall over a hosted instance (Gemini `text-embedding-004` = native 768). |

```python
# Bulk-ingest a persona's history fast (parallel HNSW build)
db.add_batch(ids, vecs, metas)

# Store vectors as int8 in RAM — ~1.7x less memory (opt-in, lossy)
db.set_int8_ram("text", max_abs=1.0)
```

```bash
# Claude Code → hosted Feather as a persona context engine (real embeddings)
GOOGLE_API_KEY=… claude mcp add feather -- feather-serve \
  --api-url http://HOST:8000 --namespace persona --dim 768 --embed-provider gemini
```

---

## What's New in v0.11–v0.12 — Query Performance & Compression (Phase 7)

| Capability | Version | Notes |
|---|---|---|
| **Secondary metadata indexes** | v0.11.0 | Inverted indexes on `namespace_id` / `entity_id` / `attributes` — namespace & attribute lookups go from O(n) scans to **O(matches)**. New `DB` methods: `ids_in_namespace`, `ids_for_entity`, `ids_with_attribute`, `namespace_size`, `list_namespaces`. |
| **Pre-filtered ANN search** | v0.11.0 | `search(filter=…)` with a `namespace`/`entity`/`attribute` constraint now ranks **exactly** over the indexed candidate set, returning a **complete top-k** instead of HNSW's `ef`-bounded under-return — and is O(matches), so selective filters are *faster*. |
| **Incremental auto-compaction** | v0.11.0 | `set_auto_compact(ratio)` rebuilds a modality index once its deleted/total ratio crosses a threshold, reclaiming `forget()`/`purge()`'d vectors automatically. `compact()` also fixed to reclaim forgotten records and never resurrect purged ones. |
| **On-disk int8 quantization** | v0.12.0 | `set_quantized(modality)` persists vectors as int8 + per-vector scale (**file format v7**) — **~2.5–4× smaller `.feather` files**, dequantized to float32 on load (search unchanged). |

```python
# Pre-filtered exact search — reliably returns a full k even under a selective filter
f = FilterBuilder().namespace("acme").attribute("channel", "instagram").build()
results = db.search(query_vec, k=10, filter=f)        # complete top-10, exact ranking

# Auto-compaction — reclaim deleted vectors past 20% dead
db.set_auto_compact(0.2)

# int8 on-disk compression — ~3x smaller files, opt-in per modality
db.set_quantized("text", True)
db.save()                                             # persisted as int8 (format v7)
```

> **Scope note:** int8 quantization reduces **disk footprint and load I/O**; the
> in-memory HNSW index remains float32. In-RAM int8 indexing is a future step.

---

## What's New in v0.10 — Feather DB Cloud Edition

The `feather-api/` package now ships a **production-ready admin SPA** + a
**pluggable embedding service**, so you can run Feather as a managed context
engine for downstream consumers (e.g. brand teams, agents, internal tools).

| Capability | Where | Notes |
|---|---|---|
| **Atlas-style admin SPA** | `/admin/` route on the FastAPI server | Custom HTML + Tailwind + Alpine.js, brand-aligned, zero build step |
| **Pluggable embeddings** | Settings → Embedding service | OpenAI · Azure OpenAI · Gemini · Voyage · Cohere · Ollama with curated model dropdowns |
| **Ingest text** | `POST /v1/{ns}/ingest_text` | Server embeds via the configured provider, then stores — single call |
| **Bulk import** | `POST /v1/{ns}/import` | Paste a JSON array of `{id, vector, metadata}` |
| **Hierarchy navigator** | Namespace detail → Hierarchy tab | Brand → Channel → Campaign → AdSet → Ad → Creative tree from `metadata.attributes` |
| **Marketing profile card** | Record drawer | Auto-renders KPIs (CTR, ROAS, channel) when present |
| **Cmd-K palette** | Press ⌘K / Ctrl+K | Fuzzy search namespaces · `id:123` to open a record · `/seed`, `/import` actions |
| **Live observability** | Overview screen | p50 / p95 / p99 latency, ops-per-minute sparkline, recent activity feed |
| **Delete + purge + compact** | Namespace header buttons | Per-record DELETE, bulk PURGE by `namespace_id`, COMPACT to reclaim |
| **Schema discovery** | Namespace detail → Schema tab | Distinct attribute keys + type inference + sample values |
| **Connection panel** | Settings → Connection | Copy-paste cURL / Python / JS snippets pre-filled with your URL |

See [docs/quickstart.md](docs/quickstart.md) for a self-hosted setup walkthrough.

> **Deployment note**: `feather-api/` runs single-tenant with one shared
> `FEATHER_API_KEY`. Multi-tenant key isolation + HTTPS are on the roadmap.

---

## What's Inside

| Capability | Description |
|------------|-------------|
| **ANN Search** | Sub-millisecond approximate nearest-neighbor search via HNSW |
| **Multimodal Pockets** | Text, image, audio vectors per entity under a single ID |
| **Context Graph** | Typed + weighted edges, reverse index, auto-link by similarity |
| **Context Chain** | One call: vector search + n-hop BFS graph expansion |
| **Living Context** | Recall-count stickiness — frequently accessed items resist temporal decay |
| **Namespace / Entity / Attributes** | Generic partition + subject + KV metadata for any domain |
| **Graph Visualizer** | Self-contained D3 force-graph HTML — fully offline, no CDN |
| **LLM Agent Connectors** | Claude, OpenAI, Gemini tool-use/function-calling with 14 Feather tools |
| **MCP Server** | `feather-serve` — connects Feather to Claude Desktop, Cursor, and any MCP client |
| **LangChain / LlamaIndex** | Drop-in `FeatherVectorStore`, `FeatherMemory`, `FeatherRetriever` adapters |
| **Self-Aligned Context Engine** | LLM-powered ingestion: auto-classifies, scores, links, and namespaces every record |
| **Single-file persistence** | `.feather` binary format (v9, persisted HNSW graph for fast cold load + optional int8 compression on-disk and in-RAM); v3–v8 files load transparently |

---

## Installation

```bash
pip install feather-db            # core
pip install "feather-db[all]"     # + langchain, llamaindex, mcp extras
```

CLI (Rust):

```bash
cargo install feather-db-cli
```

Build from source:

```bash
git clone https://github.com/feather-store/feather
cd feather
python setup.py build_ext --inplace
```

---

## Quick Start

```python
import feather_db
import numpy as np

# Open or create a database
db = feather_db.DB.open("context.feather", dim=768)

# Add a vector with metadata
meta = feather_db.Metadata()
meta.content = "User prefers dark mode"
meta.importance = 0.9
db.add(id=1, vec=np.random.rand(768).astype(np.float32), meta=meta)

# Semantic search
results = db.search(np.random.rand(768).astype(np.float32), k=5)
for r in results:
    print(r.id, r.score, r.metadata.content)

db.save()
```

---

## Self-Aligned Context Engine (v0.7.0)

The `ContextEngine` wraps `DB` with an LLM-powered ingestion pipeline. Drop in any text — the engine classifies it, scores it, links it to related records, and stores it in the right namespace. No schema to define upfront.

```python
from feather_db import ContextEngine, ClaudeProvider
import numpy as np, hashlib

def embed(text: str) -> np.ndarray:
    # replace with your real embedder
    vec = np.zeros(768, dtype=np.float32)
    for i, tok in enumerate(text.split()[:768]):
        vec[i % 768] += 1.0
    n = np.linalg.norm(vec)
    return vec / n if n > 0 else vec

engine = ContextEngine(
    db_path  = "knowledge.feather",
    dim      = 768,
    provider = ClaudeProvider(),   # or OpenAIProvider, GeminiProvider, OllamaProvider, None
    embedder = embed,
    namespace = "myapp",
)

nid = engine.ingest(
    "Competitor X launched a developer SDK with MIT license — 10k GitHub stars in 24 hours."
)
```

**The engine automatically:**
- Classifies entity type (`competitor_intel`, `user_feedback`, `strategy_brief`, …)
- Scores importance (0–1) and confidence (0–1)
- Assigns TTL and namespace
- Suggests and creates graph edges to related records

**Works offline too** — pass `provider=None` for a built-in heuristic classifier (no API key needed):

```python
engine = ContextEngine(db_path="k.feather", dim=768, provider=None, embedder=embed)
```

### Supported Providers

```python
from feather_db import ClaudeProvider, OpenAIProvider, OllamaProvider, GeminiProvider

ClaudeProvider(model="claude-haiku-4-5-20251001")         # Anthropic
OpenAIProvider(model="gpt-4o-mini")                        # OpenAI
OpenAIProvider(model="llama-3.3-70b-versatile",            # Groq
               base_url="https://api.groq.com/openai/v1",
               api_key=GROQ_KEY)
OllamaProvider(model="llama3.1:8b")                        # Ollama (local, no key)
GeminiProvider(model="gemini-2.0-flash")                   # Google Gemini
```

All providers share the same `LLMProvider` interface — swap at any time without changing the rest of your code.

---

## LLM Agent Connectors (v0.6.0)

Give any LLM agent native access to Feather DB via **14 built-in tools**.

### Claude (tool_use)

```python
import anthropic
from feather_db import ClaudeConnector

connector = ClaudeConnector(db=db, embedder=embed)
client    = anthropic.Anthropic()

messages = [{"role": "user", "content": "What competitor moves should I watch?"}]
reply    = connector.run_loop(client, messages, model="claude-opus-4-6")
print(reply)
```

### OpenAI / Groq / vLLM (function_calling)

```python
from openai import OpenAI
from feather_db import OpenAIConnector

connector = OpenAIConnector(db=db, embedder=embed)
client    = OpenAI()

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Find records about onboarding friction"}],
    tools=connector.tools(),
)
result = connector.handle(resp.choices[0].message.tool_calls[0].function.name,
                          resp.choices[0].message.tool_calls[0].function.arguments)
```

### Available Tools (14)

| Tool | Description |
|------|-------------|
| `feather_search` | Semantic vector search |
| `feather_context_chain` | Vector search + graph BFS expansion |
| `feather_get_node` | Retrieve a single record by ID |
| `feather_get_related` | Get all graph-linked records |
| `feather_add_intel` | Store a new record with metadata |
| `feather_link_nodes` | Create a typed weighted edge |
| `feather_timeline` | Time-ordered records in a range |
| `feather_forget` | Drop a record by ID |
| `feather_health` | Database health report |
| `feather_why` | Explain why a record was retrieved |
| `feather_mmr_search` | Maximal marginal relevance search |
| `feather_consolidate` | Merge near-duplicate records |
| `feather_episode_get` | Retrieve an episode by ID |
| `feather_expire` | Purge records past their TTL |

---

## MCP Server (v0.6.0)

Connect Feather DB to **Claude Desktop**, **Cursor**, or any MCP-compatible client:

```bash
pip install "feather-db[mcp]"
feather-serve --db knowledge.feather --dim 768
```

Add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "feather": {
      "command": "feather-serve",
      "args": ["--db", "/path/to/knowledge.feather", "--dim", "768"]
    }
  }
}
```

All 14 tools become available to Claude Desktop immediately — no code required.

---

## LangChain Integration (v0.6.0)

```python
from feather_db.integrations import FeatherVectorStore, FeatherMemory, FeatherRetriever

# Drop-in VectorStore
store     = FeatherVectorStore(db=db, embedder=embed)
retriever = store.as_retriever(search_kwargs={"k": 5})

# Semantic conversation memory with adaptive decay
memory = FeatherMemory(db=db, embedder=embed, k=5)

# context_chain retriever
retriever = FeatherRetriever(db=db, embedder=embed, k=5, hops=2)
```

---

## LlamaIndex Integration (v0.6.0)

```python
from feather_db.integrations import FeatherVectorStoreIndex, FeatherReader

# Index documents
index = FeatherVectorStoreIndex.from_documents(documents, db=db, embed_model=embed_model)
query_engine = index.as_query_engine()
response = query_engine.query("What is our retention strategy?")

# Load existing Feather DB as LlamaIndex Documents
reader = FeatherReader(db=db)
docs   = reader.load_data()
```

---

## Core Features

### Multimodal Pockets

Each named modality gets its own independent HNSW index and dimensionality. A single entity ID can hold text, visual, and audio vectors simultaneously.

```python
db.add(id=42, vec=text_vec,   modality="text")    # 768-dim
db.add(id=42, vec=image_vec,  modality="visual")  # 512-dim
db.add(id=42, vec=audio_vec,  modality="audio")   # 256-dim

results = db.search(query_vec, k=10, modality="visual")
```

### Context Graph

Typed, weighted edges between records. Nine built-in relationship types plus free-form strings.

```python
from feather_db import RelType

db.link(from_id=1, to_id=2, rel_type=RelType.CAUSED_BY, weight=0.9)
db.link(from_id=1, to_id=3, rel_type=RelType.SUPPORTS,  weight=0.7)

edges    = db.get_edges(1)      # outgoing edges
incoming = db.get_incoming(2)   # reverse index

db.auto_link(modality="text", threshold=0.85, rel_type=RelType.RELATED_TO)
```

Built-in types: `related_to`, `derived_from`, `caused_by`, `contradicts`, `supports`, `precedes`, `part_of`, `references`, `multimodal_of`.

### Context Chain

One call that combines semantic search with n-hop BFS graph traversal:

```python
result = db.context_chain(query=query_vec, k=5, hops=2, modality="text")

for node in result.nodes:
    print(node.id, node.score, node.hop_distance)
for edge in result.edges:
    print(edge.source_id, "->", edge.target_id, edge.rel_type)
```

### Filtered Search

```python
from feather_db import FilterBuilder

results = db.search(
    query_vec, k=10,
    filter=FilterBuilder()
        .namespace("acme")
        .entity("user_123")
        .attribute("channel", "instagram")
        .importance_gte(0.5)
        .build()
)
```

When the filter constrains a `namespace`, `entity`, or `attribute`, Feather
resolves the candidate set from its secondary indexes and ranks **exactly** over
just those vectors — so a selective filter returns a **complete top-k** (no
`ef`-bounded under-return) and runs in O(matches).

### Living Context / Adaptive Decay

```python
from feather_db import ScoringConfig

cfg     = ScoringConfig(half_life=30.0, weight=0.3, min=0.0)
results = db.search(query_vec, k=10, scoring=cfg)
```

Formula:
```
stickiness    = 1 + log(1 + recall_count)
effective_age = age_in_days / stickiness
recency       = 0.5 ^ (effective_age / half_life_days)
final_score   = ((1 - time_weight) * similarity + time_weight * recency) * importance
```

`touch()` is called automatically on every search hit.

### Memory Layer (v0.6.0)

```python
from feather_db import MemoryManager

mm = MemoryManager(db)
print(mm.health_report())          # cluster stats + stale records
diverse = mm.search_mmr(vec, k=10) # maximal marginal relevance
mm.consolidate(threshold=0.95)     # merge near-duplicate records
mm.assign_tiers()                  # hot / warm / cold tiering by recall_count
```

### Episodes (v0.6.0)

Group related records into named episodes:

```python
from feather_db import EpisodeManager

em  = EpisodeManager(db)
eid = em.begin_episode("onboarding_analysis")
em.add_to_episode(eid, node_id)
ep  = em.get_episode(eid)
em.close_episode(eid)
```

### Triggers & Contradiction Detection (v0.6.0)

```python
from feather_db import WatchManager, ContradictionDetector

wm = WatchManager(db)
wm.watch(namespace="acme", callback=lambda record: print("New:", record.content))

cd = ContradictionDetector(db)
conflicts = cd.check(new_meta)     # returns list of conflicting record IDs
```

### Namespace / Entity / Attributes

```python
meta = feather_db.Metadata()
meta.namespace_id = "acme"
meta.entity_id    = "user_123"
meta.set_attribute("channel", "instagram")   # use this, NOT meta.attributes['k'] = v
val = meta.get_attribute("channel")
```

Domain profiles for typed helpers:

```python
from feather_db import MarketingProfile

p = MarketingProfile()
p.set_brand("nike")
p.set_user("user_8821")
p.set_channel("instagram")
p.set_ctr(0.045)
meta = p.to_metadata()
```

### Graph Visualization

```python
from feather_db.graph import visualize, export_graph

visualize(db, output_path="/tmp/graph.html")          # self-contained D3 HTML
data = export_graph(db, namespace_filter="nike")      # Python dict for D3/Cytoscape
```

---

## Rust CLI

```bash
feather add    --db my.feather --id 1 --vec "0.1,0.2,0.3" --modality text
feather search --db my.feather --vec "0.1,0.2,0.3" --k 5
feather link   --db my.feather --from 1 --to 2
feather save   --db my.feather
```

---

## Performance

| Metric | Value |
|--------|-------|
| Add rate | 2,000–5,000 vectors/sec |
| Search latency p50 (k=10, 500K × 128-dim, real SIFT data) | **0.19 ms** |
| Search latency p99 (k=10, 500K × 128-dim, real SIFT data) | **0.13 ms @ ef=10**, **1.03 ms @ ef=200** |
| Recall@10 (500K × 128-dim, ef=50, real SIFT) | **0.972** |
| Max vectors per modality | unbounded — capacity grows adaptively (starts 4096) |
| HNSW params | M=16, ef_construction=200, ef=50 (default in v0.8.0) |
| File format | Binary `.feather` v9 (persisted HNSW graph: 5–25× faster cold load; optional int8: ~3× smaller on disk, ~1.7× less RAM) |

SIMD (AVX2/AVX512) optimizations are available in `space_l2.h`. Enable with `-DUSE_AVX -march=native` in `setup.py`.

Reproducible benchmark harness lives in [`bench/`](./bench/). Run any benchmark with `python -m bench run <scenario>`.

---

## Benchmarks

### Memory benchmark — LongMemEval (Xu et al., 2024)

500-question end-to-end memory QA benchmark, the standard for long-term memory in chat assistants. Full report: [`docs/benchmarks/longmemeval.md`](./docs/benchmarks/longmemeval.md).

| Run | Variant | Answerer | Overall | Notes |
|---|---|---|---|---|
| Feather DB v0.8.0 + decay | **S** | gpt-4o | **0.693** | best run; same model as Supermemory |
| Feather DB v0.8.0 + decay | **S** | gemini-2.5-flash | **0.657** | cheap-tier; ~$2.40 per full run |
| Feather DB v0.8.0 + decay | oracle | gemini-2.5-flash | 0.670 | retrieval-easy ceiling |

| System | Variant | Answerer | Overall |
|---|---|---|---|
| **Feather DB v0.8.0 + decay** | **S** | **gemini-2.5-flash** | **0.657** |
| Zep (graphiti) | S | gpt-4o-mini | 0.638 |
| Full-context GPT-4o (paper "ceiling") | S | gpt-4o + CoN | 0.640 |
| Full-context GPT-4o-mini | S | gpt-4o-mini | 0.554 |
| Mem0 (prior algo) | S | gpt-4o-mini | 0.678 |
| Supermemory | S | gpt-4o | 0.816 |

Cost for the full Feather S run: **~$2.40** (Azure embeddings + Gemini answer + judge). Wall time 4.5 hours. 5 failures / 500 questions.

Reproduce:
```bash
python -m bench run longmemeval --dataset s --limit 0 \
  --embedder openai --judge llm \
  --judge-provider gemini --judge-model gemini-2.0-flash \
  --answerer-provider gemini --answerer-model gemini-2.5-flash \
  --decay-half-life 14 --decay-time-weight 0.4 --k 10
```

### ANN benchmark — SIFT1M

Standard ANN benchmark. Full sweep results in [`bench/results/`](./bench/results/).

| Config | p50 | p99 | Recall@10 |
|---|---|---|---|
| 500K × 128, ef=10 | 0.07 ms | 0.13 ms | 0.774 |
| 500K × 128, ef=50 (default) | 0.19 ms | 0.23 ms | **0.972** |
| 500K × 128, ef=200 | 0.56 ms | 0.69 ms | 0.998 |

---

## Cloud Deployment (Azure / Docker)

Feather DB ships with a production-ready FastAPI wrapper and the **Atlas-style
admin SPA** (custom HTML + Tailwind + Alpine.js — no build step) you can deploy
on any Linux VM.

```bash
git clone https://github.com/feather-store/feather.git
cd feather
FEATHER_API_KEY="feather-$(openssl rand -hex 16)" \
  docker compose -f feather-api/docker-compose.yml up -d --build
```

| URL | Description |
|-----|-------------|
| `http://<VM_IP>:8000/health` | Health check |
| `http://<VM_IP>:8000/docs` | Swagger / OpenAPI |
| `http://<VM_IP>:8000/admin/` | Admin SPA — namespaces, search, graph, embedding settings |

**Self-hosted walkthrough → [`docs/quickstart.md`](docs/quickstart.md)** ·
**Azure guide → [`docs/deploy-azure.md`](docs/deploy-azure.md)**

> Data is stored in a Docker named volume (`feather-data → /data`) and persists
> across restarts and rebuilds.

---

## Examples

| File | Description |
|------|-------------|
| `examples/context_engine_demo.py` | Self-Aligned Context Engine — all four providers + heuristic fallback |
| `examples/context_graph_demo.py` | Context graph — auto-link, context_chain, D3 HTML export |
| `examples/marketing_living_context.py` | Multi-brand namespace/entity/attribute filtering |
| `examples/feather_inspector.py` | Local HTTP inspector — force graph, PCA scatter, edit, delete |

Run:

```bash
python setup.py build_ext --inplace
python3 examples/context_engine_demo.py

# With a provider:
ANTHROPIC_API_KEY=sk-ant-... python3 examples/context_engine_demo.py
OLLAMA_MODEL=mistral:7b       python3 examples/context_engine_demo.py
```

---

## Architecture

```
[Generic Core — C++17]
feather::DB
  ├── modality_indices_  (unordered_map<string, ModalityIndex>)  — one HNSW per modality
  ├── metadata_store_    (unordered_map<uint64_t, Metadata>)     — shared metadata by ID
  └── Methods: add, search, link, context_chain, auto_link, export_graph_json, …

[Python Layer — feather_db]
  ├── DB, Metadata, ContextType, ScoringConfig
  ├── Edge, IncomingEdge, ContextNode, ContextEdge, ContextChainResult
  ├── FilterBuilder         — fluent search filter helper
  ├── DomainProfile         — generic namespace/entity/attributes base class
  ├── MarketingProfile      — digital marketing typed adapter
  ├── RelType               — standard relationship type constants
  ├── graph.visualize()     — D3 force-graph HTML exporter
  ├── MemoryManager         — health reports, MMR, consolidate, tiering
  ├── WatchManager          — namespace/entity watch callbacks
  ├── ContradictionDetector — conflict detection on ingest
  ├── EpisodeManager        — grouped episode records
  ├── merge()               — merge two .feather files
  ├── LLMProvider / ClaudeProvider / OpenAIProvider / OllamaProvider / GeminiProvider
  ├── ContextEngine         — self-aligned LLM-powered ingestion pipeline
  └── integrations/
      ├── ClaudeConnector   — Claude tool_use with 14 Feather tools
      ├── OpenAIConnector   — OpenAI/Groq/Mistral function_calling
      ├── GeminiConnector   — Gemini function_calling + GeminiEmbedder
      ├── FeatherVectorStore / FeatherMemory / FeatherRetriever  (LangChain)
      ├── FeatherVectorStoreIndex / FeatherReader                 (LlamaIndex)
      └── mcp_server        — feather-serve MCP endpoint

[Rust CLI]
feather-db-cli (FFI via extern "C" from src/feather_core.cpp)
```

---

## File Format

```
[magic: 4B = "FEAT"] [version: 4B = 8]
--- Metadata Section ---
[meta_count: 4B]
  for each record:
    [id: 8B] [serialized Metadata including namespace/entity/attributes/edges]
--- Modality Indices Section ---
[modal_count: 4B]
  for each modality:
    [name_len: 2B] [name: N bytes]
    [dim: 4B] [quantized: 1B]            # on-disk int8 flag (v7+)
    [int8_ram: 1B] [scale: 4B if int8_ram]   # in-RAM int8 flag + global scale (v8+)
    [persist_graph: 1B]                  # v9: 1 → prebuilt HNSW graph blob follows
    if persist_graph:                    # fast cold load — no rebuild
      [HNSW graph: header + base layer (vectors) + per-element link lists]
    else:
      [element_count: 4B]
      for each element:
        [id: 8B] then, per `quantized`:
          0 → [float32 vector: dim * 4 bytes]
          1 → [scale: 4B float] [int8 vector: dim bytes]
```

v3–v8 files load transparently (the `quantized` flag is read for v7+, the
`int8_ram` flag for v8+, the `persist_graph` flag for v9+); missing fields
default to empty. The graph is persisted only for a clean, non-on-disk-quantized
modality; otherwise load rebuilds it (parallel).

---

## Known Limitations

| Issue | Detail |
|-------|--------|
| No concurrent writes | HNSW is not thread-safe for simultaneous adds |
| Soft deletes reclaimed on compaction | `forget()`/`purge()` mark vectors deleted; space is reclaimed by `compact()` or `set_auto_compact(ratio)` |
| int8 quantization (two kinds) | `set_quantized()` = smaller files; `set_int8_ram()` = ~1.7× less RAM (opt-in, lossy) |
| Index capacity | Adaptive (v0.15.3): starts at 4096, grows via `resizeIndex` on demand — no hard cap, RAM tracks the working set |
| `meta.attributes['k'] = v` silent no-op | pybind11 map copy; use `meta.set_attribute(k, v)` |
| Rust CLI missing v0.6.0+ features | namespace/entity/context_chain/integrations are Python-only |

---

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes with tests
4. Submit a pull request

---

## License

MIT — see [LICENSE](LICENSE)

---

## Acknowledgments

- HNSW algorithm: [hnswlib](https://github.com/nmslib/hnswlib)
- Python bindings: [pybind11](https://github.com/pybind/pybind11)
- Rust CLI: [clap](https://github.com/clap-rs/clap)
- Graph visualization: [D3.js](https://d3js.org/)
