Metadata-Version: 2.4
Name: mcp-fts5-starter-gemini
Version: 0.1.0
Summary: Gemini embedder for mcp-fts5-starter — drop-in hybrid BM25 + dense retrieval via Google's Gemini Embeddings API.
Project-URL: Homepage, https://github.com/zx22413/mcp-fts5-starter-gemini
Project-URL: Repository, https://github.com/zx22413/mcp-fts5-starter-gemini
Project-URL: Documentation, https://github.com/zx22413/mcp-fts5-starter-gemini/blob/main/README.md
Project-URL: Changelog, https://github.com/zx22413/mcp-fts5-starter-gemini/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/zx22413/mcp-fts5-starter-gemini/issues
Project-URL: Sibling: mcp-fts5-starter, https://github.com/zx22413/mcp-fts5-starter
Author: LBDog
License: MIT
License-File: LICENSE
Keywords: embedding,fts5,gemini,hybrid-search,mcp,model-context-protocol,rag,search
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.11
Requires-Dist: google-genai>=1.0
Requires-Dist: mcp-fts5-starter>=0.1.0
Description-Content-Type: text/markdown

# mcp-fts5-starter-gemini

> Gemini embedder for [`mcp-fts5-starter`](https://github.com/zx22413/mcp-fts5-starter) — drop-in hybrid BM25 + dense retrieval via Google's Gemini Embeddings API.

[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.11+-blue)](pyproject.toml)

## What this is

The parent project ships with [an `Embedder` Protocol](https://github.com/zx22413/mcp-fts5-starter/blob/main/src/mcp_fts5_starter/embedding.py) and stops short of any actual provider — by design. This repo is the canonical example of that Protocol implemented against a real backend, so you can graduate from BM25-only to hybrid retrieval (BM25 + dense vectors fused with RRF) by changing one line:

```python
from mcp_fts5_starter.search import SearchDB
from mcp_fts5_starter_gemini import GeminiEmbedder

db = SearchDB("data/index.db", embedder=GeminiEmbedder())  # GOOGLE_API_KEY in env
```

Use this as a working reference. To wire OpenAI, Ollama, sentence-transformers, or anything else, copy the ~150 lines in [`src/mcp_fts5_starter_gemini/embedder.py`](src/mcp_fts5_starter_gemini/embedder.py) and swap the API call.

## Quick demo (no API key needed)

The repo ships with five sample notes and a JSON cache of their pre-computed embeddings. The demo runs the same four paraphrased queries against two indexes — BM25 only and BM25 + Gemini — to show what hybrid retrieval actually buys you:

```
git clone https://github.com/zx22413/mcp-fts5-starter-gemini
cd mcp-fts5-starter-gemini
uv sync
uv run python scripts/demo.py
```

Real output:

```
Query: 'speeding up code with memoization'
  BM25 only       :
    (no hits)
  BM25 + Gemini   :
    [ +0.016]  Caching strategies                (notes/caching.md)

Query: 'managing many simultaneous database connections'
  BM25 only       :
    (no hits)
  BM25 + Gemini   :
    [ +0.016]  Database connection pooling       (notes/connection-pooling.md)

Query: 'throttling clients that send too many requests'
  BM25 only       :
    (no hits)
  BM25 + Gemini   :
    [ +0.016]  API rate limits                   (notes/rate-limits.md)
    ...

Query: 'deferring slow work outside the request'
  BM25 only       :
    (no hits)
  BM25 + Gemini   :
    [ +0.016]  Background job queues             (notes/job-queues.md)
    ...
```

The queries are intentionally written without any of the keywords the documents actually use. BM25 has zero recall on all four. Gemini bridges the semantic gap and the right document surfaces every time.

## Real usage

```python
import os
from mcp_fts5_starter.search import SearchDB
from mcp_fts5_starter.ingest import sync
from mcp_fts5_starter_gemini import GeminiEmbedder

embedder = GeminiEmbedder(api_key=os.environ["GOOGLE_API_KEY"])
db = SearchDB("data/index.db", embedder=embedder)
sync(db, "data/my-notes")             # embeds each doc once during ingest
results = db.search("how do I cache?") # query-time embedding + RRF fusion
```

The `Embedder` Protocol is just `embed(texts: list[str]) -> list[list[float]]` — `GeminiEmbedder` adds:

- API-key resolution (`GOOGLE_API_KEY` → `GEMINI_API_KEY` → constructor arg)
- Batching to Gemini's 100-input cap
- Exponential-backoff retries on `429 Too Many Requests`
- A defensive raise when Gemini's response is missing the expected shape

## Configuration

| Env var | Purpose |
|---------|---------|
| `GOOGLE_API_KEY` | Primary Gemini API key |
| `GEMINI_API_KEY` | Fallback — checked if the above is unset |

| Constructor arg | Default | Notes |
|-----------------|---------|-------|
| `model` | `"gemini-embedding-001"` | Pin to a specific model so dimensions are stable |
| `output_dim` | `1536` | Supported: 128 / 256 / 512 / 768 / 1536 / 3072 |
| `task_type` | `None` (symmetric `SEMANTIC_SIMILARITY`) | Set `"RETRIEVAL_DOCUMENT"` / `"RETRIEVAL_QUERY"` for asymmetric retrieval |
| `batch_size` | `100` | Gemini's per-call cap |

### Task type and the symmetry trade-off

The parent's `Embedder` Protocol takes a flat `list[str]` — there's no in-band signal for "is this a document or a query." `GeminiEmbedder` defaults to `SEMANTIC_SIMILARITY`, which is symmetric — cosine similarity stays meaningful in both directions.

Gemini's docs claim a small recall improvement from using `RETRIEVAL_DOCUMENT` for ingest and `RETRIEVAL_QUERY` for search. To get that, instantiate two embedders and wire them at different points in your stack. For most personal-corpus workloads, the symmetric default is good enough.

## Refreshing the embedding cache

The committed `data/sample/embeddings.json` was generated by:

```
GEMINI_API_KEY=... uv run python scripts/precompute-embeddings.py
```

Re-run that script if you change the sample corpus, the demo queries, or the embedding model/dimension. The keys in the JSON are SHA-256 hashes of normalized text — ChangeLog records every refresh.

## Status

🚧 **Alpha — pre-PyPI.** Install from source: `pip install git+https://github.com/zx22413/mcp-fts5-starter-gemini`. v0.1.0 ships to PyPI once the Trusted Publisher is configured.

## Companion repos

| Repo | Role |
|------|------|
| [`mcp-fts5-starter`](https://github.com/zx22413/mcp-fts5-starter) | The MCP server template + FTS5 index this plugs into |
| [`forget-rag`](https://github.com/zx22413/forget-rag) | Memory-decay layer for FTS5 — different problem, same backend |

## License

MIT — see [LICENSE](LICENSE).
