Metadata-Version: 2.4
Name: semantic-cache-lib
Version: 0.1.0
Summary: Semantic caching for LLM calls via vector similarity search
Project-URL: Homepage, https://github.com/iamsuryansh/semantic-cache-lib
Project-URL: Repository, https://github.com/iamsuryansh/semantic-cache-lib
Project-URL: Issues, https://github.com/iamsuryansh/semantic-cache-lib/issues
Author: Suryansh
License: MIT License
        
        Copyright (c) 2026 Suryansh
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: ai,cache,embeddings,llm,nlp,semantic
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Provides-Extra: all
Requires-Dist: chromadb>=0.4; extra == 'all'
Requires-Dist: faiss-cpu>=1.7; extra == 'all'
Requires-Dist: httpx>=0.24; extra == 'all'
Requires-Dist: openai>=1.0; extra == 'all'
Requires-Dist: redis>=5.0; extra == 'all'
Requires-Dist: redisvl>=0.1; extra == 'all'
Requires-Dist: requests>=2.28; extra == 'all'
Requires-Dist: sentence-transformers>=2.2; extra == 'all'
Provides-Extra: chroma
Requires-Dist: chromadb>=0.4; extra == 'chroma'
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: numpy>=1.24; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: faiss
Requires-Dist: faiss-cpu>=1.7; extra == 'faiss'
Provides-Extra: huggingface
Requires-Dist: httpx>=0.24; extra == 'huggingface'
Requires-Dist: requests>=2.28; extra == 'huggingface'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Provides-Extra: redis
Requires-Dist: redis>=5.0; extra == 'redis'
Requires-Dist: redisvl>=0.1; extra == 'redis'
Provides-Extra: transformers
Requires-Dist: sentence-transformers>=2.2; extra == 'transformers'
Description-Content-Type: text/markdown

# semantic-cache-lib

[![PyPI version](https://badge.fury.io/py/semantic-cache-lib.svg)](https://pypi.org/project/semantic-cache-lib/)
[![CI](https://github.com/iamsuryansh/semantic-cache-lib/actions/workflows/ci.yml/badge.svg)](https://github.com/iamsuryansh/semantic-cache-lib/actions)
[![Python](https://img.shields.io/pypi/pyversions/semantic-cache-lib)](https://pypi.org/project/semantic-cache-lib/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

**Semantic caching for LLM calls via vector similarity search.**

Instead of exact-match caching, `semantic-cache-lib` embeds your queries and finds semantically similar past answers — so `"What is the capital of France?"` and `"Tell me the capital city of France"` both hit the same cache entry.

Works as a **decorator** on any function that calls an LLM. Fully configurable: choose your embedder, backend, similarity threshold, and TTL.

---

## Installation

```bash
# Core (in-memory backend only)
pip install semantic-cache-lib

# With specific extras
pip install semantic-cache-lib[openai]        # OpenAI embedder
pip install semantic-cache-lib[huggingface]   # HuggingFace Inference API embedder
pip install semantic-cache-lib[transformers]  # Local SentenceTransformers
pip install semantic-cache-lib[redis]         # Redis backend
pip install semantic-cache-lib[faiss]         # FAISS backend
pip install semantic-cache-lib[chroma]        # ChromaDB backend
pip install semantic-cache-lib[all]           # Everything
```

---

## Quick Start

```python
from semantic_cache import SemanticCache
from semantic_cache.embedders import HuggingFaceEmbedder

cache = SemanticCache(
    embedder=HuggingFaceEmbedder(
        token="hf_...",                              # or set HF_TOKEN env var
        model="sentence-transformers/all-MiniLM-L6-v2",
    ),
    threshold=0.90,   # 0.0–1.0 cosine similarity cutoff
    ttl=3600,         # cache entries expire after 1 hour (optional)
)

@cache
def call_llm(prompt: str) -> str:
    # your actual LLM call here
    return openai_client.chat.completions.create(...).choices[0].message.content

# First call → hits the LLM
response = call_llm("What is the capital of France?")

# Second call (semantically similar) → served from cache, LLM never called
response = call_llm("Tell me the capital city of France")
```

### Async support

```python
@cache
async def call_llm_async(prompt: str) -> str:
    return await async_openai_client.chat.completions.create(...)

response = await call_llm_async("What is the capital of France?")
```

### Programmatic API

```python
result = cache.get_or_set("my query", lambda: llm("my query"))
result = await cache.aget_or_set("my query", async_llm_call)
```

---

## Embedders

### HuggingFace Inference API

No GPU needed — calls the remote HF API.

```python
from semantic_cache.embedders import HuggingFaceEmbedder

embedder = HuggingFaceEmbedder(
    token="hf_...",        # or HF_TOKEN env var
    model="sentence-transformers/all-MiniLM-L6-v2",
)
```

### Local SentenceTransformers

Runs fully on-device. Downloads model from HF Hub on first use.

```python
from semantic_cache.embedders import SentenceTransformerEmbedder

embedder = SentenceTransformerEmbedder(
    model="all-MiniLM-L6-v2",   # any sentence-transformers model
    device="cpu",                 # or "cuda"
)
```

### OpenAI

```python
from semantic_cache.embedders import OpenAIEmbedder

embedder = OpenAIEmbedder(
    api_key="sk-...",             # or OPENAI_API_KEY env var
    model="text-embedding-3-small",
)
```

---

## Backends

### In-Memory (default)

```python
from semantic_cache.backends import MemoryBackend

backend = MemoryBackend(
    save_path="./cache.pkl",   # optional: persist across restarts
)
```

### Redis

Requires [Redis Stack](https://redis.io/docs/stack/) for vector search.

```python
from semantic_cache.backends import RedisBackend

backend = RedisBackend(
    url="redis://localhost:6379",
    vector_dim=384,              # must match your embedder's output dimension
)
```

### FAISS

```python
from semantic_cache.backends import FAISSBackend

backend = FAISSBackend(
    vector_dim=384,
    save_path="./faiss_cache",   # optional: persist across restarts
)
```

### ChromaDB

```python
from semantic_cache.backends import ChromaBackend

backend = ChromaBackend(
    url="http://localhost:8000",   # omit for in-process ephemeral client
    collection="llm_cache",
)
```

---

## Full Configuration

```python
from semantic_cache import SemanticCache
from semantic_cache.embedders import OpenAIEmbedder
from semantic_cache.backends import RedisBackend

cache = SemanticCache(
    embedder=OpenAIEmbedder(api_key="sk-..."),
    backend=RedisBackend(url="redis://localhost:6379", vector_dim=1536),
    threshold=0.92,                # similarity cutoff (default: 0.85)
    ttl=86400,                     # 24h TTL (default: None = no expiry)
    top_k=1,                       # candidates to retrieve (default: 1)
    on_hit=lambda q, r, s: print(f"HIT  sim={s:.3f} | {q[:60]}"),
    on_miss=lambda q: print(f"MISS | {q[:60]}"),
)
```

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `embedder` | `AbstractEmbedder` | required | Embedding model |
| `backend` | `AbstractBackend` | `MemoryBackend()` | Storage backend |
| `threshold` | `float` | `0.85` | Min cosine similarity for a cache hit |
| `ttl` | `int \| None` | `None` | Entry expiry in seconds |
| `top_k` | `int` | `1` | Number of candidates to retrieve before thresholding |
| `on_hit` | `Callable` | `None` | Called with `(query, response, similarity)` on hit |
| `on_miss` | `Callable` | `None` | Called with `(query,)` on miss |

---

## Observability

```python
stats = cache.stats()
# CacheStats(hits=42, misses=8, total=50, hit_rate=84.00%, avg_similarity=0.9541)

cache.reset_stats()
cache.clear()          # remove all cache entries
```

### Logging

Enable structured logging at any level:

```python
import logging
logging.getLogger("semantic_cache").setLevel(logging.DEBUG)
```

---

## Bring Your Own Embedder / Backend

```python
from semantic_cache.embedders import AbstractEmbedder
from semantic_cache.backends import AbstractBackend

class MyEmbedder(AbstractEmbedder):
    def embed(self, text: str) -> list[float]:
        return my_model.encode(text)

class MyBackend(AbstractBackend):
    def store(self, key, vector, response, ttl=None): ...
    def search(self, vector, top_k=1) -> list[tuple[str, float]]: ...
    def delete(self, key): ...
    def clear(self): ...
```

---

## Publishing / PyPI

```bash
# Install build tools
pip install hatch

# Build
hatch build

# Publish (or push a git tag to trigger GitHub Actions)
hatch publish
```

---

## License

MIT — see [LICENSE](LICENSE).
