Metadata-Version: 2.4
Name: temporal-cache-llm
Version: 0.1.0
Summary: Time-aware semantic LLM cache — automatically invalidates stale AI answers
Project-URL: Homepage, https://github.com/shubhamdusane/temporal-cache
Project-URL: Repository, https://github.com/shubhamdusane/temporal-cache
Author-email: Shubham Dusane <sdusane4@gmail.com>
License: MIT
License-File: LICENSE
Keywords: ai,anthropic,cache,cost-optimization,llm,openai,semantic-cache
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Requires-Dist: numpy>=1.24.0
Provides-Extra: all
Requires-Dist: anthropic>=0.30.0; extra == 'all'
Requires-Dist: openai>=1.30.0; extra == 'all'
Requires-Dist: redis>=5.0.0; extra == 'all'
Requires-Dist: sentence-transformers>=2.6.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.30.0; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: build>=1.2.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.5.0; extra == 'dev'
Requires-Dist: twine>=5.0.0; extra == 'dev'
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=2.6.0; extra == 'embeddings'
Provides-Extra: openai
Requires-Dist: openai>=1.30.0; extra == 'openai'
Provides-Extra: redis
Requires-Dist: redis>=5.0.0; extra == 'redis'
Provides-Extra: test
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'test'
Requires-Dist: pytest>=8.0.0; extra == 'test'
Description-Content-Type: text/markdown

# temporal-cache

**Time-aware semantic caching for LLM responses**

[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Tests](https://img.shields.io/badge/tests-16-passing-brightgreen)](#testing)

> Cache LLM responses by semantic meaning AND freshness. Stop serving stale answers.

---

## Table of Contents

- [What is temporal-cache?](#what-is-temporal-cache)
- [The Problem](#the-problem)
- [How It Works](#how-it-works)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Python API Reference](#python-api-reference)
- [Staleness Classes](#staleness-classes)
- [Architecture](#architecture)
- [Configuration](#configuration)
- [Testing](#testing)
- [Contributing](#contributing)
- [Roadmap](#roadmap)
- [License](#license)

---

## What is temporal-cache?

`temporal-cache` is a semantic cache for LLM API responses that understands **when** an answer was relevant, not just **what** it says.

Traditional semantic caches store responses and return them when a similar query arrives. But they don't know if that response is still *correct*. A cached answer about "current US president" from 2024 is semantically relevant in 2026 — but wrong.

`temporal-cache` solves this with a **StalenessClass** classifier that evaluates how time-sensitive a query is, and an optional **LLM-based semantic classifier** that detects when cached content has drifted from current facts.

**Use cases:**

- Knowledge bases with evolving information
- Customer support bots with changing policies
- Financial/news Q&A systems
- Any LLM application where answers expire

---

## The Problem

```
┌─────────────────────────────────────────────────────────────────┐
│  Traditional Semantic Cache                                      │
│                                                                  │
│  Query: "What is the capital of France?"                         │
│  Cache hit → "Paris" ✓                                           │
│                                                                  │
│  Query: "What's the current CEO of Tesla?"                       │
│  Cache hit → "Elon Musk" (cached 2024)                           │
│  Reality: CEO changed in 2025 ✗                                   │
│                                                                  │
│  Problem: Cache doesn't know the answer is stale                 │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│  temporal-cache                                                  │
│                                                                  │
│  Query: "What's the current CEO of Tesla?"                       │
│  StalenessClass: DYNAMIC (changes frequently)                    │
│  Action: Revalidate → check if cache is still valid              │
│  Result: Fresh answer or cache with metadata about freshness     │
└─────────────────────────────────────────────────────────────────┘
```

---

## How It Works

### Cache Flow

```
                          ┌──────────────┐
                          │   Incoming   │
                          │    Query     │
                          └──────┬───────┘
                                 │
                                 ▼
                    ┌────────────────────────┐
                    │  StalenessClass        │
                    │  Classifier            │
                    │                        │
                    │  STATIC │ EVOLVING │   │
                    │  SEMI-  │ DYNAMIC  │   │
                    │  STATIC │          │   │
                    └────────┬───────────┘
                             │
              ┌──────────────┼──────────────┐
              │              │              │
              ▼              ▼              ▼
         ┌─────────┐   ┌─────────┐   ┌─────────┐
         │ STATIC  │   │ SEMI-   │   │ DYNAMIC │
         │         │   │ STATIC  │   │         │
         │ Cache   │   │ Cache   │   │ Reval-  │
         │ Forever │   │ + TTL   │   │ idate   │
         └─────────┘   └─────────┘   └─────────┘
              │              │              │
              │              │              │
              ▼              ▼              ▼
         ┌─────────────────────────────────────────┐
         │           Semantic Similarity           │
         │              (embedding)                │
         └──────────────────┬──────────────────────┘
                            │
              ┌─────────────┴─────────────┐
              │                           │
              ▼                           ▼
         Hit (≥ threshold)          Miss
              │                           │
              ▼                           ▼
         ┌──────────┐              ┌──────────────┐
         │ Return   │              │ Call LLM     │
         │ Cached   │              │ Store with   │
         │ Response │              │ metadata     │
         └──────────┘              └──────────────┘
```

### Staleness Classification

Each query is classified into one of four staleness classes:

```
┌──────────────────────────────────────────────────────────────────┐
│                                                                  │
│  Query Analysis                                                  │
│  ═══════════════                                                 │
│                                                                  │
│  "What is 2+2?"                    → STATIC                     │
│  "What is Python?"                 → STATIC                     │
│  "Explain quantum computing"       → SEMI_STATIC                │
│  "What are Python best practices?" → SEMI_STATIC                │
│  "Who is the current US president?"→ DYNAMIC                    │
│  "Tesla stock price today"         → DYNAMIC                    │
│  "Latest news about OpenAI"        → EVOLVING                   │
│  "What changed in Python 3.12?"    → EVOLVING                   │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘
```

---

## Installation

```bash
pip install temporal-cache
```

From source:

```bash
git clone https://github.com/yourusername/temporal-cache.git
cd temporal-cache
pip install -e .
```

With Redis support:

```bash
pip install temporal-cache[redis]
```

### Requirements

- Python 3.9+
- `sentence-transformers` (for embeddings)
- `redis` (optional, for Redis backend)

---

## Quick Start

```python
from temporal_cache import TemporalCache

# Initialize with default settings (in-memory backend)
cache = TemporalCache()

# Or with Redis backend
cache = TemporalCache(
    backend="redis",
    redis_url="redis://localhost:6379",
    embedding_model="all-MiniLM-L6-v2"
)

# Store a response
cache.set(
    query="What is the capital of France?",
    response="Paris is the capital of France.",
    metadata={"source": "wikipedia", "verified": True}
)

# Retrieve a response (semantic match)
result = cache.get("What's the capital of France?")

if result:
    print(f"Cache hit: {result.response}")
    print(f"Staleness: {result.staleness_class}")
    print(f"Freshness: {result.freshness_score}")
else:
    print("Cache miss - call your LLM here")
```

---

## Python API Reference

### `TemporalCache`

```python
TemporalCache(
    backend: str = "memory",           # "memory" or "redis"
    redis_url: str = "redis://localhost:6379",
    embedding_model: str = "all-MiniLM-L6-v2",
    similarity_threshold: float = 0.85,
    ttl_config: dict = None,           # Custom TTL per staleness class
    llm_classifier: bool = False,      # Enable LLM-based revalidation
    classifier_model: str = None       # Model for LLM classifier
)
```

#### Methods

**`set(query, response, metadata=None, staleness_class=None)`**

Store a query-response pair in the cache.

| Parameter | Type | Description |
|-----------|------|-------------|
| `query` | `str` | The input query |
| `response` | `str` | The LLM response to cache |
| `metadata` | `dict` | Optional metadata (source, timestamp, etc.) |
| `staleness_class` | `StalenessClass` | Override auto-detected class |

**`get(query, force_refresh=False)`**

Retrieve a cached response by semantic similarity.

| Parameter | Type | Description |
|-----------|------|-------------|
| `query` | `str` | The query to look up |
| `force_refresh` | `bool` | Bypass cache, always return None |

Returns `CacheResult` or `None`.

**`invalidate(query=None, pattern=None)`**

Remove entries from cache.

```python
# Invalidate specific query
cache.invalidate(query="What is the capital of France?")

# Invalidate by pattern
cache.invalidate(pattern="*stock*")
```

**`stats()`**

Return cache statistics.

```python
stats = cache.stats()
# {
#     "total_entries": 1523,
#     "hits": 892,
#     "misses": 231,
#     "stale_rejections": 45,
#     "avg_freshness_score": 0.87
# }
```

### `CacheResult`

```python
@dataclass
class CacheResult:
    response: str
    similarity_score: float
    staleness_class: StalenessClass
    freshness_score: float          # 0.0 (stale) to 1.0 (fresh)
    cached_at: datetime
    metadata: dict
```

### `StalenessClass`

```python
class StalenessClass(Enum):
    STATIC = "static"           # Never changes
    SEMI_STATIC = "semi_static" # Changes rarely
    EVOLVING = "evolving"       # Changes periodically
    DYNAMIC = "dynamic"         # Changes frequently
```

---

## Staleness Classes

| Class | Description | TTL | Revalidation | Examples |
|-------|-------------|-----|--------------|----------|
| **STATIC** | Facts that don't change | ∞ | Never | "What is π?", "2+2", "What is DNA?" |
| **SEMI_STATIC** | Rarely changes | 30 days | On major events | "Python docs", "Physics laws", "Best practices" |
| **EVOLVING** | Changes periodically | 24 hours | On schedule | "Python 3.12 features", "Company policies" |
| **DYNAMIC** | Changes constantly | 1 hour | Every query | "Stock price", "Current president", "Live news" |

### Custom TTL Configuration

```python
cache = TemporalCache(
    ttl_config={
        "static": None,              # No expiration
        "semi_static": 86400 * 30,   # 30 days
        "evolving": 86400,           # 1 day
        "dynamic": 3600              # 1 hour
    }
)
```

---

## Architecture

```
temporal_cache/
├── __init__.py
├── cache.py              # TemporalCache engine
├── classifiers/
│   ├── __init__.py
│   ├── staleness.py      # StalenessClass rule-based classifier
│   └── llm.py            # LLM-based semantic classifier
├── backends/
│   ├── __init__.py
│   ├── memory.py         # In-memory backend (dict)
│   └── redis.py          # Redis backend
├── embeddings.py         # Embedding generation
├── models.py             # CacheResult, CacheEntry dataclasses
├── config.py             # Configuration and defaults
└── utils.py              # Helpers
```

### Data Flow

```
┌─────────────────────────────────────────────────────────────────────┐
│                                                                     │
│  User Query                                                         │
│       │                                                             │
│       ▼                                                             │
│  ┌─────────────┐     ┌──────────────────┐                           │
│  │ Staleness   │────▶│ Embedding        │                           │
│  │ Classifier  │     │ Generator        │                           │
│  └──────┬──────┘     └────────┬─────────┘                           │
│         │                     │                                     │
│         ▼                     ▼                                     │
│  ┌─────────────┐     ┌──────────────────┐                           │
│  │ TTL         │     │ Backend          │                           │
│  │ Manager     │     │ (Memory/Redis)   │                           │
│  └──────┬──────┘     └────────┬─────────┘                           │
│         │                     │                                     │
│         └──────────┬──────────┘                                     │
│                    │                                                │
│                    ▼                                                │
│         ┌──────────────────────┐                                    │
│         │ Response Validator   │                                    │
│         │                      │                                    │
│         │ - Check similarity   │                                    │
│         │ - Check freshness    │                                    │
│         │ - LLM revalidation   │                                    │
│         └──────────┬───────────┘                                    │
│                    │                                                │
│         ┌──────────┴──────────┐                                     │
│         │                     │                                     │
│         ▼                     ▼                                     │
│    Cache Hit              Cache Miss                                │
│    (return)               (call LLM, store)                         │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Configuration

### Environment Variables

```bash
TEMPORAL_CACHE_BACKEND=redis
TEMPORAL_CACHE_REDIS_URL=redis://localhost:6379
TEMPORAL_CACHE_SIMILARITY_THRESHOLD=0.85
TEMPORAL_CACHE_DEFAULT_TTL=3600
```

### Configuration File

```yaml
# temporal_cache.yaml
backend: redis
redis_url: redis://localhost:6379
embedding_model: all-MiniLM-L6-v2
similarity_threshold: 0.85
llm_classifier:
  enabled: true
  model: gpt-4
  temperature: 0.0
ttl:
  static: null
  semi_static: 2592000
  evolving: 86400
  dynamic: 3600
```

---

## Testing

Run the test suite:

```bash
# All tests
pytest

# With coverage
pytest --cov=temporal_cache --cov-report=html

# Verbose output
pytest -v
```

The test suite includes **16 tests** covering:

- Cache storage and retrieval
- Semantic similarity matching
- Staleness classification accuracy
- TTL expiration behavior
- Redis backend integration
- LLM classifier revalidation
- Edge cases and error handling

---

## Contributing

Contributions are welcome! Here's how to get started:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Run the tests (`pytest`)
5. Submit a pull request

### Development Setup

```bash
git clone https://github.com/yourusername/temporal-cache.git
cd temporal-cache
pip install -e ".[dev]"
pre-commit install
```

### Code Style

- Follow PEP 8
- Use type hints
- Write docstrings for public APIs
- Add tests for new features

---

## Roadmap

- [ ] **v0.2.0** - Async support (`async/await`)
- [ ] **v0.2.0** - PostgreSQL backend
- [ ] **v0.3.0** - Bloom filter pre-check for fast misses
- [ ] **v0.3.0** - Distributed cache invalidation
- [ ] **v0.4.0** - Embedding cache (reuse embeddings across queries)
- [ ] **v0.4.0** - Web dashboard for cache analytics
- [ ] **v1.0.0** - Production-ready release

---

## License

MIT License. See [LICENSE](LICENSE) for details.

---

<p align="center">
  Built with care for the LLM community
</p>
