Metadata-Version: 2.4
Name: demystify-platform-cache
Version: 0.3.0
Summary: Production caching seam for AI workloads (Python mirror): exact-match cache, verified semantic cache, embedding cache, singleflight request coalescing, negative cache, and epoch-based invalidation. In-memory reference default; Redis extra.
Project-URL: Homepage, https://github.com/demystify-systems/ai-services-tools/tree/main/packages/platform-cache
Project-URL: Repository, https://github.com/demystify-systems/ai-services-tools
Author: Demystify Systems
License: MIT
License-File: LICENSE
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: httpx<1,>=0.27; extra == 'dev'
Requires-Dist: mypy<2,>=1.13; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest<9,>=8; extra == 'dev'
Requires-Dist: ruff<0.9,>=0.8; extra == 'dev'
Provides-Extra: redis
Requires-Dist: redis<6,>=5; extra == 'redis'
Description-Content-Type: text/markdown

# @demystify/platform-cache · demystify-platform-cache

Production caching seam for AI workloads — the layer that makes the Demystify
suite **fast and cost-efficient** without ever trading away correctness.
Dual-language leaf package: `@demystify/platform-cache` (TypeScript) and
`demystify-platform-cache` (Python), with an **in-memory reference adapter**
(keyless, offline, deterministic — the default) and a **Redis adapter**
(integration-marked, opt-in).

Built from the production playbook, not the demo playbook:

- Real-world semantic-cache hit rates are 20–40%, and the hard problem is
  **false hits** — embedding-close is not meaning-equal ("capital of France" ≈
  "capital of Germany" in dense-embedding space).
- A cache **amplifies whatever it stores** — errors, refusals, and
  low-groundedness answers must never be admitted.
- TTLs don't know when your data changed — **event-driven (epoch)
  invalidation** does.

## What's in the box

| Primitive | What it does | Guarantee |
| --- | --- | --- |
| `ExactCache` + `exactKey` | digest of the full canonical request → value | zero false hits by construction |
| `SemanticResponseCache` | embed → nearest-neighbour ≥ threshold → **verify** before serve; **admission-gate** before store | verified paraphrase hits |
| `TieredCache` | exact tier first, then semantic | free/safe hits absorbed before any similarity math |
| `withEmbeddingCache` | content-hash → vector decorator over any batch embedder | pure-function cache: can never be wrong |
| `SingleFlight` | N concurrent identical misses → 1 upstream call | thundering-herd guard |
| `NegativeCache` | short-TTL tombstones for known-empty results | bounded staleness |
| `EpochStore` | monotonic epoch per scope, folded into keys | O(1) invalidation on ingest/delete |
| `CacheStats` | hit/miss/store/reject events + µUSD saved/spent | measure the **false-hit rate**, not just the hit rate |

Every namespace is **tenant-scoped by the caller** and every key carries a
**version segment** (embedder/model/schema/epoch), so version swaps invalidate
instead of corrupting.

## Quickstart (TypeScript)

```ts
import {
  createMemoryCacheBackend,
  HashCacheEmbedder,
  SemanticResponseCache,
  TieredCache,
} from "@demystify/platform-cache";

const backend = createMemoryCacheBackend();
const semantic = new SemanticResponseCache<string>({
  embedder: new HashCacheEmbedder(), // deterministic reference; swap a real embedder in prod
  store: backend.semantic,
  admission: (answer) =>
    answer.startsWith("ERROR") ? { admit: false, reason: "error response" } : { admit: true },
});
const cache = new TieredCache<string>({
  exact: backend.exact,
  semantic,
  keyVersion: `epoch:${await backend.epochs.current("tenant:corpus")}`,
});

const hit = await cache.lookup("tenant-1", requestPayload, questionText);
if (!hit) {
  const answer = await expensiveCall();
  await cache.store("tenant-1", requestPayload, questionText, answer, { costMicroUsd: 1200 });
}
```

## Quickstart (Python)

```python
from demystify_platform_cache import (
    HashCacheEmbedder, MemoryCacheBackend, SemanticResponseCache, TieredCache,
)

backend = MemoryCacheBackend()
semantic = SemanticResponseCache(embedder=HashCacheEmbedder(), store=backend.semantic)
cache = TieredCache(exact=backend.exact, semantic=semantic, key_version="epoch:0")

hit = await cache.lookup("tenant-1", request_payload, question_text)
```

## The two guards (why this is safe to run in production)

1. **Admission** — a quality gate before *store*: never cache errors, tool
   calls, abstentions, or low-groundedness answers. The gateway and RAG
   integrations pass their own policies.
2. **Verification** — a guard before *serve* on every semantic candidate. The
   deterministic reference is content-token overlap (kills the France/Germany
   class of false hit); real deployments swap a cross-encoder or LLM judge
   behind the same `HitVerifier` signature.

## Who uses it in the suite

- **dmstfy-gateway** — exact + verified-semantic completion cache
  (`DMSTFY_GATEWAY_SEMANTIC_CACHE*`, OFF by default), singleflight, saved-µUSD
  accounting.
- **dmstfy-rag** — embedding cache (ingest + query), retrieval cache and answer
  cache keyed by **corpus epoch** (every ingest/delete invalidates instantly),
  groundedness-gated admission (`DMSTFY_RAG_CACHE*`, OFF by default).
- **dmstfy-cache** (module) — `make cachebench`: the offline proof harness that
  reports per-tier hit rates, false-hit rate, and µUSD saved, ON vs OFF.

## Install

```bash
npm add @demystify/platform-cache          # TypeScript
pip install demystify-platform-cache        # Python (redis extra: [redis])
```

Caches are **never the default** (CONVENTIONS.md §14.7): every consumer wires
them behind an explicit opt-in.

## License

MIT © Demystify Systems
