Metadata-Version: 2.4
Name: agenticstack-cache
Version: 0.1.0
Summary: LLM response cache for AgenticStack: provider wrapper with in-memory LRU and SQLite backends
Author: The AgenticStack Authors
License: Apache-2.0
Requires-Python: >=3.11
Requires-Dist: agenticstack>=0.2.0
Description-Content-Type: text/markdown

# agenticstack-cache

LLM response cache for [AgenticStack](https://github.com/AiConglomerate/AgentStack):
serve repeated prompts from a local store instead of paying for the same
completion twice.

## Why a provider wrapper, not a hook plugin

AgenticStack's hook points (`BEFORE_LLM_CALL` / `AFTER_LLM_CALL`) can
observe and transform requests and responses, but they **cannot skip
the provider call** — a filter chain always falls through to the actual
`generate()`. A cache's whole job is to short-circuit that call, so the
cache is implemented as a **provider wrapper**: `CachedProvider` is
itself an `LLMProvider` that wraps any other provider and answers hits
locally. It drops in anywhere a provider does, including
`Agent(provider=CachedProvider(inner))`.

## Install

```bash
pip install agenticstack-cache
```

## Use

```python
from agenticstack_cache import CachedProvider, InMemoryCache, SQLiteCache

# In-memory LRU (per-process)
provider = CachedProvider(inner, cache=InMemoryCache(max_entries=512))

# Persistent across processes (stdlib sqlite3)
provider = CachedProvider(inner, cache=SQLiteCache("cache.db", ttl=3600))

response = await provider.generate(messages)   # miss -> calls inner
response = await provider.generate(messages)   # hit  -> no inner call

provider.hits, provider.misses  # -> 1, 1
provider.stats                  # -> {"hits": 1, "misses": 1}
```

With an agent:

```python
from agenticstack import Agent

agent = Agent(name="Helper", provider=CachedProvider(inner))
```

## Cache key

`sha256` over the request's semantic identity:

- inner provider's model name
- message `(role, content)` pairs
- tool names offered
- sorted extra kwargs

Anything that changes the request (different history, different
temperature kwarg, different tools) is a different key.

## What gets cached

**Only pure-text responses.** Responses carrying `tool_calls` are
passed through uncached — tool-call turns are stateful (their IDs must
pair with fresh tool-result messages), so replaying them would corrupt
conversations. Cached entries store `content`, `model`, and
`finish_reason` only; a hit reconstructs an `LLMResponse` with zero
token usage (no tokens were spent) and `raw_response["cached"] = True`.

## Backends

| Backend | Storage | Eviction | TTL |
|---|---|---|---|
| `InMemoryCache(max_entries=1024, ttl=None)` | process dict | LRU beyond `max_entries` | optional seconds |
| `SQLiteCache(path, ttl=None)` | SQLite file | none (unbounded) | optional seconds |

`ttl=None` (the default) means entries never expire. Expired entries
read as misses and are purged on access.

## Plugin form

`CachePlugin` participates in the plugin runtime (manifest, lifecycle)
but registers no capabilities — a cache must wrap a *specific* provider
instance, so the primary API is the explicit wrapper above.

## Permissions

`filesystem` — the SQLite backend writes a local database file.
