Metadata-Version: 2.5
Name: waggle-cache
Version: 0.4.0
Summary: Waggle — local-first semantic caching for LLM, RAG, and agent workloads
Project-URL: Homepage, https://github.com/Abhigyan-Shekhar/Waggle-SemCache
Project-URL: Repository, https://github.com/Abhigyan-Shekhar/Waggle-SemCache
Project-URL: Waggle MCP, https://github.com/Abhigyan-Shekhar/Waggle-mcp
Author: Abhigyan Shekhar
License: Apache License
        Version 2.0, January 2004
        http://www.apache.org/licenses/
        
        TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
        1. Definitions.
        
        "License" shall mean the terms and conditions for use, reproduction, and
        distribution as defined by Sections 1 through 9 of this document.
        
        "Licensor" shall mean the copyright owner or entity authorized by the copyright
        owner that is granting the License.
        
        "Legal Entity" shall mean the union of the acting entity and all other entities
        that control, are controlled by, or are under common control with that entity.
        For the purposes of this definition, "control" means (i) the power, direct or
        indirect, to cause the direction or management of such entity, whether by
        contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
        outstanding shares, or (iii) beneficial ownership of such entity.
        
        "You" (or "Your") shall mean an individual or Legal Entity exercising
        permissions granted by this License.
        
        "Source" form shall mean the preferred form for making modifications, including
        but not limited to software source code, documentation source, and configuration
        files.
        
        "Object" form shall mean any form resulting from mechanical transformation or
        translation of a Source form, including but not limited to compiled object code,
        generated documentation, and conversions to other media types.
        
        "Work" shall mean the work of authorship, whether in Source or Object form, made
        available under the License, as indicated by a copyright notice that is included
        in or attached to the work (an example is provided in the Appendix below).
        
        "Derivative Works" shall mean any work, whether in Source or Object form, that
        is based on (or derived from) the Work and for which the editorial revisions,
        annotations, elaborations, or other modifications represent, as a whole, an
        original work of authorship. For the purposes of this License, Derivative Works
        shall not include works that remain separable from, or merely link (or bind by
        name) to the interfaces of, the Work and Derivative Works thereof.
        
        "Contribution" shall mean any work of authorship, including the original version
        of the Work and any modifications or additions to that Work or Derivative Works
        thereof, that is intentionally submitted to Licensor for inclusion in the Work
        by the copyright owner or by an individual or Legal Entity authorized to submit
        on behalf of the copyright owner. For the purposes of this definition,
        "submitted" means any form of electronic, verbal, or written communication sent
        to the Licensor or its representatives, including but not limited to
        communication on electronic mailing lists, source code control systems, and
        issue tracking systems that are managed by, or on behalf of, the Licensor for
        the purpose of discussing and improving the Work, but excluding communication
        that is conspicuously marked or otherwise designated in writing by the copyright
        owner as "Not a Contribution."
        
        "Contributor" shall mean Licensor and any individual or Legal Entity on behalf
        of whom a Contribution has been received by Licensor and subsequently
        incorporated within the Work.
        
        2. Grant of Copyright License. Subject to the terms and conditions of this
        License, each Contributor hereby grants to You a perpetual, worldwide,
        non-exclusive, no-charge, royalty-free, irrevocable copyright license to
        reproduce, prepare Derivative Works of, publicly display, publicly perform,
        sublicense, and distribute the Work and such Derivative Works in Source or
        Object form.
        
        3. Grant of Patent License. Subject to the terms and conditions of this License,
        each Contributor hereby grants to You a perpetual, worldwide, non-exclusive,
        no-charge, royalty-free, irrevocable (except as stated in this section) patent
        license to make, have made, use, offer to sell, sell, import, and otherwise
        transfer the Work, where such license applies only to those patent claims
        licensable by such Contributor that are necessarily infringed by their
        Contribution(s) alone or by combination of their Contribution(s) with the Work
        to which such Contribution(s) was submitted. If You institute patent litigation
        against any entity (including a cross-claim or counterclaim in a lawsuit)
        alleging that the Work or a Contribution incorporated within the Work
        constitutes direct or contributory patent infringement, then any patent licenses
        granted to You under this License for that Work shall terminate as of the date
        such litigation is filed.
        
        4. Redistribution. You may reproduce and distribute copies of the Work or
        Derivative Works thereof in any medium, with or without modifications, and in
        Source or Object form, provided that You meet the following conditions:
        
        (a) You must give any other recipients of the Work or Derivative Works a copy of
        this License; and
        
        (b) You must cause any modified files to carry prominent notices stating that
        You changed the files; and
        
        (c) You must retain, in the Source form of any Derivative Works that You
        distribute, all copyright, patent, trademark, and attribution notices from the
        Source form of the Work, excluding those notices that do not pertain to any part
        of the Derivative Works; and
        
        (d) If the Work includes a "NOTICE" text file as part of its distribution, then
        any Derivative Works that You distribute must include a readable copy of the
        attribution notices contained within such NOTICE file, excluding those notices
        that do not pertain to any part of the Derivative Works, in at least one of the
        following places: within a NOTICE text file distributed as part of the
        Derivative Works; within the Source form or documentation, if provided along
        with the Derivative Works; or, within a display generated by the Derivative
        Works, if and wherever such third-party notices normally appear. The contents of
        the NOTICE file are for informational purposes only and do not modify the
        License. You may add Your own attribution notices within Derivative Works that
        You distribute, alongside or as an addendum to the NOTICE text from the Work,
        provided that such additional attribution notices cannot be construed as
        modifying the License.
        
        You may add Your own copyright statement to Your modifications and may provide
        additional or different license terms and conditions for use, reproduction, or
        distribution of Your modifications, or for any such Derivative Works as a whole,
        provided Your use, reproduction, and distribution of the Work otherwise complies
        with the conditions stated in this License.
        
        5. Submission of Contributions. Unless You explicitly state otherwise, any
        Contribution intentionally submitted for inclusion in the Work by You to the
        Licensor shall be under the terms and conditions of this License, without any
        additional terms or conditions. Notwithstanding the above, nothing herein shall
        supersede or modify the terms of any separate license agreement you may have
        executed with Licensor regarding such Contributions.
        
        6. Trademarks. This License does not grant permission to use the trade names,
        trademarks, service marks, or product names of the Licensor, except as required
        for reasonable and customary use in describing the origin of the Work and
        reproducing the content of the NOTICE file.
        
        7. Disclaimer of Warranty. Unless required by applicable law or agreed to in
        writing, Licensor provides the Work (and each Contributor provides its
        Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
        KIND, either express or implied, including, without limitation, any warranties
        or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
        PARTICULAR PURPOSE. You are solely responsible for determining the
        appropriateness of using or redistributing the Work and assume any risks
        associated with Your exercise of permissions under this License.
        
        8. Limitation of Liability. In no event and under no legal theory, whether in
        tort, contract, or otherwise, unless required by applicable law (such as
        deliberate and grossly negligent acts) or agreed to in writing, shall any
        Contributor be liable to You for damages, including any direct, indirect,
        special, incidental, or consequential damages of any character arising as a
        result of this License or out of the use or inability to use the Work (including
        but not limited to damages for loss of goodwill, work stoppage, computer failure
        or malfunction, or any and all other commercial damages or losses), even if such
        Contributor has been advised of the possibility of such damages.
        
        9. Accepting Warranty or Additional Liability. While redistributing the Work or
        Derivative Works thereof, You may choose to offer, and charge a fee for,
        acceptance of support, warranty, indemnity, or other liability obligations
        and/or rights consistent with this License. However, in accepting such
        obligations, You may act only on Your own behalf and on Your sole responsibility,
        not on behalf of any other Contributor, and only if You agree to indemnify,
        defend, and hold each Contributor harmless for any liability incurred by, or
        claims asserted against, such Contributor by reason of your accepting any such
        warranty or additional liability.
        
        END OF TERMS AND CONDITIONS
        
        APPENDIX: How to apply the Apache License to your work.
        
        To apply the Apache License to your work, attach the following boilerplate
        notice, with the fields enclosed by brackets "[]" replaced with your own
        identifying information. (Don't include the brackets!) The text should be
        enclosed in the appropriate comment syntax for the file format. We also
        recommend that a file or class name and description of purpose be included on
        the same "printed page" as the copyright notice for easier identification within
        third-party archives.
        
        Copyright 2026 Waggle contributors
        
        Licensed under the Apache License, Version 2.0 (the "License");
        you may not use this file except in compliance with the License.
        You may obtain a copy of the License at
        
            http://www.apache.org/licenses/LICENSE-2.0
        
        Unless required by applicable law or agreed to in writing, software
        distributed under the License is distributed on an "AS IS" BASIS,
        WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
        See the License for the specific language governing permissions and
        limitations under the License.
License-File: LICENSE
Keywords: embeddings,llm,rag,semantic-cache,sqlite,waggle
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Provides-Extra: ann
Requires-Dist: hnswlib>=0.8; extra == 'ann'
Requires-Dist: numpy>=1.26; extra == 'ann'
Provides-Extra: cli
Requires-Dist: rich>=13; extra == 'cli'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3; extra == 'langchain'
Provides-Extra: litellm
Requires-Dist: litellm>=1.60; extra == 'litellm'
Provides-Extra: local
Requires-Dist: sentence-transformers>=2.6; extra == 'local'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Description-Content-Type: text/markdown

<p align="center">
  <a href="https://github.com/Abhigyan-Shekhar/Waggle-SemCache">
    <img src="https://raw.githubusercontent.com/Abhigyan-Shekhar/Waggle-SemCache/main/assets/waggle-logo.png" alt="Waggle logo" width="520" />
  </a>
</p>

<h1 align="center">Waggle</h1>

<p align="center"><strong>Repository: Waggle-SemCache</strong></p>

<p align="center">
  <strong>Built by <a href="https://github.com/Abhigyan-Shekhar">Abhigyan Shekhar</a></strong>
</p>

Local semantic caching for LLMs, RAG pipelines, and AI agents.

Reuse expensive AI computations when requests are semantically equivalent—without Redis, a vector database, an API key, or a cloud service.

- Local-first and SQLite-backed
- Exact caching in the dependency-free core; semantic matching through pluggable providers
- Context-aware validation, TTLs, metadata, and invalidation
- Sync and async APIs with duplicate-work coalescing
- Atomic LRU/FIFO storage budgets and optional cross-process single-flight
- Inspectable match decisions
- No telemetry

## Quick start

```bash
pip install waggle-cache
```

```python
from waggle import Waggle

cache = Waggle("./cache.db")
answer = cache.get_or_compute(
    key="Explain gradient descent",
    compute=lambda: expensive_llm_call(),
    namespace="education",
)
```

The base package uses only the Python standard library and starts in exact-only mode. It does not install Torch, Transformers, an SDK, or a model. Exact entries do not contain vectors and are not reported as missing from the vector index.

For local semantic matching, install the optional provider and configure it explicitly:

```bash
pip install "waggle-cache[local]"
```

```python
from waggle import Waggle
from waggle.embeddings import LocalEmbeddingProvider

cache = Waggle(
    "./cache.db",
    embedding_provider=LocalEmbeddingProvider(),
)
```

`LocalEmbeddingProvider` lazily downloads `all-MiniLM-L6-v2` on its first semantic operation and then runs locally on CPU. You can instead pass any object implementing `model_id` and `embed(list[str])` to use an embedding API, another local runtime, or an application-owned model.

The bundled local provider is pinned to MiniLM revision `1110a243fdf4706b3f48f1d95db1a4f5529b4d41` so persisted vectors have reproducible identity. Custom unpinned models are explicitly stored with an `@unversioned` identity.

When `semantic` is omitted, Waggle enables semantic lookup only when an embedding provider is configured. Passing `semantic=True` without a provider raises `MissingEmbeddingProviderError` instead of silently pretending to perform semantic matching.

## Bounded local storage

Production caches should not grow forever. Configure entry and/or logical-byte budgets directly on the cache:

```python
cache = Waggle(
    "./cache.db",
    max_entries=10_000,
    max_size_mb=500,
    eviction="lru",  # or "fifo"
)
```

Every insertion purges expired and invalidated rows, commits the new entries, and evicts older live entries inside one short SQLite transaction. Atomic batches larger than the configured capacity raise `CacheCapacityError` without changing the cache. `stats()` reports logical `storage_bytes` and `evictions`; call `vacuum()` explicitly when physical SQLite page reclamation is required.

## Safe semantic reuse

Nearest does not mean reusable. Waggle first tries a normalized exact lookup, then embeds on an exact miss. Semantic candidates must pass every configured gate:

1. same namespace and embedding model;
2. similarity threshold;
3. unexpired and not invalidated;
4. matching context fingerprint;
5. matching metadata filter; and
6. the optional verifier.

```python
result = cache.get(
    "Summarize this document",
    namespace="docs",
    context={"document_version": "v13", "model": "my-model"},
    metadata_filter={"tenant": "acme", "language": "en"},
    debug=True,
)

print(result.hit, result.match_type, result.similarity)
for candidate in result.candidates:
    print(candidate.similarity, candidate.accepted, candidate.reason)
```

Context is canonicalized and SHA-256 fingerprinted; raw context is not stored. Configure `context_fields` or `ignore_context_fields` on `Waggle` to define compatibility. A missing context and a supplied context are intentionally different.

### Application cache policies

Similarity cannot determine whether a workload is cacheable or which business fields must remain identical. Supply a policy to bypass unsafe requests and extract structured reuse invariants before any cache lookup:

```python
from waggle import CachePolicyDecision, CacheRequest, Waggle
from waggle.embeddings import LocalEmbeddingProvider


def support_policy(request: CacheRequest) -> CachePolicyDecision:
    context = request.context or {}
    if context.get("personalized"):
        return CachePolicyDecision.bypass("personalized request")
    return CachePolicyDecision(
        invariants={
            "tenant": context.get("tenant"),
            "knowledge_base_version": context.get("knowledge_base_version"),
            "operation": context.get("operation"),
        },
        policy_version="support-v1",
    )


cache = Waggle(
    "./cache.db",
    embedding_provider=LocalEmbeddingProvider(),
    policy=support_policy,
    context_fields=("model",),
)
```

A bypassed `get_or_compute` runs the function without reading or writing the cache and returns `match_type="bypass"` when `return_result=True`. Invariants and the policy version are canonicalized into a separate SHA-256 fingerprint; raw invariant values are not persisted. A changed invariant is reported as `invariant mismatch`. Increment `policy_version` whenever a policy's compatibility meaning changes.

Policies may also be supplied per call or decorator with `policy=...`. An explicit `set` or `set_many` rejected by policy raises before embedding or writing; a mixed rejected batch remains atomic.

## OpenAI-compatible integration

Bring the provider client configured by your application; the base Waggle installation adds no OpenAI SDK dependency:

```python
from openai import OpenAI

from waggle import Waggle
from waggle.embeddings import LocalEmbeddingProvider
from waggle.integrations.openai import wrap_openai

client = OpenAI()
client = wrap_openai(
    client,
    Waggle("./cache.db", embedding_provider=LocalEmbeddingProvider()),
)

response = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Explain gradient descent"}],
    temperature=0,
)
```

Existing call sites keep the same `client.chat.completions.create(...)` and `client.responses.create(...)` paths and receive native SDK objects on misses and hits. The wrapper fingerprints the model, conversation history, tools, response format, and generation parameters while using the final text user input for semantic matching. Multimodal requests fall back to exact hashed keys and streaming passes through uncached. See the [OpenAI-compatible integration guide](docs/openai-compatible.md).

## LangChain integration

Register Waggle through LangChain's native global cache hook; existing model calls do not change:

```bash
pip install 'waggle-cache[langchain,local]'
```

```python
from langchain_core.globals import set_llm_cache

from waggle import Waggle
from waggle.embeddings import LocalEmbeddingProvider
from waggle.integrations.langchain import WaggleLangChainCache

cache = Waggle("./cache.db", embedding_provider=LocalEmbeddingProvider())
set_llm_cache(WaggleLangChainCache(cache))
answer = model.invoke("Explain gradient descent")
```

The adapter implements `BaseCache` sync and async methods, preserves native `Generation` and `ChatGeneration` values, and fingerprints the complete LangChain model configuration as a strict reuse boundary. It also accepts per-integration namespaces, TTLs, thresholds, exact-only mode, and cache policies. See the [LangChain integration guide](docs/langchain.md).

## LiteLLM integration

Enable Waggle once, then keep existing LiteLLM completion calls unchanged:

```bash
pip install 'waggle-cache[litellm,local]'
```

```python
import litellm

from waggle import Waggle
from waggle.embeddings import LocalEmbeddingProvider
from waggle.integrations.litellm import enable_litellm

cache = Waggle("./cache.db", embedding_provider=LocalEmbeddingProvider())
enable_litellm(cache)
response = litellm.completion(model="openai/gpt-4.1-mini", messages=[...])
```

The native backend supports `completion()` and `acompletion()`, preserves LiteLLM `ModelResponse` objects, and isolates provider/model parameters and conversation history while matching the final text user message semantically. See the [LiteLLM integration guide](docs/litellm.md).

## API

```python
cache.set(key, value, namespace="support", ttl=3600, context={...}, metadata={...})
cache.set_many([("first request", value1), ("second request", value2)], namespace="support")
result = cache.get(key, namespace="support", threshold=0.94)
value = cache.get_or_compute(key, compute=callable, namespace="support")

cache.invalidate(entry_id=result.entry_id)
cache.invalidate(key=key, namespace="support")
cache.invalidate_where({"document_version": "v1"}, namespace="support")
cache.clear(namespace="support")
cache.clear_expired()
cache.stats()
cache.vacuum()
```

`get()` always returns a `CacheResult`. `get_or_compute()` returns the cached/computed value by default; use `return_result=True` for provenance. Values and metadata must be JSON serializable. Waggle never uses pickle.

### Decorators

```python
@cache.semantic(
    namespace="support",
    ttl=3600,
    threshold=0.85,
    key=lambda question, user_id: question,
    context=lambda question, user_id: {"user_id": user_id},
)
def answer(question: str, user_id: str) -> str:
    return call_llm(question)
```

Use `@cache.exact(...)` when only identical normalized inputs may share results. Both decorators support async functions.

### Async

```python
answer = await cache.aget_or_compute(
    key=query,
    compute=lambda: ask_llm(query),
    namespace="support",
)
```

Blocking SQLite and embedding work runs off the event loop. The awaited computation itself stays async. Concurrent identical misses are coalesced per process without holding a global lock during the slow call.

For Gunicorn, multiprocessing, and other multi-worker deployments, opt into SQLite-backed leases:

```python
cache = Waggle(
    "./cache.db",
    cross_process_singleflight=True,
    lease_ttl=60,
)
```

Lease acquisition and renewal use short transactions; the expensive computation runs without a SQLite lock. Waiting workers recheck the cache, crashed owners are replaced after expiry, and a heartbeat protects long-running computations. Only identical scoped request keys are coalesced—semantic neighbors are never merged in flight.

### Large local indexes

The default exact cosine backend is dependency-free and deterministic. For large namespaces, install the optional persistent HNSW backend:

```bash
pip install 'waggle-cache[ann,local]'
```

```python
cache = Waggle(
    "./cache.db",
    embedding_provider=LocalEmbeddingProvider(),
    index_backend="hnsw",
)
```

HNSW files are stored beside the SQLite database, separately for each namespace and embedding model. SQLite remains the source of truth. Every HNSW file carries the corresponding SQLite generation and is rebuilt automatically when missing, stale, corrupt, or incompatible. Candidate similarities are recalculated exactly. Hard compatibility filters run before ranking, and HNSW retrieval widens adaptively until a valid candidate is found, the scope is exhausted, or `max_semantic_candidates` is reached.

## Exact embedding cache vs semantic response cache

An embedding computation cache maps an exact text to its previous vector. A semantic response cache maps a sufficiently equivalent request to a previous computation. For an embedding cache, use `semantic=False` or `@cache.exact`; never semantically reuse an embedding for different text.

## CLI

```bash
waggle -d ./cache.db stats
waggle -d ./cache.db namespaces
waggle -d ./cache.db inspect "How does binary search work?" --namespace docs
waggle -d ./cache.db clear --namespace support
waggle -d ./cache.db clear-expired
waggle -d ./cache.db vacuum
waggle -d ./cache.db rebuild-index
waggle -d ./cache.db doctor
waggle -d ./cache.db --index-backend hnsw rebuild-index
waggle -d ./cache.db benchmark-thresholds benchmarks/paraphrases.json
waggle -d ./cache.db evaluate-policies benchmarks/policy_cases.json --thresholds 0.80 0.85 0.90
```

## Storage, privacy, and recovery

The configured SQLite file contains keys, JSON values, metadata, fingerprints, timestamps, and float32 vectors. WAL mode supports concurrent readers and safe transactions. No data or telemetry leaves the machine, and secrets should be represented by fingerprints or version identifiers—not stored as metadata.

SQLite is the source of truth. The dependency-free cosine index is rebuilt from stored vectors and can later be replaced behind the same boundary by an approximate index. `doctor` runs database integrity and vector compatibility checks.

## Benchmarks

```bash
python benchmarks/benchmark.py --entries 1000
python benchmarks/thresholds.py benchmarks/paraphrases.json
```

The scripts report measurements from the current machine. This README intentionally contains no invented performance numbers. See [benchmark methodology](docs/benchmarks.md), [threshold and exact-index calibration](benchmarks/results-2026-08-09.md), [transactional batching with persistent HNSW](benchmarks/results-2026-08-21-ann.md), and the [multi-domain cache-policy evaluation](benchmarks/results-2026-08-21-policy.md).

The default threshold is `0.85`, not a universal optimum. The generic verifier is intentionally too small to encode domain correctness. Run `evaluate-policies` on your own workload and put cacheability rules, tenant boundaries, versions, languages, operations, and other business constraints in application policy or structured invariants.

## Development

```bash
python -m pip install -e '.[dev]'
pytest
ruff check .
```

See [architecture](docs/architecture.md) and [migrations](docs/migrations.md). Waggle is built by [Abhigyan Shekhar](https://github.com/Abhigyan-Shekhar). This Python cache library is standalone from [Waggle MCP](https://github.com/Abhigyan-Shekhar/Waggle-mcp): it has no MCP runtime dependency and does not reuse the memory engine's knowledge-graph architecture or source code.

## License

Apache-2.0.
