Metadata-Version: 2.4
Name: django-llm-cache
Version: 0.1.0
Summary: Production-grade, drop-in caching utility for LLM API prompts and responses in Django, cutting token costs and latency.
Home-page: https://github.com/Hafiz-alt/django-llm-cache
Author: Hafiz Muhammed P H
Author-email: hafizmdph88172@gmail.com
Project-URL: Documentation, https://github.com/Hafiz-alt/django-llm-cache#readme
Project-URL: Source, https://github.com/Hafiz-alt/django-llm-cache
Project-URL: Tracker, https://github.com/Hafiz-alt/django-llm-cache/issues
Keywords: django,llm,cache,openai,gemini,groq,claude,anthropic,tokens,caching,redis,ai
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Web Environment
Classifier: Framework :: Django
Classifier: Framework :: Django :: 3.2
Classifier: Framework :: Django :: 4.0
Classifier: Framework :: Django :: 4.1
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Framework :: Django :: 6.0
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Django>=3.2
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-django>=4.5; extra == "dev"
Requires-Dist: flake8>=5.0; extra == "dev"
Requires-Dist: black>=22.0; extra == "dev"
Requires-Dist: twine>=4.0; extra == "dev"
Requires-Dist: build>=0.10.0; extra == "dev"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license-file
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# django-llm-cache ⚡

[![PyPI version](https://img.shields.io/badge/pypi-v0.1.0-blue.svg)](https://pypi.org/project/django-llm-cache/)
[![Python](https://img.shields.io/badge/python-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/)
[![Django](https://img.shields.io/badge/django-3.2%20%7C%204.x%20%7C%205.x%20%7C%206.x-green)](https://www.djangoproject.com/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

A production-grade, modular, drop-in caching layer for LLM prompts and completions in Django applications.

Stop burning thousands of dollars every month on duplicate LLM API tokens across **OpenAI**, **Gemini**, **Groq**, and **Anthropic**.

---

## 💡 The Problem: The Duplicate Token Burn

Modern AI-enabled web apps, SaaS platforms, and enterprise dashboards suffer from a hidden margin killer: **repetitive LLM prompts**.

* Users repeatedly ask the same questions in chatbots and customer support portals.
* Dashboard widgets and analytics summary cards refresh on every page reload.
* Automated workflow triggers re-process identical document fragments.
* Rate limits on fast inference engines (like Groq) are hit unnecessarily by repeated queries.

### 💰 Return on Investment (ROI) Breakdown

Here is what caching 35% of identical queries looks like for a startup serving **100,000 queries per day**:

| Metric | Without Cache | With `django-llm-cache` | Savings / Advantage |
| :--- | :--- | :--- | :--- |
| **API Calls to LLM Provider** | 100,000 / day | 65,000 / day | **35,000 fewer API calls daily** |
| **Average Response Latency** | ~1,200 ms | **3 ms** (Cached Hits) | **99.7% latency drop on hits** |
| **Monthly Token Cost (GPT-4o)** | ~\$15,000 / mo | ~\$9,750 / mo | **\$5,250 / month saved (\$63,000 / yr)** |
| **Rate Limit Headroom** | High risk of 429 errors | Unlocked 35%+ extra throughput | **Zero throttle downtime** |
| **Database/Cache Failures** | Typically crashes app | **Fails open** to live API | **Zero downtime guarantee** |

---

## 🏗️ Architecture & Request Lifecycle

```mermaid
sequenceDiagram
    autonumber
    actor Client as User / Frontend
    participant View as Django View / DRF
    participant Cache as django-llm-cache (LLMResponseCache)
    participant Redis as Django Cache Backend (Redis / Memcached)
    participant LLM as LLM Provider (Groq / OpenAI / Gemini)

    Client->>View: POST /api/chat/ {prompt: "Summarize Q3 report"}
    View->>Cache: cache.get(prompt, provider="groq", model="llama-3.3-70b-versatile")
    Cache->>Cache: Generate SHA-256 Key(provider + model + cleaned prompt)
    Cache->>Redis: GET key

    alt Cache Hit (Instant Return)
        Redis-->>Cache: Return cached response payload
        Cache-->>View: Cached response
        View-->>Client: 200 OK (Latency: ~3ms, Cost: $0.00)
    else Cache Miss / First Query
        Redis-->>Cache: None
        Cache-->>View: None
        View->>LLM: Live API call (Inference)
        LLM-->>View: Return LLM response
        View->>Cache: cache.set(prompt, provider, model, response, timeout=300)
        Cache->>Redis: SET key response (TTL 300s)
        View-->>Client: 200 OK (Latency: ~1200ms)
    else Cache Backend Offline / Network Error
        Redis--xCache: ConnectionError / Timeout
        Note over Cache: Caught exception, logged warning.<br/>Fails open without crashing!
        Cache-->>View: None
        View->>LLM: Fallback directly to live API
        LLM-->>View: Return response
        View-->>Client: 200 OK (Zero interruption to end user)
    end
```

---

## ✨ Key Features

* **Deterministic SHA-256 Hashing**: Strips prompt whitespace and creates canonical keys combining `provider`, `model_name`, prompt content, and optional deterministic hyperparameters (`temperature`, `system_prompt`).
* **Resilient Fail-Open Design**: If your Redis server crashes, your database runs out of connections, or network lag occurs, `django-llm-cache` catches the error, logs it via Python's standard `logging`, and cleanly returns `None`. **Your live user request never crashes.**
* **Lazy Django Imports**: Safe to import in CLI scripts, background workers, or standalone scripts without throwing `AppRegistryNotReady` or `ImproperlyConfigured`. Emits clean, non-fatal warnings when settings are unconfigured.
* **Pluggable Cache Backends**: Works with any Django cache backend (`django-redis`, Memcached, Local Memory, FileSystem, or Database cache). You can specify dedicated cache partitions using the `cache_alias` parameter.
* **Flexible Expiration**: Configurable default TTL (default: 300 seconds) with per-call overrides.

---

## 📦 Installation

```bash
pip install django-llm-cache
```

Ensure Django (>=3.2) is configured in your project settings:

```python
# settings.py
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
    }
}
```

---

## 🚀 Quickstart & Code Examples

### 1. Integration with Groq Python SDK

```python
from groq import Groq
from django_llm_cache import LLMResponseCache

# Initialize cache manager (defaults to 300s expiration)
llm_cache = LLMResponseCache(default_timeout=300)
client = Groq()

def ask_groq(prompt: str) -> str:
    provider = "groq"
    model = "llama-3.3-70b-versatile"

    # 1. Check local cache
    cached_content = llm_cache.get(prompt, provider=provider, model_name=model)
    if cached_content is not None:
        return cached_content

    # 2. Cache miss: Execute live API call
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": prompt}],
        model=model,
    )
    result_text = chat_completion.choices[0].message.content

    # 3. Store result in cache for future calls
    llm_cache.set(prompt, provider=provider, model_name=model, response=result_text)
    return result_text
```

---

### 2. Integration with OpenAI Python SDK

```python
from openai import OpenAI
from django_llm_cache import LLMResponseCache

llm_cache = LLMResponseCache(default_timeout=600)  # 10 minutes TTL
client = OpenAI()

def ask_openai(prompt: str) -> str:
    provider = "openai"
    model = "gpt-4o"

    # One-line lookup or fetch pattern
    def fetch_from_api():
        response = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
        )
        return response.choices[0].message.content

    # Automatically checks cache; calls fetch_from_api() only on cache miss!
    return llm_cache.get_or_set(
        prompt=prompt,
        provider=provider,
        model_name=model,
        default_callable=fetch_from_api,
    )
```

---

### 3. Integration with Google Gemini SDK

```python
from google import genai
from django_llm_cache import LLMResponseCache

llm_cache = LLMResponseCache(default_timeout=900)
client = genai.Client()

def ask_gemini(prompt: str) -> str:
    provider = "gemini"
    model = "gemini-2.5-flash"

    cached_text = llm_cache.get(prompt, provider, model)
    if cached_text:
        return cached_text

    response = client.models.generate_content(
        model=model,
        contents=prompt,
    )
    llm_cache.set(prompt, provider, model, response.text)
    return response.text
```

---

### 4. Integration in a Django REST Framework (DRF) View

```python
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from django_llm_cache import LLMResponseCache
from groq import Groq

cache = LLMResponseCache(default_timeout=600, key_prefix="customer_chat")
groq_client = Groq()

class AssistantView(APIView):
    def post(self, request):
        user_prompt = request.data.get("prompt", "").strip()
        if not user_prompt:
            return Response({"error": "Prompt is required"}, status=status.HTTP_400_BAD_REQUEST)

        provider = "groq"
        model = "llama-3.3-70b-versatile"

        # Check cache
        cached_response = cache.get(user_prompt, provider, model)
        if cached_response is not None:
            return Response({"answer": cached_response, "source": "cache"})

        # Call live LLM
        completion = groq_client.chat.completions.create(
            messages=[{"role": "user", "content": user_prompt}],
            model=model,
        )
        answer = completion.choices[0].message.content

        # Cache response
        cache.set(user_prompt, provider, model, answer)

        return Response({"answer": answer, "source": "live_api"})
```

---

## ⚙️ Advanced Configuration

### Dedicated Redis Cache Alias

If you want to isolate your LLM cache to a specific Redis database or Redis cluster without polluting your session or template cache:

```python
# settings.py
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.locmem.LocMemCache",
    },
    "llm_redis": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/2",
        "TIMEOUT": None,  # Handled per key by django-llm-cache
    },
}
```

Then pass the alias to `LLMResponseCache`:

```python
cache = LLMResponseCache(
    default_timeout=1800,          # 30 minutes
    cache_alias="llm_redis",       # Routes to 'llm_redis' cache in settings.py
    key_prefix="prod_v1",          # Custom key namespace
)
```

### Deterministic Hyperparameter Sensitivity

If responses vary based on sampling parameters like `temperature` or `system_prompt`, pass them as keyword arguments into `.get()`, `.set()`, or `.generate_key()`:

```python
# Keys generated will differ if temperature or system_prompt change
cached = cache.get(
    prompt="Explain photosynthesis",
    provider="openai",
    model_name="gpt-4o",
    temperature=0.2,
    system_prompt="Explain for a 5-year old",
)
```

### Manual Key Invalidation

```python
# Invalidate a specific prompt entry
cache.delete("Explain photosynthesis", provider="openai", model_name="gpt-4o")
```

---

## 🧪 Running Tests

The test suite covers deterministic key hashing, error handling, lazy configuration checks, and cache routing.

```bash
# Clone the repository
git clone https://github.com/Hafiz-alt/django-llm-cache.git
cd django-llm-cache

# Run tests
python -m unittest discover -s tests -v
```

---

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

## 👤 Author & Support

* **Author**: Hafiz Muhammed P H
* **Email**: [hafizmdph88172@gmail.com](mailto:hafizmdph88172@gmail.com)
* **GitHub**: [@Hafiz-alt](https://github.com/Hafiz-alt)
* **Bug Tracker**: [Issues](https://github.com/Hafiz-alt/django-llm-cache/issues)
