Metadata-Version: 2.4
Name: llmguard-py
Version: 0.1.1
Summary: A lightweight Python decorator for caching, rate limiting, and retrying LLM API calls.
Author-email: Aadhya <aadhyagarg07@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/aadhya-code/llmguard
Project-URL: Repository, https://github.com/aadhya-code/llmguard
Project-URL: Issues, https://github.com/aadhya-code/llmguard/issues
Keywords: llm,api,cache,retry,rate-limiter,gemini,openai
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# llmguard — A Lightweight Python Decorator for Reliable LLM API Calls

A lightweight Python library that wraps any LLM API call (or any external, flaky, rate-limited API) with **response caching**, **token-bucket rate limiting**, **automatic retries with exponential backoff**, and **usage statistics** using a single decorator.

Instead of repeatedly solving these problems in every project, `llmguard` provides a reusable, production-friendly solution with a clean and simple API.

---

## Why I Built This

While developing my **AI Resume Matcher** project using the Gemini API, I repeatedly encountered the same issues during development:

* Running the same prompt multiple times consumed API credits unnecessarily.
* Temporary API failures interrupted requests.
* Rate limits caused delays and failed requests.
* There was no simple way to measure how much caching was reducing API usage.

Rather than solving these problems inside one application, I extracted them into a reusable Python package that can be integrated into any LLM-powered project.

---

## Features

* 🚀 Simple `@llmguard` decorator
* ⚡ In-memory TTL response caching
* 🔁 Automatic retries with exponential backoff and jitter
* 🪣 Token-bucket rate limiting
* 📊 Built-in usage statistics
* 💰 Estimated API cost savings
* 🖥️ Command-line interface for viewing statistics
* 🧪 Comprehensive unit tests
* 📦 Packaged as a reusable Python library

---

## Installation

### Install from source

```bash
git clone https://github.com/aadhya-code/llmguard.git
cd llmguard
pip install -e .
```

### Build the package

```bash
python -m build
```

---

## Quick Start

```python
from llmguard import llmguard

@llmguard(
    cache_ttl=3600,
    max_calls_per_period=10,
    period_seconds=60,
    max_retries=3,
    cost_per_call=0.002
)
def ask_llm(prompt: str) -> str:
    return call_some_llm_api(prompt)

# First call → API request
response = ask_llm("Explain Retrieval-Augmented Generation")

# Second identical call → Served from cache
response = ask_llm("Explain Retrieval-Augmented Generation")
```

---

## Example

```python
from llmguard import llmguard

@llmguard(
    cache_ttl=1800,
    max_calls_per_period=30,
    period_seconds=60,
    max_retries=5,
)
def generate(prompt):
    return llm.generate(prompt)

print(generate("Summarise this research paper"))
```

---

## CLI Usage

View usage statistics:

```bash
llmguard-stats
```

Example output:

```text
llmguard usage stats
================================
Total wrapped calls:     47
Cache hits:              19
Cache misses:            28
Cache hit rate:          40.4%
Total retries used:      3
Rate-limit waits:        5
Estimated $ saved:       $0.0380
```

Reset statistics:

```bash
llmguard-stats --reset
```

---

## How It Works

### Response Caching

* Generates a SHA-256 hash using the function name and arguments.
* Returns cached responses for identical requests.
* Supports configurable Time-To-Live (TTL) expiration.

### Token-Bucket Rate Limiting

* Smoothly limits request rate.
* Prevents bursts that commonly occur with fixed-window rate limiters.
* Blocks only when necessary.

### Automatic Retry

* Retries failed requests using exponential backoff.
* Adds random jitter to prevent retry storms.
* Raises a descriptive exception if all retry attempts fail.

### Usage Statistics

Records:

* Total wrapped calls
* Cache hits
* Cache misses
* Retry count
* Rate-limit waits
* Estimated API cost savings

Statistics are persisted locally so they remain available across different program executions.

---

## Project Structure

```text
llmguard/
│
├── llmguard/
│   ├── __init__.py
│   ├── cache.py
│   ├── retry.py
│   ├── rate_limiter.py
│   ├── guard.py
│   ├── stats.py
│   └── cli.py
│
├── tests/
│   ├── test_cache.py
│   ├── test_retry.py
│   ├── test_rate_limiter.py
│   └── test_guard.py
│
├── pyproject.toml
├── README.md
└── LICENSE
```

---

## Architecture

```text
                    User Function
                          │
                          ▼
                   @llmguard(...)
                          │
          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
      TTL Cache     Retry Handler    Token Bucket
          │               │               │
          └───────────────┼───────────────┘
                          ▼
                     External LLM API
```

---

## Running Tests

Run all tests:

```bash
pytest -v
```

Generate a coverage report:

```bash
pytest --cov=llmguard
```

---

## Version

Current Version:

```
v0.1.0
```

---

## Future Improvements

* Async (`async/await`) support
* Persistent cache backend (SQLite/Redis)
* Additional provider examples (Gemini, OpenAI, Anthropic)
* GitHub Actions CI/CD
* Benchmark suite
* Configurable logging

---

## License

This project is licensed under the MIT License.

---

## Contributing

Contributions, suggestions, and bug reports are welcome.

If you find an issue or have an improvement, feel free to open an issue or submit a pull request.
