Metadata-Version: 2.5
Name: llmrivotril
Version: 0.1.3
Summary: A lightweight framework to reduce LLM hallucinations, enforce guardrails, manage stateful memory, and monitor performance via local dashboard.
Project-URL: Homepage, https://github.com/ailake-io/llmrivotril
Project-URL: Repository, https://github.com/ailake-io/llmrivotril
Project-URL: Issues, https://github.com/ailake-io/llmrivotril/issues
Project-URL: Documentation, https://github.com/ailake-io/llmrivotril/tree/main/docs
Project-URL: Changelog, https://github.com/ailake-io/llmrivotril/blob/main/CHANGELOG.md
Author-email: Thiago Egon Lange <contato@dadosidados.net.br>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,guardrails,hallucination,language-models,llm,rag
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: click>=8.0.0
Requires-Dist: fastapi>=0.100.0
Requires-Dist: instructor>=1.0.0
Requires-Dist: openai<3.0.0,>=2.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: tenacity>=8.0.0
Requires-Dist: tiktoken>=0.5.0
Requires-Dist: tomli>=1.2.0; python_version < '3.11'
Requires-Dist: uvicorn>=0.22.0
Provides-Extra: adk
Requires-Dist: google-adk>=1.0.0; extra == 'adk'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.25.0; extra == 'anthropic'
Provides-Extra: autogen
Requires-Dist: ag2<2.0.0,>=1.0.0; extra == 'autogen'
Provides-Extra: bedrock
Requires-Dist: boto3>=1.28.0; extra == 'bedrock'
Provides-Extra: ci
Requires-Dist: build>=1.0.0; extra == 'ci'
Requires-Dist: httpx>=0.24.0; extra == 'ci'
Requires-Dist: mypy>=1.5.0; extra == 'ci'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'ci'
Requires-Dist: pytest>=7.0.0; extra == 'ci'
Requires-Dist: ruff>=0.1.0; extra == 'ci'
Requires-Dist: twine>=5.0.0; extra == 'ci'
Provides-Extra: cohere
Requires-Dist: cohere>=5.0.0; extra == 'cohere'
Provides-Extra: crewai
Requires-Dist: crewai<2.0.0,>=1.0.0; extra == 'crewai'
Provides-Extra: dev
Requires-Dist: ag2<2.0.0,>=1.0.0; extra == 'dev'
Requires-Dist: anthropic>=0.25.0; extra == 'dev'
Requires-Dist: beautifulsoup4>=4.12.0; extra == 'dev'
Requires-Dist: boto3>=1.28.0; extra == 'dev'
Requires-Dist: build>=1.0.0; extra == 'dev'
Requires-Dist: cohere>=5.0.0; extra == 'dev'
Requires-Dist: crewai<2.0.0,>=1.0.0; extra == 'dev'
Requires-Dist: google-adk>=1.0.0; extra == 'dev'
Requires-Dist: google-genai>=1.0.0; extra == 'dev'
Requires-Dist: httpx>=0.24.0; extra == 'dev'
Requires-Dist: langchain-core>=0.3.0; extra == 'dev'
Requires-Dist: mypy>=1.5.0; extra == 'dev'
Requires-Dist: pinecone>=10.0.0; extra == 'dev'
Requires-Dist: psycopg[binary]>=3.1.0; extra == 'dev'
Requires-Dist: pypdf>=4.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: qdrant-client>=1.10.0; extra == 'dev'
Requires-Dist: redis>=5.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'dev'
Requires-Dist: twine>=5.0.0; extra == 'dev'
Requires-Dist: weaviate-client>=4.4.0; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: google-genai>=1.0.0; extra == 'gemini'
Provides-Extra: integrations
Requires-Dist: ag2<2.0.0,>=1.0.0; extra == 'integrations'
Requires-Dist: crewai<2.0.0,>=1.0.0; extra == 'integrations'
Requires-Dist: google-adk>=1.0.0; extra == 'integrations'
Requires-Dist: langchain-core>=0.3.0; extra == 'integrations'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3.0; extra == 'langchain'
Provides-Extra: pgvector
Requires-Dist: psycopg[binary]>=3.1.0; extra == 'pgvector'
Provides-Extra: pinecone
Requires-Dist: pinecone>=10.0.0; extra == 'pinecone'
Provides-Extra: providers
Requires-Dist: anthropic>=0.25.0; extra == 'providers'
Requires-Dist: boto3>=1.28.0; extra == 'providers'
Requires-Dist: cohere>=5.0.0; extra == 'providers'
Requires-Dist: google-genai>=1.0.0; extra == 'providers'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.10.0; extra == 'qdrant'
Provides-Extra: rag
Requires-Dist: beautifulsoup4>=4.12.0; extra == 'rag'
Requires-Dist: pypdf>=4.0.0; extra == 'rag'
Provides-Extra: redis
Requires-Dist: redis>=5.0.0; extra == 'redis'
Provides-Extra: semantic
Requires-Dist: sentence-transformers>=2.2.0; extra == 'semantic'
Provides-Extra: vector-stores
Requires-Dist: pinecone>=10.0.0; extra == 'vector-stores'
Requires-Dist: psycopg[binary]>=3.1.0; extra == 'vector-stores'
Requires-Dist: qdrant-client>=1.10.0; extra == 'vector-stores'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'vector-stores'
Requires-Dist: weaviate-client>=4.4.0; extra == 'vector-stores'
Provides-Extra: weaviate
Requires-Dist: weaviate-client>=4.4.0; extra == 'weaviate'
Description-Content-Type: text/markdown

# LLM-Rivotril

![LLM-Rivotril logo](https://github.com/ailake-io/llmrivotril/blob/main/docs/assets/logo.jpeg?raw=true)

*[Português](https://github.com/ailake-io/llmrivotril/blob/main/README.pt-BR.md)*

A lightweight Python framework to reduce LLM hallucinations, enforce guardrails, manage stateful memory, and monitor performance through a local web dashboard.

## Features

- **Guardrails** — Block disallowed keywords, enforce allowed topics, limit output size, and validate JSON schemas on outputs.
- **Semantic Guardrails** *(optional)* — Match prompts against allowed topics using dense embeddings instead of exact keywords.
- **Moderation Guardrail** — Flag adversarial/unsafe content via OpenAI's moderation endpoint.
- **Stateful Memory** — Sliding-window conversation store to prevent context drift, with optional automatic disk persistence.
- **Anti-Hallucination Verifier** — Pluggable grounding checks, including keyword overlap, citation markers, and embedding-based faithfulness.
- **Resilience** — Built-in rate limiting, retry with backoff, and circuit breaker for LLM calls.
- **Token-Saving Controls** *(optional)* — Semantic (similarity-based) response cache, token-budget-aware memory trimming, and LLM-summarized history compaction.
- **Telemetry & Dashboard** — Built-in FastAPI dashboard with live request logs, token usage, latency, and success rate; optional per-token auth.
- **Benchmark / Red-Team Evaluator** — Labeled suite to measure guardrail and verifier accuracy without API costs.
- **CLI** — Launch the dashboard with a single command.
- **Type-Safe Responses** — Optional Pydantic response models via `instructor`, including streamed structured output.
- **Async API** — `run_async()` for non-blocking execution.
- **Multi-Provider** — OpenAI-compatible servers (Ollama, vLLM, ...), plus native Anthropic, Cohere, Gemini, Azure OpenAI, and AWS Bedrock adapters.
- **Vector-Store Retrievers** *(optional)* — pgvector, Qdrant, Weaviate, and Pinecone adapters for RAG beyond in-memory scale.
- **Framework Integrations** *(optional)* — Drop-in adapters for CrewAI, AG2/AutoGen, LangChain/LangGraph, and Google ADK, so guardrails/PII/RAG/observability apply inside those frameworks too.

## Installation

```bash
pip install llmrivotril
```

For semantic (embedding-based) guardrails and verifiers:

```bash
pip install llmrivotril[semantic]
```

For local development:

```bash
git clone https://github.com/ailake-io/llmrivotril.git
cd llmrivotril
pip install -e ".[dev,semantic]"
```

This installs test, lint, type-check, and packaging tools (`pytest`, `ruff`, `mypy`, `build`, `twine`) plus every optional runtime dependency (all providers, RAG loaders, vector stores, Redis, framework integrations).

## Quick Start

```python
import os
from llmrivotril import RivotrilAgent, Guardrail
from llmrivotril.verifier import KeywordOverlapVerifier

agent = RivotrilAgent(
    model="gpt-4o-mini",
    api_key=os.getenv("OPENAI_API_KEY"),
    guardrails=[
        Guardrail(
            name="safe-content",
            allowed_topics=["AI safety", "machine learning"],
            disallowed_keywords=["password", "secret"],
            max_tokens=500,
        )
    ],
    verifier=KeywordOverlapVerifier(threshold=0.1),
)

response = agent.run("Explain what a guardrail is in AI safety.")
print(response)
```

## Documentation

- [Providers](https://github.com/ailake-io/llmrivotril/blob/main/docs/providers.md) — OpenAI-compatible servers, Anthropic, Cohere, Gemini, Azure OpenAI, AWS Bedrock.
- [Guardrails & Safety](https://github.com/ailake-io/llmrivotril/blob/main/docs/guardrails-and-safety.md) — Semantic guardrails, moderation, PII redaction, token budget.
- [RAG](https://github.com/ailake-io/llmrivotril/blob/main/docs/rag.md) — Local pipeline, vector-store retrievers (pgvector/Qdrant/Weaviate/Pinecone), grounding verification.
- [Framework Integrations](https://github.com/ailake-io/llmrivotril/blob/main/docs/integrations.md) — CrewAI, AG2/AutoGen, LangChain/LangGraph, Google ADK adapters, and multi-agent setup notes.
- [Async, Streaming & Function Calling](https://github.com/ailake-io/llmrivotril/blob/main/docs/streaming-and-tools.md)
- [Reliability](https://github.com/ailake-io/llmrivotril/blob/main/docs/reliability.md) — Resilience, response caching (including semantic cache), memory token budget & summarization, schema-repair fallback.
- [Observability](https://github.com/ailake-io/llmrivotril/blob/main/docs/observability.md) — Metrics persistence, local dashboard, benchmark, cost tracking.
- [Configuration](https://github.com/ailake-io/llmrivotril/blob/main/docs/configuration.md) — Config files, environment variables, plugins, project scaffolding.
- [Releasing](https://github.com/ailake-io/llmrivotril/blob/main/docs/releasing.md) — Build validation, TestPyPI, and the PyPI release workflow.

## Interactive Demo

Run a complete walkthrough with mock LLM responses (no API key, no cost):

```bash
python examples/demo.py --mock --dashboard
```

Then open http://127.0.0.1:8767 to watch the dashboard update live.

## Comparison: With vs. Without `llmrivotril`

See the same scenarios side-by-side:

```bash
python examples/comparison.py --mock
```

The comparison highlights how plain LLM calls return harmful or hallucinated
answers that are only caught manually afterwards, while `llmrivotril` blocks
them at runtime and records structured telemetry.

## Project Structure

```
llmrivotril/
├── src/llmrivotril/
│   ├── agent.py              # RivotrilAgent orchestrator (sync + async)
│   ├── config.py             # Environment-variable and file configuration loader
│   ├── guardrails.py         # Input/output guardrails
│   ├── moderation.py         # OpenAI moderation-endpoint guardrail
│   ├── memory.py             # Conversation memory store
│   ├── verifier.py           # Hallucination / grounding checks
│   ├── providers.py          # OpenAI / Azure / Anthropic / Cohere / Gemini / Bedrock adapters
│   ├── rag/                  # RAG loaders, chunkers, retrievers, vector stores, and pipeline
│   ├── integrations/         # CrewAI / AG2 / LangChain / Google ADK adapters
│   ├── resilience.py         # Rate limiter, retry, and circuit breaker
│   ├── semantic.py           # Optional embedding-based guardrails/verifiers
│   ├── metrics.py            # Telemetry collector
│   ├── server.py             # FastAPI dashboard
│   ├── cli.py                # Click CLI
│   ├── exceptions.py         # Custom exceptions
│   ├── templates/
│   │   └── dashboard.html    # Dashboard UI template
│   └── static/
│       └── tailwind.min.js   # Bundled Tailwind CSS for offline dashboard
├── tests/
├── examples/
└── docs/
```

## Development

Run lint, type checks, and tests:

```bash
pip install -e ".[dev,semantic]"
ruff check src tests examples scripts
ruff format --check src tests examples scripts
mypy src
pytest -v
```

Slow integration tests (e.g. loading `sentence-transformers` models) are skipped by
default. Run them with:

```bash
pytest -v --run-slow
```

For a reproducible provider/integration environment, apply the versions validated
locally before installing the desired extras:

```bash
python -m pip install -c .github/constraints-runtime.txt \
  -e ".[integrations,providers,semantic,qdrant]"
```

## CI/CD

![CI](https://github.com/ailake-io/llmrivotril/workflows/CI/badge.svg)

The GitHub Actions workflow runs linting, type checking, tests, and package builds on Python 3.10–3.13.

## License

MIT License — see [LICENSE](https://github.com/ailake-io/llmrivotril/blob/main/LICENSE).
