Metadata-Version: 2.4
Name: retune
Version: 0.1.0
Summary: Retune your AI agents — self-improving evaluation & optimization framework
Project-URL: Homepage, https://agentretune.com
Project-URL: Repository, https://github.com/agentretune/retune
Project-URL: Documentation, https://agentretune.com/docs
Project-URL: Changelog, https://github.com/agentretune/retune/blob/main/CHANGELOG.md
Author-email: Md Shariar Hossain <shmozumder2@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,ai,langchain,langgraph,optimization,rag,retune,self-improving
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: pydantic>=2.0
Provides-Extra: agents
Requires-Dist: langchain-core>=0.2; extra == 'agents'
Requires-Dist: langchain-openai>=0.1; extra == 'agents'
Requires-Dist: langgraph>=0.2; extra == 'agents'
Provides-Extra: all
Requires-Dist: fastapi>=0.110; extra == 'all'
Requires-Dist: langchain-anthropic>=0.1; extra == 'all'
Requires-Dist: langchain-core>=0.2; extra == 'all'
Requires-Dist: langchain-google-genai>=1.0; extra == 'all'
Requires-Dist: langchain-openai>=0.1; extra == 'all'
Requires-Dist: langgraph>=0.2; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.29; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: langchain-anthropic>=0.1; extra == 'anthropic'
Requires-Dist: langchain-core>=0.2; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: google
Requires-Dist: langchain-core>=0.2; extra == 'google'
Requires-Dist: langchain-google-genai>=1.0; extra == 'google'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.2; extra == 'langchain'
Provides-Extra: langgraph
Requires-Dist: langchain-core>=0.2; extra == 'langgraph'
Requires-Dist: langgraph>=0.2; extra == 'langgraph'
Provides-Extra: llm
Requires-Dist: langchain-core>=0.2; extra == 'llm'
Requires-Dist: langchain-openai>=0.1; extra == 'llm'
Provides-Extra: ollama
Requires-Dist: langchain-core>=0.2; extra == 'ollama'
Requires-Dist: langchain-ollama>=0.1; extra == 'ollama'
Provides-Extra: server
Requires-Dist: fastapi>=0.110; extra == 'server'
Requires-Dist: uvicorn[standard]>=0.29; extra == 'server'
Description-Content-Type: text/markdown

# retune

**Retune your AI agents** -- self-improving evaluation & optimization framework.

> Make any LLM agent, RAG pipeline, or workflow self-improving in production.

[![PyPI version](https://img.shields.io/pypi/v/retune)](https://pypi.org/project/retune/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://python.org)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![Tests](https://github.com/agentretune/retune/actions/workflows/test.yml/badge.svg)](https://github.com/agentretune/retune/actions/workflows/test.yml)

[Website](https://agentretune.com) | [Documentation](https://agentretune.com/docs) | [GitHub](https://github.com/agentretune/retune)

---

## What is retune?

`retune` is a framework-agnostic SDK that wraps any LLM-based system and makes it self-improving through automated observation, evaluation, and optimization. It works with LangChain, LangGraph, or any custom pipeline.

## Installation

```bash
pip install retune
```

| Extra | Command | What you get |
|-------|---------|--------------|
| LangChain | `pip install retune[langchain]` | LangChain adapter |
| LangGraph | `pip install retune[langgraph]` | LangGraph adapter |
| LLM Judge | `pip install retune[llm]` | OpenAI-powered evaluator |
| Anthropic | `pip install retune[anthropic]` | Claude models |
| Google | `pip install retune[google]` | Gemini models |
| Ollama | `pip install retune[ollama]` | Local models |
| Server | `pip install retune[server]` | FastAPI dashboard |
| Everything | `pip install retune[all]` | All of the above |
| Dev | `pip install retune[dev]` | pytest, ruff, mypy |

## Quickstart

```python
from retune import Retuner, Mode

# Your existing agent (any callable)
def my_agent(query: str) -> str:
    return call_llm(query)

# Wrap it
retuner = Retuner(
    agent=my_agent,
    adapter="custom",
    mode=Mode.OBSERVE,
)

# Use it -- same interface, now with traces
response = retuner.run("What is machine learning?")
print(response.output)
```

## Framework Adapters

### Custom (any callable)

```python
from retune import Retuner, Mode

retuner = Retuner(agent=my_fn, adapter="custom", mode=Mode.OBSERVE)
response = retuner.run("Hello")
```

### LangChain

```python
from retune import Retuner, Mode

chain = prompt | llm | parser  # your LangChain chain

retuner = Retuner(
    agent=chain,
    adapter="langchain",
    mode=Mode.EVALUATE,
    evaluators=["llm_judge", "retrieval", "latency"],
)

response = retuner.run("What is RAG?")
print(response.eval_results)  # quality scores
```

### LangGraph

```python
from retune import Retuner, Mode

compiled_graph = graph.compile()

retuner = Retuner(
    agent=compiled_graph,
    adapter="langgraph",
    mode=Mode.OBSERVE,
)

response = retuner.run("Plan a trip to Paris")
for step in response.trace.steps:
    print(f"[{step.step_type}] {step.name}")
```

## The Fan Regulator Model

Control how much post-processing happens after each run:

| Mode | What it does | Overhead |
|------|-------------|----------|
| `OFF` | Pure passthrough | None |
| `OBSERVE` | Capture execution traces | Low |
| `EVALUATE` | + Run evaluators, score quality | Medium |
| `IMPROVE` | + Generate optimization suggestions | High |

```python
retuner.set_mode(Mode.IMPROVE)  # Turn up when tuning
# ... run evaluation dataset ...
retuner.set_mode(Mode.OFF)      # Turn off for production
```

## Accept/Reject Optimization Flow

When in `IMPROVE` mode, retune generates suggestions that you can review:

```python
response = retuner.run("test query")
for suggestion in response.suggestions:
    print(f"{suggestion.param_name}: {suggestion.old_value} -> {suggestion.new_value}")
    # Accept or reject each suggestion
    retuner.accept_suggestion(suggestion)
    # or: retuner.reject_suggestion(suggestion)
```

## Beam Search APO (Automatic Prompt Optimization)

retune includes a beam-search-based prompt optimizer that iteratively rewrites your system prompt:

```python
from retune import Retuner, BeamSearchConfig

config = BeamSearchConfig(
    beam_width=2,         # candidates kept per round
    branch_factor=2,      # rewrites per candidate
    rounds=3,             # search depth
    max_rollout_queries=5,
)

retuner = Retuner(agent=my_agent, adapter="custom", mode=Mode.IMPROVE)
best_prompt = retuner.optimize_prompt(
    initial_prompt="You are a helpful assistant.",
    eval_dataset=dataset,
    beam_config=config,
)
```

## Multi-Provider LLM Support

retune works with any LLM provider through LangChain integrations:

```python
from retune import set_default_llm

# OpenAI (default)
set_default_llm("openai", model="gpt-4o")

# Anthropic
set_default_llm("anthropic", model="claude-sonnet-4-20250514")

# Google
set_default_llm("google", model="gemini-pro")

# Local via Ollama
set_default_llm("ollama", model="llama3")
```

## Evaluators

| Evaluator | What it scores | Install |
|-----------|---------------|---------|
| `llm_judge` | Correctness, completeness, relevance | `retune[llm]` |
| `retrieval` | Document retrieval quality | Built-in |
| `latency` | Execution speed | Built-in |
| `cost` | Token usage efficiency | Built-in |

## Self-Improvement Loop

```python
# 1. Collect traces
retuner.set_mode(Mode.OBSERVE)
for query in queries:
    retuner.run(query)

# 2. Evaluate
retuner.set_mode(Mode.EVALUATE)
summary = retuner.get_eval_summary()

# 3. Get improvement suggestions
retuner.set_mode(Mode.IMPROVE)
response = retuner.run("test query")
for s in response.suggestions:
    print(f"{s.param_name}: {s.old_value} -> {s.new_value}")

# 4. Apply and run in production
retuner.set_mode(Mode.OFF)
```

## Architecture

```
Your Agent/RAG (LangGraph, LangChain, custom)
        |
   Adapter Layer  -- framework-specific -> universal trace
        |
   Execution Trace  -- standard format for all frameworks
        |
   Evaluation Engine  -- modular scorers (LLM judge, retrieval, latency, cost)
        |
   Optimization Engine  -- beam search APO, rule-based suggestions
        |
   Accept/Reject  -- human-in-the-loop or auto-apply
        |
   Improved Config  -- better prompts, parameters, retrieval
```

## Environment Variables

```bash
# LLM provider keys
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

# retune config (all optional, sensible defaults)
RETUNE_STORAGE_PATH=./retune.db
RETUNE_DEFAULT_MODE=observe
RETUNE_LOG_LEVEL=INFO
RETUNE_EVAL_LLM_MODEL=gpt-4o-mini
```

## License

MIT -- see [LICENSE](LICENSE).
