Metadata-Version: 2.4
Name: tei-loop
Version: 0.1.3
Summary: Target, Evaluate, Improve: A self-improving loop for agentic systems
Author-email: Orkhan Javadli <ojavadli@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/ojavadli/tei-loop
Project-URL: Repository, https://github.com/ojavadli/tei-loop
Keywords: agents,evaluation,improvement,llm,agentic-systems,self-improving
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0
Provides-Extra: openai
Requires-Dist: openai>=1.40; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.34; extra == "anthropic"
Provides-Extra: google
Requires-Dist: google-generativeai>=0.8; extra == "google"
Provides-Extra: all
Requires-Dist: openai>=1.40; extra == "all"
Requires-Dist: anthropic>=0.34; extra == "all"
Requires-Dist: google-generativeai>=0.8; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Dynamic: license-file

# TEI Loop

**Target, Evaluate, Improve** — a self-improving loop for agentic systems.

## Get Started (2 steps)

```bash
pip install tei-loop
```

```bash
tei your_agent.py
```

That's it. TEI auto-detects your agent function, generates a test query, evaluates across 4 dimensions, and applies targeted improvements.

> If `pip` is not found, use `pip3`. If `tei` is not found, use `python3 -m tei_loop.cli`.

## What Happens When You Run `tei your_agent.py`

1. TEI loads your file and finds the agent function (any callable that takes input and returns output)
2. Auto-generates a relevant test query based on your agent's purpose
3. Runs your agent and evaluates the output across 4 dimensions
4. If any dimension can be improved, applies a targeted fix and re-runs
5. Prints before/after scores

## Options

```bash
tei agent.py                          # Full loop with auto-generated query
tei agent.py --query "custom input"   # Use your own test query
tei agent.py --mode compare           # Side-by-side before/after
tei agent.py --mode evaluate          # Baseline only (no improvement)
tei agent.py --retries 5              # More improvement cycles
tei agent.py --verbose                # Detailed output
```

## Python API

```python
import asyncio
from tei_loop import TEILoop

def my_agent(query: str) -> str:
    # your agent logic
    return result

async def main():
    loop = TEILoop(agent=my_agent)
    result = await loop.run("your test query")
    print(result.summary())

asyncio.run(main())
```

## 4 Evaluation Dimensions

| Dimension | What it checks |
|---|---|
| **Target Alignment** | Did the agent pursue the correct objective? |
| **Reasoning Soundness** | Was the reasoning logical and non-contradictory? |
| **Execution Accuracy** | Were the right tools called with correct parameters? |
| **Output Integrity** | Is the output complete, accurate, and consistent? |

## Two Modes

**Runtime mode** (default): Per-query, 1-3 retries, fixes individual failures in seconds.

**Development mode**: Across many queries, proposes permanent prompt improvements.

```python
dev_results = await loop.develop(
    queries=["query1", "query2", "query3", ...],
    max_iterations=50,
)
```

## How It Works

TEI wraps any Python callable as a black box. No code changes to your agent needed.

When a dimension fails, TEI applies the right fix strategy:

| Failure | Fix Strategy |
|---|---|
| Target drift | Re-anchor to original objective |
| Flawed reasoning | Regenerate plan with failure context |
| Execution errors | Correct tool calls and parameters |
| Output issues | Repair factual errors and fill gaps |

## Configuration

TEI auto-detects your LLM provider from environment variables (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`). No new accounts or API keys needed.

```python
loop = TEILoop(
    agent=my_agent,
    eval_llm="gpt-5.2",       # Smartest for evaluation
    improve_llm="gpt-5-mini", # Cost-effective for fixes
)
```

## Cost

| Scenario | Cost |
|---|---|
| Agent passes all dimensions | ~$0.05 |
| One improvement cycle | ~$0.10 |
| Full 3-retry loop | ~$0.25 |

## Works With Any Agent

TEI wraps any Python callable. No framework lock-in:

- LangGraph agents
- CrewAI crews
- Custom Python functions
- FastAPI endpoints
- Any callable that takes input and returns output

## License

MIT
