Metadata-Version: 2.4
Name: atlas-eval
Version: 0.1.0
Summary: Evaluate your AI agent or LLM/SLM on the Bharat AI Index / Atlas Agent Arena.
Author: Atlas AI Labs
License: MIT
Project-URL: Homepage, https://github.com/atlas-ai-labs/bharat-ai-index
Project-URL: Documentation, https://github.com/atlas-ai-labs/bharat-ai-index/tree/main/sdk
Keywords: llm,agent,evaluation,benchmark,india,sovereign-ai,agno,langchain
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Intended Audience :: Developers
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# atlas-eval (Python)

Evaluate your AI agent / LLM on the Bharat AI Index — Atlas Agent Arena.

> Requires a running Atlas instance (default `http://localhost:3000`). Start one with
> `npm run dev` in the repo, or `docker run -p 3000:3000 ... bharat-ai-index`.

## Install

```bash
# until published to PyPI:
pip install -e sdk/python        # from the repo root
# or copy sdk/python/atlas_eval.py into your project
```

## Use

```python
from atlas_eval import evaluate_agent, AccuracyEval, PerformanceEval, ReliabilityEval

# whole-agent benchmark (sandboxed tasks, agentic score)
def my_agent(ctx):
    return {"tool": "fs_read", "args": {"path": "invoice_a.txt"}}
report = evaluate_agent(my_agent, base_url="http://localhost:3000")

# code-first evals (Agno-style)
AccuracyEval(agent=lambda q: "New Delhi", input="Capital of India?",
             expected_output="New Delhi").run(print_results=True)
PerformanceEval(func=lambda: my_model("ping"), num_iterations=5).run(print_results=True)
ReliabilityEval(tool_calls=["search"], expected_tool_calls=["search"]).run(print_results=True)
```

`PerformanceEval` and `ReliabilityEval` run fully locally (no server). `evaluate_agent`
and `AccuracyEval` call the Atlas instance (and a configured judge model for accuracy).

See the top-level [`sdk/README.md`](../README.md) for the protocol and JS equivalents.
