Metadata-Version: 2.4
Name: atlas-eval
Version: 0.2.0
Summary: Evaluate your AI agent or LLM/SLM on the Bharat AI Index / Atlas Agent Arena.
Author: Atlas AI Labs
License: MIT
Project-URL: Homepage, https://github.com/info-gallary/BharatAIndex
Project-URL: Repository, https://github.com/info-gallary/BharatAIndex
Project-URL: Documentation, https://github.com/info-gallary/BharatAIndex/tree/main/sdk
Project-URL: Bug Tracker, https://github.com/info-gallary/BharatAIndex/issues
Keywords: llm,slm,agent,agentic,evaluation,eval,benchmark,india,indic,sovereign-ai,agno,langchain,langgraph
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# atlas-eval (Python)

Evaluate your AI agent / LLM on the Bharat AI Index — Atlas Agent Arena.

> Requires a running Atlas instance (default `http://localhost:3000`). Start one with
> `npm run dev` in the repo, or `docker run -p 3000:3000 ... bharat-ai-index`.

## Install

```bash
pip install atlas-eval           # live on PyPI
```

## Use

```python
from atlas_eval import evaluate_agent, AccuracyEval, PerformanceEval, ReliabilityEval

# whole-agent benchmark (sandboxed tasks, agentic score)
def my_agent(ctx):
    return {"tool": "fs_read", "args": {"path": "invoice_a.txt"}}
report = evaluate_agent(my_agent, base_url="http://localhost:3000")

# code-first evals (Agno-style)
AccuracyEval(agent=lambda q: "New Delhi", input="Capital of India?",
             expected_output="New Delhi").run(print_results=True)
PerformanceEval(func=lambda: my_model("ping"), num_iterations=5).run(print_results=True)
ReliabilityEval(tool_calls=["search"], expected_tool_calls=["search"]).run(print_results=True)
```

`PerformanceEval` and `ReliabilityEval` run fully locally (no server). `evaluate_agent`
and `AccuracyEval` call the Atlas instance (and a configured judge model for accuracy).

## CLI

```bash
atlas-eval --version
atlas-eval check --base-url http://localhost:3000   # verify a reachable Atlas instance
```

## What you get back

`evaluate_agent` returns the full agentic run: `agenticScore`, `successRate`, `efficiency`,
`toolValidity`, `safetyPass`, and per-task `results` with trajectories. The eval classes return
typed results (`AccuracyResult`, `PerformanceResult`, `ReliabilityResult`).

- Zero runtime dependencies (Python stdlib only). Ships type hints (PEP 561 `py.typed`).
- Docs & JS equivalents: https://github.com/info-gallary/BharatAIndex/tree/main/sdk

MIT © Atlas AI Labs
