Metadata-Version: 2.4
Name: space-evals-client-anthropic
Version: 0.2.0
Summary: Anthropic client plugin for space-evals
Author: Raghav Sahai
License: MIT
Project-URL: Homepage, https://github.com/Raghav-Sahai/Evals
Project-URL: Repository, https://github.com/Raghav-Sahai/Evals
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: space-evals<0.3.0,>=0.2.0
Requires-Dist: anthropic>=0.30
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"

# space-evals-client-anthropic

Anthropic client plugin for [space-evals](https://pypi.org/project/space-evals/). Lets you use Anthropic models (Claude Sonnet, Claude Opus, Claude Haiku, etc.) as the target, customer, or judge in your space-evals test suite.

## Installation

```bash
pip install space-evals-client-anthropic
```

This installs the core `space-evals` package and the `anthropic` SDK as dependencies.

## Setup

Set your API key:

```bash
export ANTHROPIC_API_KEY=sk-ant-...
```

Or use a custom env var name in your config (see below).

## Usage

### Via config file

Add `provider: anthropic` to any client role in your `space-evals.yaml`:

```yaml
clients:
  target:
    provider: anthropic
    model: claude-sonnet-4-20250514

  customer:
    provider: anthropic
    model: claude-haiku-4-5-20251001

  judge:
    provider: anthropic
    model: claude-sonnet-4-20250514
```

Then run your tests:

```bash
space-evals run ./tests/
```

### Via programmatic API

```python
import asyncio
from space_evals.engine import run_tests
from space_evals.clients.base import get_client

target = get_client("anthropic", model="claude-sonnet-4-20250514")

result = asyncio.run(run_tests(
    path="./tests/",
    target_client=target,
))
```

## Configuration

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `provider` | string | -- | Must be `"anthropic"` |
| `model` | string | -- | Model ID (e.g. `claude-sonnet-4-20250514`, `claude-opus-4-20250514`, `claude-haiku-4-5-20251001`) |
| `api_key_env` | string | `ANTHROPIC_API_KEY` | Env var containing your API key |
| `params` | object | `{}` | Extra parameters passed to the Anthropic API (e.g. `temperature`, `max_tokens`) |

### Example with extra params

```yaml
clients:
  target:
    provider: anthropic
    model: claude-sonnet-4-20250514
    api_key_env: MY_ANTHROPIC_KEY
    params:
      temperature: 0
      max_tokens: 4096
```

## What this plugin does

- Translates space-evals's `Message` format to Anthropic's messages API
- Extracts tool use blocks from responses into space-evals's `ToolCall` format
- Handles system prompts via Anthropic's `system` parameter (used by the customer and judge roles)
- Lazy-loads the Anthropic SDK on first use
- Defaults `max_tokens` to 1024 if not specified

## Requirements

- Python >= 3.11
- `space-evals` >= 0.1.0
- `anthropic` >= 0.30
- `ANTHROPIC_API_KEY` environment variable (or custom via `api_key_env`)

## License

MIT
