Metadata-Version: 2.4
Name: space-evals-client-openai
Version: 0.2.0
Summary: OpenAI client plugin for space-evals
Author: Raghav Sahai
License: MIT
Project-URL: Homepage, https://github.com/Raghav-Sahai/Evals
Project-URL: Repository, https://github.com/Raghav-Sahai/Evals
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: space-evals<0.3.0,>=0.2.0
Requires-Dist: openai>=1.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"

# space-evals-client-openai

OpenAI client plugin for [space-evals](https://pypi.org/project/space-evals/). Lets you use OpenAI models (GPT-4o, GPT-4, etc.) as the target, customer, or judge in your space-evals test suite.

## Installation

```bash
pip install space-evals-client-openai
```

This installs the core `space-evals` package and the `openai` SDK as dependencies.

## Setup

Set your API key:

```bash
export OPENAI_API_KEY=sk-...
```

Or use a custom env var name in your config (see below).

## Usage

### Via config file

Add `provider: openai` to any client role in your `space-evals.yaml`:

```yaml
clients:
  target:
    provider: openai
    model: gpt-4o

  customer:
    provider: openai
    model: gpt-4o-mini

  judge:
    provider: openai
    model: gpt-4o
```

Then run your tests:

```bash
space-evals run ./tests/
```

### Via programmatic API

```python
import asyncio
from space_evals.engine import run_tests
from space_evals.clients.base import get_client

target = get_client("openai", model="gpt-4o")

result = asyncio.run(run_tests(
    path="./tests/",
    target_client=target,
))
```

## Configuration

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `provider` | string | -- | Must be `"openai"` |
| `model` | string | -- | Model ID (e.g. `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`) |
| `api_key_env` | string | `OPENAI_API_KEY` | Env var containing your API key |
| `params` | object | `{}` | Extra parameters passed to the OpenAI API (e.g. `temperature`, `max_tokens`) |

### Example with extra params

```yaml
clients:
  target:
    provider: openai
    model: gpt-4o
    api_key_env: MY_OPENAI_KEY
    params:
      temperature: 0
      max_tokens: 2048
```

## What this plugin does

- Translates space-evals's `Message` format to OpenAI's chat completions API
- Extracts tool calls from responses into space-evals's `ToolCall` format
- Handles system prompts (used by the customer and judge roles)
- Lazy-loads the OpenAI SDK on first use

## Requirements

- Python >= 3.11
- `space-evals` >= 0.1.0
- `openai` >= 1.0
- `OPENAI_API_KEY` environment variable (or custom via `api_key_env`)

## License

MIT
