Metadata-Version: 2.5
Name: evalwatch
Version: 0.1.1
Summary: Python client for sending LLM traces to EvalWatch
Project-URL: Homepage, https://github.com/tanishqsrivastavaa/EvalWatch
Project-URL: Repository, https://github.com/tanishqsrivastavaa/EvalWatch
Project-URL: Issues, https://github.com/tanishqsrivastavaa/EvalWatch/issues
License: MIT
Keywords: evals,evaluation,llm,observability,tracing
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: requests>=2.28
Provides-Extra: async
Requires-Dist: httpx>=0.28.1; extra == 'async'
Description-Content-Type: text/markdown

# evalwatch

Python client for sending LLM traces to [EvalWatch](https://github.com/tanishqsrivastavaa/EvalWatch) — a SaaS observability and evaluation platform for LLM products.

## Install

```bash
# Sync client only (uses requests)
pip install evalwatch

# Sync + async client (adds httpx)
pip install 'evalwatch[async]'
```

## Quick start — sync

Get an API key from your EvalWatch dashboard (`/settings`), then instrument your LLM calls:

```python
from evalwatch import EvalWatchClient

client = EvalWatchClient("https://your-evalwatch-instance.com", api_key="sk-...")

# 1. Manual
client.send_trace(
    prompt="What is the capital of France?",
    response="Paris.",
    latency_ms=120,
    model_name="gpt-4o",
    provider="openai",
    token_usage=30,
)

# 2. Context manager — measures latency automatically
with client.capture(prompt="...", model_name="gpt-4o", provider="openai") as trace:
    response = openai_client.chat.completions.create(...)
    trace.finish(response=response.choices[0].message.content, token_usage=response.usage.total_tokens)

# 3. Decorator — wraps any function whose first arg is the prompt
@client.watch(model_name="gpt-4o", provider="openai")
def call_llm(prompt: str) -> str:
    return openai_client.complete(prompt)

# Retry on transient failures (ConnectionError or 5xx)
client.send_trace(..., max_retries=2)
```

## Quick start — async

```python
from evalwatch import AsyncEvalWatchClient

client = AsyncEvalWatchClient("https://your-evalwatch-instance.com", api_key="sk-...")

# 1. Manual (awaitable)
await client.send_trace(
    prompt="What is the capital of France?",
    response="Paris.",
    latency_ms=120,
    model_name="gpt-4o",
    provider="openai",
    token_usage=30,
)

# 2. Async context manager
async with client.capture(prompt="...", model_name="gpt-4o") as trace:
    response = await openai_client.chat.completions.acreate(...)
    await trace.finish(response=response.choices[0].message.content)

# 3. Async decorator
@client.watch(model_name="gpt-4o", provider="openai")
async def call_llm(prompt: str) -> str:
    return await openai_client.acomplete(prompt)
```

All methods send traces to `POST /api/v1/observe/traces` and are visible in the Traces dashboard immediately.

## License

MIT
