Metadata-Version: 2.4
Name: valohai-llm
Version: 0.1.0
Summary: Track LLM evaluations to Valohai LLM
Project-URL: Homepage, https://llm.valohai.com
Project-URL: Repository, https://github.com/valohai/valohai-llm
Author-email: Valohai <info@valohai.com>
License-Expression: MIT
License-File: LICENSE
Keywords: ai,evaluation,llm,machine-learning,valohai
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.9
Requires-Dist: httpx>=0.25
Description-Content-Type: text/markdown

# valohai-llm

[vh]: https://valohai.com/
[vh-llm]: https://llm.valohai.com/

Python instrumentation library for [Valohai][vh] large language model and generative AI tools.

## Installation

```shell
pip install valohai-llm

# `uv` project with pip-compatibility
uv pip install valohai-llm

# `uv`-managed project
uv add valohai-llm
```

## Usage

1. Get `VALOHAI_LLM_API_KEY` from [Valohai LLM][vh-llm]
2. Make sure `VALOHAI_LLM_API_KEY` is set in your environment
3. Use the library e.g. the simplest case:

```python
import valohai_llm

def main():
    result = valohai_llm.post_result(
        task="my-evaluation",
        labels={"model": "gpt-4", "dataset": "mmlu", "category": "math"},
        metrics={"accuracy": 0.85, "latency_ms": 150},
    )
    print("Result posted:", result)

if __name__ == "__main__":
    main()
```

## Development

```shell
# auto-format and lint code
just format

# run unit tests
uv run pytest

# configure setup to run examples
cp .env.dev-example .env
# edit the .env
uv run --env-file=.env --with-editable=. examples/simple_example.py
```
