Metadata-Version: 2.4
Name: openrunner-sdk
Version: 1.5.0
Summary: OpenRunner SDK - W&B-compatible ML experiment tracking client
Project-URL: Homepage, https://github.com/jqueguiner/openrunner
Project-URL: Repository, https://github.com/jqueguiner/openrunner
Project-URL: Issues, https://github.com/jqueguiner/openrunner/issues
Author-email: JL Queguiner <jl@gladia.io>
License: MIT
Keywords: experiment-tracking,machine-learning,ml,mlops,wandb
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: click>=8.1
Requires-Dist: httpx>=0.27
Requires-Dist: nvidia-ml-py>=12.0
Requires-Dist: pillow>=10.0
Requires-Dist: psutil>=6.0
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.20; extra == 'anthropic'
Provides-Extra: catboost
Requires-Dist: catboost>=1.2; extra == 'catboost'
Provides-Extra: dev
Requires-Dist: numpy>=1.24; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: fastai
Requires-Dist: fastai>=2.7; extra == 'fastai'
Provides-Extra: forced-alignment
Requires-Dist: matplotlib>=3.7; extra == 'forced-alignment'
Requires-Dist: numpy>=1.24; extra == 'forced-alignment'
Provides-Extra: gpu
Requires-Dist: nvidia-ml-py>=12.0; extra == 'gpu'
Provides-Extra: huggingface
Requires-Dist: transformers>=4.30; extra == 'huggingface'
Provides-Extra: hydra
Requires-Dist: hydra-core>=1.3; extra == 'hydra'
Requires-Dist: omegaconf>=2.3; extra == 'hydra'
Provides-Extra: jax
Requires-Dist: flax>=0.7; extra == 'jax'
Requires-Dist: jax>=0.4; extra == 'jax'
Provides-Extra: keras
Requires-Dist: keras>=3.0; extra == 'keras'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.2; extra == 'langchain'
Provides-Extra: lightgbm
Requires-Dist: lightgbm>=4.0; extra == 'lightgbm'
Provides-Extra: lightning
Requires-Dist: lightning>=2.0; extra == 'lightning'
Provides-Extra: llamaindex
Requires-Dist: llama-index-core>=0.10; extra == 'llamaindex'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Provides-Extra: optuna
Requires-Dist: optuna>=3.0; extra == 'optuna'
Provides-Extra: pytorch
Requires-Dist: torch>=2.0; extra == 'pytorch'
Provides-Extra: sb3
Requires-Dist: stable-baselines3>=2.0; extra == 'sb3'
Provides-Extra: sklearn
Requires-Dist: scikit-learn>=1.3; extra == 'sklearn'
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.12; extra == 'tensorflow'
Provides-Extra: trl
Requires-Dist: transformers>=4.30; extra == 'trl'
Requires-Dist: trl>=0.7; extra == 'trl'
Provides-Extra: tts
Requires-Dist: matplotlib>=3.7; extra == 'tts'
Requires-Dist: numpy>=1.24; extra == 'tts'
Provides-Extra: ultralytics
Requires-Dist: ultralytics>=8.0; extra == 'ultralytics'
Provides-Extra: voice-agent
Provides-Extra: whisper
Requires-Dist: openai-whisper>=20231117; extra == 'whisper'
Provides-Extra: xgboost
Requires-Dist: xgboost>=2.0; extra == 'xgboost'
Description-Content-Type: text/markdown

# OpenRunner SDK

[![PyPI](https://img.shields.io/pypi/v/openrunner-sdk)](https://pypi.org/project/openrunner-sdk/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

Open-source, self-hosted ML experiment tracking — a drop-in replacement for [Weights & Biases](https://wandb.ai).

## Install

```bash
pip install openrunner-sdk
```

## Setup

```bash
export OPENRUNNER_API_KEY="or_your_key"
export OPENRUNNER_BASE_URL="https://your-server.com"
```

Or use the CLI:

```bash
openrunner login
```

## Quick Start

```python
import openrunner

# Start a run
openrunner.init(project="my-project", config={"lr": 0.001, "epochs": 10})

# Log metrics in your training loop
for epoch in range(10):
    loss = train(epoch)
    acc = evaluate()
    openrunner.log({"loss": loss, "accuracy": acc, "epoch": epoch})

# End the run
openrunner.finish()
```

## API Reference

### Core Functions

#### `openrunner.init()`

Initialize a new experiment run.

```python
run = openrunner.init(
    project="my-project",          # Project name (auto-created if missing)
    name="experiment-1",           # Optional display name
    config={"lr": 0.001},          # Hyperparameters
    tags=["baseline", "v2"],       # Optional tags
    notes="Testing new arch",      # Optional notes
    group="sweep-1",               # Optional group name
    job_type="train",              # Optional job type
    resume=True,                   # Resume a previous run by ID
)
```

#### `openrunner.log()`

Log metrics. Non-blocking — never slows down training.

```python
# Basic logging
openrunner.log({"loss": 0.5, "accuracy": 0.85})

# With explicit step
openrunner.log({"loss": 0.3}, step=100)

# Log images
openrunner.log({"predictions": openrunner.Image(img_array, caption="epoch 5")})

# Log tables
table = openrunner.Table(
    columns=["input", "predicted", "actual"],
    data=[["img_01", 7, 7], ["img_02", 3, 5]],
)
openrunner.log({"eval_results": table})
```

#### `openrunner.finish()`

End the current run. Flushes all buffered metrics.

```python
openrunner.finish()
openrunner.finish(exit_code=0)    # With exit code
openrunner.finish(quiet=True)     # Suppress output
```

### Config

Dict-like object with dot notation. Set at `init()`, accessible throughout the run.

```python
openrunner.init(config={"optimizer": {"lr": 0.001, "weight_decay": 1e-5}})

# Access
print(openrunner.config["optimizer.lr"])    # 0.001 (flattened keys)
print(openrunner.config.optimizer.lr)       # 0.001 (dot notation)

# Update after init
openrunner.config.update({"batch_size": 64})
openrunner.config["new_param"] = "value"
```

### Summary

Auto-updated with the last logged value for each key. Can also be set explicitly.

```python
# Auto-populated from log()
openrunner.log({"loss": 0.5})
openrunner.log({"loss": 0.3})
print(openrunner.summary["loss"])  # 0.3 (last value)

# Explicit set
openrunner.summary["best_accuracy"] = 0.95
openrunner.summary["final_loss"] = 0.1
```

### Artifacts

Version datasets, models, and checkpoints with content-hash deduplication.

```python
# Log a model artifact
artifact = openrunner.Artifact(name="my-model", type="model")
artifact.add_file("model.pth")
artifact.add_file("config.json")
run.log_artifact(artifact)

# Use an artifact from a previous run
artifact = run.use_artifact("my-model:v2")
artifact.download("/path/to/dir")
```

### Media Types

#### Images

```python
import numpy as np

# From numpy array
img = openrunner.Image(np.random.rand(28, 28, 3), caption="sample")

# From PIL Image
from PIL import Image as PILImage
pil_img = PILImage.open("photo.png")
img = openrunner.Image(pil_img, caption="photo")

# From file path
img = openrunner.Image("output.png", caption="result")

openrunner.log({"examples": img})
```

#### Tables

```python
table = openrunner.Table(
    columns=["epoch", "loss", "accuracy"],
    data=[
        [1, 0.9, 0.65],
        [2, 0.5, 0.82],
        [3, 0.3, 0.91],
    ],
)
openrunner.log({"metrics_table": table})
```

### Run Properties

```python
run = openrunner.init(project="test")

print(run.id)          # "a1b2c3d4" (8-char ID)
print(run.name)        # Display name
print(run.project)     # Project name
print(run.config)      # Config object
print(run.summary)     # Summary object
```

### HTML

```python
# Log raw HTML for rich reports, custom visualizations, or formatted output
openrunner.log({"report": openrunner.Html("<h1>Training Report</h1><p>Loss converged at epoch 42.</p>")})
```

### Histograms

```python
import numpy as np

weights = np.random.randn(10000)
openrunner.log({"weight_dist": openrunner.Histogram(weights, num_bins=50)})
```

### Plotly Charts

```python
import plotly.graph_objects as go

fig = go.Figure(data=go.Scatter(x=[1, 2, 3], y=[4, 5, 6]))
openrunner.log({"interactive_plot": openrunner.Plotly(fig)})
```

### Point Clouds

```python
import numpy as np

points = np.random.randn(1000, 3)
colors = np.random.randint(0, 255, (1000, 3), dtype=np.uint8)
openrunner.log({"lidar": openrunner.PointCloud3D(points, colors=colors)})
```

### Bounding Boxes

```python
img = openrunner.Image("photo.jpg")
boxes = [{"position": {"minX": 10, "minY": 20, "maxX": 100, "maxY": 150}, "class_id": 0}]
openrunner.log({"detections": openrunner.BoundingBoxes2D(img, boxes, class_labels={0: "cat"})})
```

### Audio

```python
import numpy as np

# From numpy array (mono, float32, -1 to 1)
audio = openrunner.Audio(np.random.randn(44100).astype(np.float32), sample_rate=44100)
openrunner.log({"audio_sample": audio})
```

### Video

```python
# From file path
openrunner.log({"demo": openrunner.Video("output.mp4", caption="training demo")})
```

### Matplotlib Figures

```python
import matplotlib.pyplot as plt

plt.figure()
plt.plot([1, 2, 3], [4, 5, 6])
openrunner.log({"chart": openrunner.MatplotlibFigure()})  # captures current figure
plt.close()
```

## LLM Tracing

Trace LLM API calls for debugging and cost tracking.

```python
import openrunner

openrunner.init(project="llm-app")

# Auto-trace OpenAI calls
openrunner.trace.patch_openai()

# Or manually trace any function
@openrunner.trace
def generate(prompt):
    return client.chat.completions.create(
        model="gpt-4", messages=[{"role": "user", "content": prompt}]
    )
```

## Hyperparameter Sweeps

Run distributed hyperparameter searches.

```python
import openrunner

sweep_config = {
    "method": "bayes",
    "metric": {"name": "val_loss", "goal": "minimize"},
    "parameters": {
        "lr": {"min": 1e-5, "max": 1e-2, "distribution": "log_uniform"},
        "epochs": {"values": [10, 20, 50]},
    },
}

sweep_id = openrunner.sweep(sweep_config, project="my-project")

def train():
    run = openrunner.init()
    lr = openrunner.config.lr
    # ... training loop ...
    openrunner.finish()

openrunner.agent(sweep_id, function=train, count=20)
```

## Remote Launch

Submit training jobs to remote infrastructure.

```python
import openrunner

job = openrunner.launch(
    project="my-project",
    config={"lr": 0.001, "epochs": 50},
    resource="gpu-a100",
)

job.wait()  # block until finished
print(job.state)  # "finished"
```

## Model Registry

Version and alias models for production deployment.

```python
# Log a model with aliases
artifact = openrunner.Artifact(name="classifier", type="model")
artifact.add_file("model.pt")
openrunner.link_artifact(artifact, aliases=["staging"])

# Use a model by alias
model_dir = openrunner.use_artifact("classifier:production")
```

## Alerts

Send notifications when training reaches milestones or encounters issues.

```python
openrunner.alert(title="Training complete", text="Final accuracy: 95.2%", level="INFO")
openrunner.alert(title="Loss spike detected", level="WARN")
```

## Query API

Read-only access to runs, metrics, and projects for analysis and dashboards.

```python
api = openrunner.Api()
runs = api.runs("my-project", filters={"state": "finished"})
for run in runs:
    print(f"{run.name}: {run.summary.get('accuracy')}")
```

## Migrating from W&B

Change one import — everything else stays the same:

```python
# Before
import wandb
wandb.init(project="my-project")
wandb.log({"loss": 0.5})
wandb.finish()

# After
import openrunner as wandb
wandb.init(project="my-project")
wandb.log({"loss": 0.5})
wandb.finish()
```

## Framework Integrations

### PyTorch

```python
from openrunner.integration.pytorch import log_gradients

openrunner.init(project="pytorch-example")

for batch in dataloader:
    loss = model(batch)
    loss.backward()
    log_gradients(model)  # Logs gradient norms
    optimizer.step()

openrunner.finish()
```

### HuggingFace Transformers

```python
from openrunner.integration.huggingface import OpenRunnerCallback

openrunner.init(project="hf-example")

trainer = Trainer(
    model=model,
    args=training_args,
    callbacks=[OpenRunnerCallback()],
)
trainer.train()

openrunner.finish()
```

### PyTorch Lightning

```python
from openrunner.integration.lightning import OpenRunnerLogger

logger = OpenRunnerLogger(project="lightning-example")

trainer = pl.Trainer(logger=logger)
trainer.fit(model)
```

### Keras

```python
from openrunner.integration.keras import OpenRunnerCallback

openrunner.init(project="keras-example")

model.fit(x_train, y_train, callbacks=[OpenRunnerCallback()])

openrunner.finish()
```

### XGBoost

```python
from openrunner.integration.xgboost import OpenRunnerCallback

openrunner.init(project="xgboost-example")

bst = xgb.train(params, dtrain, callbacks=[OpenRunnerCallback()])

openrunner.finish()
```

### scikit-learn

```python
from openrunner.integration.sklearn import log_model

openrunner.init(project="sklearn-example")
model.fit(X_train, y_train)
log_model(model)  # Logs parameters and metrics
openrunner.finish()
```

### FastAI

```python
from openrunner.integration.fastai import OpenRunnerCallback

openrunner.init(project="fastai-example")

learn = cnn_learner(dls, resnet34, cbs=[OpenRunnerCallback()])
learn.fine_tune(5)

openrunner.finish()
```

### LangChain

```python
from openrunner.integration.langchain import OpenRunnerTracer

openrunner.init(project="langchain-example")

tracer = OpenRunnerTracer()
chain.invoke({"input": "Hello"}, config={"callbacks": [tracer]})

openrunner.finish()
```

## Offline Mode

Train without connectivity, sync later:

```bash
export OPENRUNNER_MODE=offline
python train.py

# When back online
openrunner sync
```

Offline runs are stored as JSONL files (human-readable, crash-safe). Sync is additive and idempotent — interrupted syncs resume without data loss.

## CLI

```bash
# Authenticate
openrunner login

# Sync offline runs
openrunner sync

# List projects and runs
openrunner ls
```

## System Metrics

Automatically collected during training (enabled by default):

- CPU utilization (%)
- System memory usage (%)
- GPU utilization (%) — requires `pip install openrunner-sdk[gpu]`
- GPU memory usage (%)

Disable with:

```bash
export OPENRUNNER_SYSTEM_METRICS=false
```

## Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `OPENRUNNER_API_KEY` | API key for authentication | (required) |
| `OPENRUNNER_BASE_URL` | Server URL | `http://localhost:8000` |
| `OPENRUNNER_PROJECT` | Default project name | (none) |
| `OPENRUNNER_MODE` | `online` or `offline` | `online` |
| `OPENRUNNER_SYSTEM_METRICS` | Enable system metrics | `true` |
| `OPENRUNNER_OFFLINE_DIR` | Offline storage directory | `~/.openrunner/offline` |

W&B env vars (`WANDB_API_KEY`, `WANDB_BASE_URL`) are also supported as fallback for migration.

## Self-Hosting

OpenRunner is designed to be self-hosted. See the [main repo](https://github.com/jqueguiner/openrunner) for server setup with Docker Compose.

## License

MIT
