# OpenRunner

> OpenRunner is a governed ML platform for the full model lifecycle: datasets,
> experiment tracking + run instrumentation, training on cloud/serverless/self-hosted
> GPUs, hyperparameter sweeps, a container registry with serverless deployments,
> model cards, decisions (ADRs), handover, papers, and inference-drift monitoring.
> It ships a Python SDK + CLI (`openrunner`), an MCP server for AI coding tools
> (Claude Code, Cursor), and `/openrunner:*` slash commands. This file is the
> LLM-oriented, end-to-end map: read it before driving OpenRunner so you use the
> real commands and conventions instead of guessing.

Base URL: https://openrun.gladia.io  ·  API: `/api/v1`  ·  SDK: `pip install openrunner-sdk`

Core mental model:
- **Org → Project → Run** is the spine. Datasets + registries are **org-level**;
  runs, deployments, decisions, papers hang off a **project**. `repo_id`/project
  refs are `"org/name"` (slug) or a UUID.
- **Governed compute**: provider/registry secrets are stored encrypted server-side
  and never reach the client. Launching, terminating, pushing, and deploying go
  through the API which holds the key.
- **Everything links**: dataset → run → artifact/model → deployment. Always pass
  lineage (`datasets=` on init, `openrunner datasets link`, `openrunner deploy link`).

---

## 1. Auth & project setup

```
openrunner login                       # browser auth (stores API key)
openrunner init                        # write .openrunner config in cwd
openrunner ls                          # list projects / runs
openrunner status                      # show config + auth state
openrunner verify                      # health check against the instance
```
Slash commands: `/openrunner:login`, `/openrunner:setup`, `/openrunner:new-project`
(create a project + pin it to the directory), `/openrunner:mcp` (register the MCP
server for native tools).

Python:
```python
import openrunner
openrunner.init(project="gladia/ner-email", name="v1")   # start a run
```

---

## 2. Datasets (HuggingFace-style, org-level)

`openrunner.datasets` mirrors `huggingface_hub` + `datasets`. Datasets are
org-level and reference-first (metadata, tags, card, optional small preview);
opt into **managed** storage to actually hold bytes.

```
openrunner datasets create ORG/NAME --modality Text --format parquet \
    [--connection ML.text-datasets]          # bind managed storage (name or UUID)
openrunner datasets upload ORG/NAME FILE_OR_DIR...   # managed upload (skip_existing = sha256, content-aware)
openrunner datasets files ORG/NAME           # manifest: path,size,sha256,content_type,uploaded_by
openrunner datasets index ORG/NAME --text-col en --text-col fr   # build LanceDB FTS index
openrunner datasets search ORG/NAME "translation" [--lang fr]
openrunner datasets link ORG/NAME RUN_ID     # lineage: dataset fed this run
openrunner datasets preview|info|list|policy|reconcile ...
```
Python:
```python
from openrunner import datasets as ord
ord.create_dataset("gladia/wmt-en-fr", modality="Text", tasks=["translation"],
                   languages=["en","fr"], format="parquet", connection="ML.text-datasets")
ord.upload_folder("gladia/wmt-en-fr", "./data")     # content-aware skip
ord.index_dataset("gladia/wmt-en-fr", text_columns=["en","fr"])
ord.search_dataset("gladia/wmt-en-fr", "medical", mode="mixed", lang="fr")
ord.update_dataset("gladia/wmt-en-fr", num_examples=40_000_000)   # top-level export
```
**Derive by query** — build a child dataset from a subset of a parent (only the
matching primary keys are stored; no media copied; lineage + de-dup automatic):
```python
ord.derive_by_query(parent, keywords_csv, name="org/medical-fr", mode="word", lang="fr")
```
Managed files land under `s3://<bucket>/<conn.prefix>/datasets/<org_id>/<name>/<path>`.
Cards render live ```echarts``` and ```mermaid``` blocks — use REAL data only.
Slash: `/openrunner:new-dataset`, `/openrunner:derive-dataset`, `/openrunner:fetch-data`.

---

## 3. Runs & instrumentation

```python
openrunner.init(project="gladia/ner-email", name="v15",
                config={...}, datasets=["gladia/fleurs", "<uuid>"])  # ALWAYS link data
openrunner.log({"loss": 0.21, "wer": 0.14}, step=1200)               # metrics live in run summary
openrunner.log_artifact("model-int8", "./out", aliases=["latest"])   # git-LFS file bucket + card
openrunner.set_model_card(name, md); openrunner.get_model_card(name)
openrunner.finish()
```
- Metrics live in the run `summary`; config/env captured for reproducibility.
- Artifacts are **untyped file buckets** each with a `card_markdown` README. Call
  `log_artifact`/`link_artifact` EXACTLY once per version (a second call duplicates).
- Model cards: FIRST get the standard (`openrunner_model_card_standard`), ground
  every number in real run data, then `set_model_card` (validates, non-blocking warns).
- **Decisions (ADRs)**: `openrunner.log_decision(...)` / `/openrunner:log-decision`
  — document param changes, strategy, architecture with rationale + source runs.
CLI: `openrunner push` (publish a model/artifact in one call), `openrunner pull`
(download a run's files), `openrunner restore` (rebuild code+env of a run).
MCP for reading: `openrunner_watch_run` (ASCII sparklines), `openrunner_get_run_metrics`,
`openrunner_list_runs`, `openrunner_monitor_training` (HEALTHY/PLATEAUED/DIVERGING…).
Slash: `/openrunner:plot-session-run`, `/openrunner:training-recommendations`,
`/openrunner:result-analyser` (is the gain real vs variance/seed/leakage/overfit).

---

## 4. Training on GPUs (governed)

All paths auto-track cost and auto-terminate; the provider key never leaves the API.

**Pod providers** (RunPod, Vast.ai, Lambda, OVHcloud) — one-shot, streams logs:
```
openrunner gpu credentials add --provider runpod --api-key rp_...
openrunner gpu run --provider runpod --gpu A100_80GB --project ORG/PROJECT "<train cmd>"
openrunner gpu ls           # tracked instances
openrunner gpu terminate <id>   # GOVERNED server-side kill (uses stored key) + verify
```
**Serverless fleets** (Modal, Forge) — no pod/SSH, run a function on demand:
```
openrunner gpu run --provider forge app_script.py     # GPU declared in @app.function(gpu=...)
```
**OVHcloud AI Training** (managed batch): `--provider ovhai --image IMG --gpu-count N "<cmd>"`
— runs server-side via `ovhai`; data comes from OpenRunner (no OVH object-storage volumes).
**Self-hosted**: `openrunner register host ...` (SSH) or `openrunner runner start`
(pull-based, outbound HTTPS, no inbound SSH) or `openrunner machine up --project ORG/NAME`.
`terminate` is governed: with no explicit `--api-key` it calls the API DELETE which
kills + verifies with the stored key, and only marks terminated once the pod is
confirmed gone (never a false "terminated" that strands a billing pod).
Slash: `/openrunner:launch-gpu`, `/openrunner:train` (NL intent → picks target),
`/openrunner:forge`. The `forge-deploy` skill covers native Forge use.

---

## 5. Hyperparameter sweeps

```
openrunner sweep create sweep.yaml         # grid/random/bayes config -> SWEEP_ID
openrunner sweep agent SWEEP_ID            # run an agent that pulls trials
openrunner controller                      # local controller for custom param loops
```
Each trial is a normal run (metrics in summary), so `result-analyser` and the sweep
heatmap/efficiency views apply. Sweeps can run across the same governed compute as §4.

---

## 6. Registry & serverless deployment

**Register + log in** to a container registry (Docker Hub, GHCR, Google Artifact
Registry, AWS ECR, Azure ACR, OVH/Harbor). The upstream credential is stored
encrypted server-side; the OpenRunner broker proxies pull/push.
```
openrunner registry add --name "Harbor ML" --url REG_URL --username U --token T --namespace NS
openrunner registry set-default <id>       # broker fallback when the repo path is ambiguous
openrunner registry login                  # docker login the OpenRunner broker (short-lived token)
docker push <registry_host>/<namespace>/<project-slug>:<user>-<name>-<tag>
openrunner registry verify|ls|edit|logout
```
**Image → project link (the deploy gate).** Repo convention is
`{namespace}/{project_slug}`; the tag leads with the user (`{user}-{name}-{tag}`),
e.g. `model-team/ner-email:jl-int8-v1`.
- **Auto-link on push**: an image pushed *through the OpenRunner broker* is linked
  to its project automatically (resolved from the repo's project-slug segment), so
  it shows up under the project's **Registry** tab/panel with no manual step.
- **Explicit link** (direct-to-upstream pushes, or when the registry hides its
  catalog): `openrunner deploy link IMAGE -p ORG/NAME`.
```
openrunner deploy images -p ORG/NAME       # list linked images (the deploy gate)
openrunner deploy link IMAGE -p ORG/NAME [--registry-id UUID]
openrunner deploy gpus -p ORG/NAME         # deployable GPU offers (price/VRAM)
openrunner deploy create ... ; openrunner deploy scale <id> N ; openrunner deploy call <id> ...
openrunner deploy ls|replicas|rm
```
API: `GET/POST /projects/{id}/images`, `.../deployments`. In the UI, images appear
on the artifact **Registry** tab and the **project overview** Registry panel.
Slash: `/openrunner:registry`, `/openrunner:deploy`, `/openrunner:build-api`
(scaffold a FastAPI /predict+/health image), `/openrunner:drift` +
`/openrunner:drift-install` (wire inference-drift reporting into the API).

---

## 7. Ship it: handover, papers, drift

- **Handover**: `openrunner handover standard|ticket|check` — the Gladia Model
  Handover Standard (13 deliverables, DoD, POC/BETA/PROD-READY). "Done" ≠ "ready for
  product"; ownership transfers only when the receiving team can operate it alone.
  Slash: `/openrunner:handover`.
- **Papers**: `/openrunner:push-paper`, `/openrunner:pull-paper` — versioned LaTeX in
  the Paper Builder. MCP: `openrunner_search_papers`, `openrunner_update_paper`.
- **Inference drift**: instrument an API to report per-prediction events (PSI /
  distribution shift, segmented by metadata; no raw payloads stored). `openrunner
  drift report|monitors|check|ab|ask`. Slash: `/openrunner:drift`.

---

## 8. AI-native surface (MCP + slash commands)

Register the MCP server (`openrunner mcp` / `/openrunner:mcp`) for native tools in
Claude Code / Cursor. Prefer MCP tools over grepping logs. Highlights:
- Runs: `openrunner_watch_run`, `openrunner_get_run_metrics`, `openrunner_list_runs`,
  `openrunner_monitor_training`.
- Knowledge: `openrunner_search_sessions`, `openrunner_search_papers`,
  `openrunner_search_research`.
- Decisions: `openrunner_log_decision`, `openrunner_list_decisions`.
- Model cards / handover: `openrunner_model_card_standard`, `openrunner_set_model_card`,
  `openrunner_handover_standard`, `openrunner_validate_handover`.

Full `/openrunner:*` command list includes: setup, new-project, new-dataset,
derive-dataset, fetch-data, train, launch-gpu, forge, deploy, build-api, registry,
drift, drift-install, log-decision, result-analyser, training-recommendations,
gt-recommendations, handover, push-paper, pull-paper, plot-session-run, reload-project,
reload-experiment, sync-env, sync-session, register-host, register-runner, sessions.

---

## 9. End-to-end: a new project from scratch

1. **Create** — `/openrunner:new-project` (or `openrunner init`) → `ORG/NAME`.
2. **Data** — `openrunner datasets create ORG/data --connection ...`;
   `openrunner datasets upload ORG/data ./data`; optionally `index` + `search`, or
   `derive_by_query` a focused subset. Note the dataset id/slug.
3. **Train** — in code: `openrunner.init(project="ORG/NAME", datasets=[...])`, then
   `openrunner.log(...)` metrics and `openrunner.log_artifact(...)` the weights.
   Run it on governed compute: `openrunner gpu run --provider ... --project ORG/NAME
   "<train>"` (or Forge/Modal/OVHai/self-hosted).
4. **Sweep** — `openrunner sweep create sweep.yaml` → `openrunner sweep agent ID`;
   pick the winner with `/openrunner:result-analyser`.
5. **Package** — write the model card against `openrunner_model_card_standard`;
   `/openrunner:build-api` to scaffold a serving image.
6. **Registry** — `openrunner registry login`; `docker push` through the broker →
   the image **auto-links** to the project; verify under the project Registry panel.
7. **Deploy** — `openrunner deploy link` (if needed) → `openrunner deploy create` →
   `scale` / `call`; add `/openrunner:drift-install` for monitoring.
8. **Decide + hand over** — `openrunner.log_decision(...)` along the way;
   `openrunner handover check` before calling it PROD-READY.

---

## 10. Conventions & gotchas

- Refs are `org/name` (slug) or UUID interchangeably; datasets/registries are
  org-level, runs/deploys are project-level.
- Always link data lineage (`datasets=` on init) — otherwise runs have a "no dataset
  linked" reproducibility gap.
- `log_artifact`/`link_artifact` exactly once per version.
- Ground every card/handover number in real run data; flag unknowns as "must benchmark".
- Governed secrets never reach the client — use `openrunner gpu credentials add` /
  `openrunner registry add`, not local env keys, for shared/governed runs.
- GPU runs auto-terminate (untracked pods get a tight hour cap); prefer `--project`
  so the run is tracked. `gpu terminate` is server-side + verified.
- Registry auto-link only sees pushes **through the broker**; direct-to-upstream
  pushes need `openrunner deploy link`.

_This is the FULL OpenRunner LLM doc, served at https://openrun.gladia.io/llms-full.txt
(the short index is at /llms.txt). It mirrors the SDK's `openrunner/CLAUDE.md`.
Commands verified against the `openrunner` CLI._
