Metadata-Version: 2.5
Name: homeostat-live
Version: 0.2.0
Summary: Take part in the Homeostat Live benchmark: download a week's plant data and submit predictions of its lab results.
Project-URL: Homepage, https://homeostat.kausalflow.com/live/
Author: Lei Ma
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# homeostat-live

Take part in the [Homeostat Live](https://homeostat.kausalflow.com/live/) benchmark from the command line. Each week, predict the lab results of a simulated process plant from its sensor data.

```bash
homeostat_live register alice                # once: your token is kept in ~/.config/homeostat-live
homeostat_live benchmarks                    # the benchmarks, the open one first
homeostat_live benchmark                     # download the open benchmark and every earlier week
homeostat_live submit 2026-10-09 isidis-01 homeostat-live-data/isidis-01/benchmarks/2026-10-09/predictions.csv --note "v1"
homeostat_live history                       # your submissions and scores
homeostat_live leaderboard                   # the ranking of the last frozen week, and over weeks
homeostat_live score my.csv --week 2026-10-09 --plant isidis-01   # backtest on a released week
```

A benchmark opens every Friday at 17:00 UTC with the week that has just closed. It asks for every lab result of that week, and it freezes a week later, when those lab results are released and every submission is scored. Your last submission before the freeze counts. A prediction for a sample may use only data up to the time the sample was taken: sensor data up to then, and the lab results of earlier, released weeks.

`homeostat_live benchmark` writes a folder with a README that explains its layout. Every command takes `--json`, which prints the API's answer as it is, for scripts and AI assistants. `--server` (or `HOMEOSTAT_LIVE_SERVER`) points the command at another site.

The command needs Python 3.10 or later and nothing beyond the standard library.

## Scoring

Every submission is scored when its benchmark freezes, with the code in `homeostat_live.metrics`; `homeostat_live score` runs the same code on any released week, so a model can be backtested with exactly the metrics it is ranked by.

- **Normalised error**: |prediction − lab| divided by half the width of the range of the grade being made when the sample was taken, so that both lab analyses and every grade count alike; each sample counts at most 5. **The ranking is its mean over every target of the week** (lower is better).
- **Accuracy, per analysis**: mean absolute error, root mean square error, bias, median, 90th-percentile and worst absolute error, in the analysis' units; the shares within a quarter and within half of the tolerance.
- **Grade changes**: a change's window runs from the start of its ramp to 12 hours after its end. Inside the windows: the normalised error and its ratio to steady state; the time until the lab and the prediction reach the new grade's range and stay there, and the difference between the two; the worst error of each change.
- **Off spec**: off spec being the positive class, precision, recall, F1 and balanced accuracy, per analysis and for the product (off spec when either analysis is), overall and inside the windows.
- **Dynamics**: the share of the lab's moves between consecutive samples that the prediction follows in direction, and the correlation of their changes.
- **Skill**: 1 − MAE / the reference model's MAE, against `ref-last-lab` (the last released result of the same grade) and `ref-soft-sensor` (a ridge regression on the sensor data, trained on the released weeks).
- **Per grade**: the normalised error and each analysis' error, grade by grade.

The leaderboard ranks reviewed users and the reference models by a frozen week, by the last four weeks and by all weeks.
