Metadata-Version: 2.5
Name: prodrome
Version: 0.1.0
Summary: Early warning for sepsis with a calibrated alert budget, validated on a hospital it never trained on.
Project-URL: Homepage, https://github.com/rhs2/prodrome
Project-URL: Documentation, https://rhs2.github.io/prodrome/
Project-URL: Repository, https://github.com/rhs2/prodrome
Project-URL: Changelog, https://github.com/rhs2/prodrome/blob/main/CHANGELOG.md
Author: Rakibul Hasan Sium
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: clinical,conformal,early-warning,icu,sepsis,time-series
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Healthcare Industry
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Requires-Python: >=3.10
Requires-Dist: lightgbm<5,>=4.3
Requires-Dist: numpy<3,>=1.26
Requires-Dist: pandas<3,>=2.1
Requires-Dist: pyarrow>=15.0
Requires-Dist: pydantic<3,>=2.9
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: rich<15,>=13.7
Requires-Dist: scikit-learn<2,>=1.4
Requires-Dist: scipy<2,>=1.11
Requires-Dist: typer<1,>=0.12
Provides-Extra: all
Requires-Dist: fastapi<1,>=0.112; extra == 'all'
Requires-Dist: flwr<2,>=1.9; extra == 'all'
Requires-Dist: huggingface-hub<1,>=0.24; extra == 'all'
Requires-Dist: opentelemetry-api<2,>=1.26; extra == 'all'
Requires-Dist: opentelemetry-sdk<2,>=1.26; extra == 'all'
Requires-Dist: prometheus-client<1,>=0.20; extra == 'all'
Requires-Dist: skops<1,>=0.10; extra == 'all'
Requires-Dist: torch>=2.2; extra == 'all'
Requires-Dist: uvicorn[standard]<1,>=0.30; extra == 'all'
Requires-Dist: websockets>=12.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: fastapi<1,>=0.112; extra == 'dev'
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: opentelemetry-api<2,>=1.26; extra == 'dev'
Requires-Dist: opentelemetry-sdk<2,>=1.26; extra == 'dev'
Requires-Dist: pandas-stubs; extra == 'dev'
Requires-Dist: prometheus-client<1,>=0.20; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: types-pyyaml; extra == 'dev'
Requires-Dist: uvicorn[standard]<1,>=0.30; extra == 'dev'
Requires-Dist: websockets>=12.0; extra == 'dev'
Provides-Extra: federated
Requires-Dist: flwr<2,>=1.9; extra == 'federated'
Provides-Extra: hub
Requires-Dist: huggingface-hub<1,>=0.24; extra == 'hub'
Requires-Dist: skops<1,>=0.10; extra == 'hub'
Provides-Extra: serve
Requires-Dist: fastapi<1,>=0.112; extra == 'serve'
Requires-Dist: opentelemetry-api<2,>=1.26; extra == 'serve'
Requires-Dist: opentelemetry-sdk<2,>=1.26; extra == 'serve'
Requires-Dist: prometheus-client<1,>=0.20; extra == 'serve'
Requires-Dist: uvicorn[standard]<1,>=0.30; extra == 'serve'
Requires-Dist: websockets>=12.0; extra == 'serve'
Provides-Extra: torch
Requires-Dist: torch>=2.2; extra == 'torch'
Description-Content-Type: text/markdown

# Prodrome

[![ci](https://github.com/rhs2/prodrome/actions/workflows/ci.yml/badge.svg)](https://github.com/rhs2/prodrome/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/prodrome)](https://pypi.org/project/prodrome/)
[![Python](https://img.shields.io/pypi/pyversions/prodrome)](https://pypi.org/project/prodrome/)
[![Licence](https://img.shields.io/badge/licence-Apache--2.0-green)](LICENSE)

**Early warning for sepsis with a calibrated alert budget, validated on a hospital it
never trained on.**

A prodrome is the early set of signs that a disease is coming, before the specific
symptoms that name it. Greek *prodromos*, "running before": a forerunner.

> **Not a medical device. Not cleared by any regulator. Not for clinical use.**
> It predicts a *recorded clinical suspicion*, not a biological event. Read
> [What this must never claim](#what-this-must-never-claim) before quoting any number
> here.

---

## The numbers

Trained on hospital system A, calibrated on held-out patients from A, then tested on
every patient of hospital system B, which it never saw. External results first,
because that is the number that means something.

| config | train | test | utility | auprc | auroc | detect | in window | lead h | alerts/pt-day | precision | ece |
|---|---|---|---|---|---|---|---|---|---|---|---|
| default | A | B | 0.2468 | 0.072 | 0.802 | 0.456 | 0.153 | 33.0 | 0.11 | 0.305 | 0.003 |
| clock | A | B | 0.2413 | 0.047 | 0.628 | 0.347 | 0.065 | 45.5 | 0.04 | 0.346 | 0.097 |
| logistic | A | B | 0.1504 | 0.043 | 0.667 | 0.394 | 0.137 | 30.0 | 0.19 | 0.166 | 0.015 |
| qsofa | A | B | 0.0000 | 0.018 | 0.578 | 0.000 | 0.000 | n/a | 0.00 | n/a | 0.173 |
| sirs | A | B | 0.0000 | 0.026 | 0.652 | 0.000 | 0.000 | n/a | 0.00 | n/a | 0.274 |
| default (internal) | A | A | 0.3338 | 0.109 | 0.834 | 0.573 | 0.207 | 23.0 | 0.17 | 0.332 | 0.004 |
| clock (internal) | A | A | 0.2938 | 0.080 | 0.642 | 0.369 | 0.101 | 30.5 | 0.04 | 0.506 | 0.090 |
| logistic (internal) | A | A | 0.1976 | 0.078 | 0.763 | 0.441 | 0.176 | 25.0 | 0.19 | 0.311 | 0.006 |

`utility` is the published PhysioNet/CinC 2019 scoring function, where alerting through
the whole beneficial window scores 1 and never alerting scores 0. `detect` is the share
of septic patients ever alerted in time; `in window` is the stricter share alerted
inside the window the utility score actually rewards. `precision` is the share of
alerted patients who go on to become septic, against a base rate of 7.3 percent.

### Three findings, and none of them flatter the model

**A three-feature clock gets 98 percent of the utility score.** The `clock` row alerts
on nothing but how long the patient has been in the unit. No physiology, no laboratory
values, nothing learned. It scores 0.2413 against the trained model's 0.2468. Septic
stays in this corpus run 59 hours on average against 38 for the rest, so "this patient
has been here a while" predicts "this patient will be recorded septic at some point",
and the utility metric rewards it. Alerting on *every* patient from ICU hour 48
onward, with no model at all, scores +0.227.

What the trained model actually buys is visible in the other columns: it ranks far
better (AUROC 0.802 against 0.628), catches more than twice as many patients inside
the window that matters (0.153 against 0.065), and its probabilities mean something
(calibration error 0.003 against 0.097, a factor of thirty). That is a real
difference, and a table reporting utility alone would have hidden all of it. This is
why the clock is a shipped configuration rather than a footnote.

**The model is substantially reading the workup, not the patient.** `prodrome
ablation` refits the same model on restricted feature sets:

| ablation | features | utility | share of full | auroc | in window |
|---|---|---|---|---|---|
| full | 205 | 0.2468 | 100% | 0.8016 | 0.153 |
| physiology_only | 125 | 0.1159 | 47% | 0.7697 | 0.117 |
| process_only | 75 | 0.2159 | 87% | 0.7263 | 0.109 |
| clock_only | 3 | 0.2428 | 98% | 0.6656 | 0.074 |
| workup_only | 72 | 0.2119 | 86% | 0.7539 | 0.115 |
| no_time_index | 202 | 0.2316 | 94% | 0.7937 | 0.156 |

Measured values and bedside scores alone reach 47 percent of the full utility. The
staleness and ordering pattern alone, with no measured value of any kind, reach 86
percent. Nobody orders a lactate for a patient they are relaxed about, so *which tests
exist* encodes clinician suspicion, and the label is a record of clinician suspicion.
That is a real signal and a useful one in a deployed system, but it is not an early
warning: by the time the workup starts, somebody is already worried.

**A guarantee calibrated at one hospital does not automatically survive the move to
another.** Each model states one sentence after calibration, and the report checks it
on held-out patients:

| model | precision on A (calibrated here) | precision on B (never seen) | promise held on B |
|---|---|---|---|
| clock | 0.506 | 0.346 | yes |
| default | 0.332 | 0.305 | yes, barely |
| logistic | 0.311 | 0.166 | **no** |

The logistic model kept its promise at home and broke it at the other hospital, by a
wide margin. Nothing about its internal report predicted that. This is the entire
argument for treating external validation as the primary result rather than a
robustness appendix.

And the two bedside scores could not make a promise at all. Neither qSOFA nor SIRS can
certify 30 percent precision at any threshold within the alert budget, so their policy
alerts on nothing and their utility is exactly zero. That is the honest output, not a
failure of the harness.

Subgroup results on site B are stable rather than uniform: utility runs from 0.200 in
the medical ICU and 0.202 for patients over 80, to 0.287 for ages 65 to 79 and 0.286
in the surgical ICU, with AUROC between 0.787 and 0.818. The full table is in
`reports/latest/default-siteB.json`.

---

## Quickstart

```bash
pip install "prodrome[dev]"
prodrome init            # writes configs/
prodrome data fetch      # 42 MB, 40,336 patients, no credentials needed
prodrome data prepare    # parses to columnar and pins the corpus digest
prodrome train           # fits on site A
prodrome calibrate       # chooses the threshold and states the promise
prodrome eval            # scores site A and site B, writes a report each
prodrome ablation        # is it reading the patient or the workup?
```

`prodrome calibrate` prints the sentence the system is committing to:

```
With probability at least 95% over the calibration draw, at most 4.0 alerts are raised
per patient-day and at least 30% of alerted patients go on to develop sepsis
(observed on the calibration slice: 0.17 alerts per patient-day, 34% precision).
```

Then make it worse on purpose and watch the gate refuse it:

```bash
prodrome eval --config configs/default.yaml --out reports/good.json
prodrome gate --baseline reports/good.json --candidate reports/degraded.json
```

```
roll back: default@A against default@A
- utility dropped by 0.1500 (baseline 0.3338, candidate 0.1838); the allowed drop is 0.0000.
```

That exit code is the point. Wire it into CI and a change that makes the system worse
cannot be merged by accident.

---

## The data

The PhysioNet/CinC 2019 challenge corpus, published under the **Open Database
License**, which is why this project can exist in public: the models can be shared and
every number can be reproduced by a stranger.

| | |
|---|---|
| patients | 40,336 across two hospital systems (A: 20,336, B: 20,000) |
| rows | 1,552,210, one per ICU hour |
| septic | 2,932 patients, 7.3 percent |
| channels | 8 vital signs, 26 laboratory values, 6 demographic fields |
| size | 42 MB |

**Nothing under `data/` is committed or packaged.** `prodrome data fetch` downloads it
and `prodrome data prepare` verifies it, computing a digest over the sorted per-patient
content hashes. Every report carries that digest and the gate refuses to compare two
reports that disagree on it.

The label is `SepsisLabel`, which the organisers set six hours before a recorded
clinical onset defined by suspicion of infection together with a rise in SOFA score.
It is a proxy for a biological event, not the event.

---

## How it works

```
hourly rows  ->  causal features  ->  model  ->  risk per hour
                       |                              |
              no feature sees the                alert policy
              future, proven per column         (threshold + promise)
                       |                              |
                       |                    above -> alert, with the guarantee
                       |                    below -> silence
                       v
        evaluate on the training site AND on a site never trained on
                       |
                  promotion gate  ->  promote or exit non-zero
```

**Causality is tested, not asserted.** Every feature at hour *t* is a function of hours
0 to *t* of that stay and nothing else. The test truncates each stay at seven different
points, rebuilds, and compares every column: if one value changes, some feature read an
hour that had not happened yet, and the test names the column. A second test rebuilds
each stay alone and checks it against the same stay inside a crowd, which catches a
window running off the start into the previous patient. Both were verified by injecting
a deliberate leak and watching them fail.

**The alert policy is a decision, not a score.** On calibration patients the model has
never seen, the threshold sweep takes the lowest cut whose precision, at its exact
Clopper-Pearson lower bound, clears the floor, and whose alert rate stays inside the
budget. When nothing qualifies, `attainable` is false, the threshold goes above every
reachable risk, and the system alerts on nothing. A guarantee that cannot be certified
must not be implied.

**An alert is an event, not an hour.** If the risk sits above the threshold for
fourteen hours a ward sees one alarm, not fourteen. Alerts are counted as rising edges,
which has a consequence worth knowing: the alert rate is *not* monotone in the
threshold, because lowering it can merge two episodes into one.

---

## Configurations

| File | Model | What it is for |
|---|---|---|
| `default.yaml` | LightGBM on 205 causal features | the model the gate protects |
| `clock.yaml` | ICU hour alone | the trivial baseline every row is read against |
| `qsofa.yaml` | partial qSOFA, two of three components | the bar a ward already clears |
| `sirs.yaml` | the four SIRS criteria | the same, older and less specific |
| `logistic.yaml` | regularised logistic regression | is the boosting earning its complexity? |

A `gru` sequence model and a `sofa` baseline are registered too; the recurrent one
needs the `torch` extra. Every score here is partial where the corpus cannot support
the published definition, and the names say so: this data carries no Glasgow Coma
Scale, so qSOFA can never reach 3 and SOFA covers four of six organ systems.

---

## Serve

```bash
pip install "prodrome[serve]"
prodrome serve --config configs/default.yaml
```

`POST /score` scores a whole stay. `WS /stream` takes one hour at a time and answers
with the risk after that hour, which is the shape a real integration has and the one
that demonstrates causality end to end: the service holds the history, appends the new
hour, and has never been sent the next one. `GET /health` reports the loaded model, the
corpus digest and the promise in force. `GET /metrics` exposes Prometheus counters.
Every response carries the threshold, the guarantee and the disclaimer, because a
clinical score arriving over an API with no context is how research code ends up
somewhere it should not be.

---

## Development

```bash
make install   # virtualenv with the dev extra
make check     # ruff, mypy strict, the offline test suite
make bench     # every configuration on both sites
make ablation  # what is the model reading?
```

The test suite runs entirely on a synthetic corpus this package generates, so a fresh
clone with no download and no data licence still proves the pipeline works. CI runs
lint, types and tests on Python 3.10 and 3.12, then the measured pipeline on synthetic
data, then the gate.

What is committed: the code, the configurations, the reports under `reports/latest/`
that the gate compares against, and the leaderboard. What is not: anything under
`data/`, any trained weight, any `.env`.

---

## What this must never claim

- **Not a medical device**, not cleared by any regulator, not for clinical use.
- It predicts **a recorded clinical suspicion**, not a biological event. Every label in
  this field is a proxy and pretending otherwise is the standard dishonesty here.
- Two hospital systems in one country is not the world. The only supported claim is
  "it generalised from A to B by this much".
- The reported lead times are generous. A median of 33 hours largely reflects patients
  flagged early in a long stay, not a targeted prediction of that episode, which is
  why the strict in-window detection rate is reported beside it.
- The model leans heavily on the workup and the clock. See the ablation above.
- No survival or treatment-effect claim. This is a warning system, not a causal one.

## Licence

Apache-2.0. The corpus is separately licensed under the Open Database License by
PhysioNet and is neither redistributed nor committed here.
