Metadata-Version: 2.4
Name: ehr2rl
Version: 0.2.1
Summary: Convert MIMIC-IV-style EHR data into offline reinforcement learning datasets.
Author: ehr2rl contributors
License-Expression: MIT
Project-URL: Homepage, https://github.com/pinktownscavenger/ehr2rl
Project-URL: Repository, https://github.com/pinktownscavenger/ehr2rl
Project-URL: Issues, https://github.com/pinktownscavenger/ehr2rl/issues
Keywords: ehr,healthcare,offline-rl,reinforcement-learning,mimic-iv
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scikit-learn>=1.3
Requires-Dist: scipy>=1.10
Provides-Extra: d3rlpy
Requires-Dist: d3rlpy>=2.0; extra == "d3rlpy"
Provides-Extra: bigquery
Requires-Dist: db-dtypes>=1.4; extra == "bigquery"
Requires-Dist: google-cloud-bigquery>=3.0; extra == "bigquery"
Requires-Dist: google-cloud-bigquery-storage>=2.0; extra == "bigquery"
Requires-Dist: pyarrow>=15.0; extra == "bigquery"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: black>=24.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Provides-Extra: docs
Requires-Dist: Sphinx<10,>=9.1; extra == "docs"
Requires-Dist: myst-parser<6,>=5.1; extra == "docs"
Requires-Dist: furo>=2025.12.19; extra == "docs"
Requires-Dist: sphinx-autodoc-typehints<4,>=3.13; extra == "docs"
Requires-Dist: pytest>=7.0; extra == "docs"
Provides-Extra: all
Requires-Dist: d3rlpy>=2.0; extra == "all"
Requires-Dist: db-dtypes>=1.4; extra == "all"
Requires-Dist: google-cloud-bigquery>=3.0; extra == "all"
Requires-Dist: google-cloud-bigquery-storage>=2.0; extra == "all"
Requires-Dist: pyarrow>=15.0; extra == "all"
Dynamic: license-file

# ehr2rl

![PyPI](https://img.shields.io/pypi/v/ehr2rl)
![CI](https://github.com/pinktownscavenger/ehr2rl/actions/workflows/ci.yml/badge.svg)
![Python](https://img.shields.io/pypi/pyversions/ehr2rl)

`ehr2rl` is a Python library for turning MIMIC-IV-style electronic health record
data into datasets ready for offline reinforcement learning research.

Healthcare ML libraries are usually built for supervised prediction, while
offline RL libraries expect a clean state-action-reward dataset that already
exists. `ehr2rl` fills the gap between those worlds: loading longitudinal EHR
tables, representing patient trajectories, shaping rewards, estimating observed
behavior, and exporting data for tools such as `d3rlpy`.

[Documentation](https://pinktownscavenger.github.io/ehr2rl/) ·
[PyPI](https://pypi.org/project/ehr2rl) ·
[GitHub](https://github.com/pinktownscavenger/ehr2rl)

## Features

- Load and align MIMIC-IV admissions, vitals, and lab tables into patient trajectories.
- Build bounded MIMIC-IV v3.1 cohorts through BigQuery with dry-run billing guards.
- Resolve validated v3.1 itemid maps through named feature presets.
- Construct fluids and vasopressor actions from ICU medication events.
- Construct reward signals, including mortality and SOFA delta, with a
  swappable reward interface, plus readmission and weighted composite rewards.
- Estimate observed clinician behavior policy from historical data.
- Export training-ready datasets for offline RL with `d3rlpy`, including
  optional provenance sidecars.
- Generate synthetic MIMIC-IV-style data for development and testing without
  credentialed access.

## Architecture

![ehr2rl architecture: data sources load into an EHRDataset of patient trajectories, which rewards and a behavior policy shape before export to d3rlpy](https://raw.githubusercontent.com/pinktownscavenger/ehr2rl/main/docs/_static/diagrams/architecture.png)

Every source produces the same `EHRDataset`, a list of `PatientTrajectory`
objects with aligned per-timestep arrays:

| Field | Shape | Meaning |
|---|---|---|
| `states` | `(T, D)` float | Hourly clinical features; names in `metadata["feature_names"]` |
| `actions` | `(T, A)` | Treatment per step; medication actions are `[vasopressor_bin, fluid_bin]` |
| `rewards` | `(T,)` float | Filled in by a reward's `shape()` |
| `terminals` | `(T,)` bool | `True` on the last step of the episode |
| `metadata` | dict | `died`, `feature_names`, `action_names`, `action_sizes`, `provenance`, … |

Module map:

| Module | What it does | Main entry points |
|---|---|---|
| `ehr2rl.bigquery` | Budget-guarded MIMIC-IV v3.1 cohort queries | `GuardedBigQueryClient`, `BigQueryCohort`, `load_mimiciv_bigquery_dataset` |
| `ehr2rl.data` | Dataset model, itemid maps, feature presets, legacy CSV loading | `EHRDataset`, `PatientTrajectory`, `load_itemid_map`, `get_feature_preset` |
| `ehr2rl.actions` | Vasopressor and fluid actions from ICU input events | `ActionConfig`, `DoseBins`, `build_medication_actions` |
| `ehr2rl.reward` | Swappable reward functions | `MortalityReward`, `SofaReward`, `ReadmissionReward`, `CompositeReward` |
| `ehr2rl.policy` | Estimate the observed clinician policy | `BehaviorPolicy` |
| `ehr2rl.export` | Training-ready datasets for `d3rlpy` | `to_d3rlpy` |
| `ehr2rl.provenance` | JSON sidecar describing where a dataset came from | `write_provenance`, `read_provenance` |
| `ehr2rl.testing` | Synthetic MIMIC-IV-style data | `make_synthetic_dataset` |

See [the pipeline concepts page](https://pinktownscavenger.github.io/ehr2rl/concepts/pipeline.html) for more detail.

## Installation

`ehr2rl` targets Python 3.10+.

Install the extras for the paths you need:

| Install | Adds | Use it for |
|---|---|---|
| `pip install ehr2rl` | NumPy, pandas, scikit-learn, SciPy | Synthetic data, rewards, behavior policy |
| `pip install "ehr2rl[d3rlpy]"` | `d3rlpy` | `to_d3rlpy` export and offline RL training |
| `pip install "ehr2rl[bigquery]"` | Google Cloud BigQuery client, `pyarrow`, `db-dtypes` | Credentialed MIMIC-IV v3.1 pipelines and smoke tests |
| `pip install "ehr2rl[all]"` | Both of the above | The full BigQuery-to-`d3rlpy` example |

## Quickstart

Start with synthetic data:

```python
from ehr2rl import make_synthetic_dataset, MortalityReward

ds = make_synthetic_dataset(n_patients=25, trajectory_length=24, seed=7)
ds = MortalityReward().shape(ds)

print(f"Loaded {len(ds)} patient trajectories")
print(f"State shape: {ds.trajectories[0].states.shape}")
```

Full synthetic-to-`d3rlpy` round trip:

```python
from ehr2rl import BehaviorPolicy, MortalityReward, make_synthetic_dataset, to_d3rlpy

ds = make_synthetic_dataset(
    n_patients=25,
    trajectory_length=24,
    seed=7,
    include_medication_metadata=True,
)

policy = BehaviorPolicy().fit(ds)
ds = MortalityReward().shape(ds, policy=policy)

# Requires: pip install ehr2rl[d3rlpy]
mdp_dataset = to_d3rlpy(ds)
```

The synthetic path is the default development path so tests and examples do not
require access to MIMIC-IV.

## BigQuery MIMIC-IV v3.1 Pipeline

v0.2 makes BigQuery the primary real-data path for MIMIC-IV v3.1. The local CSV
loader remains available for backward compatibility and offline work, but it is
frozen at its existing demo-schema behavior and does not receive new v3.1
validation, itemid maps, feature presets, or medication action construction.

Every BigQuery query goes through a dry-run estimate before execution and sets
`maximum_bytes_billed` on the real job as a server-side backstop.

```python
from google.cloud import bigquery

from ehr2rl.actions import ActionConfig, DoseBins
from ehr2rl.bigquery import (
    BigQueryCohort,
    CohortCriteria,
    GuardedBigQueryClient,
    load_mimiciv_bigquery_dataset,
)
from ehr2rl.data.itemid_maps import load_itemid_map
from ehr2rl.data.presets import get_feature_preset

client = GuardedBigQueryClient(
    bigquery.Client(project="YOUR_BILLING_PROJECT_ID"),
    maximum_bytes_billed=25_000_000_000,
    cache_dir=".ehr2rl_cache",
    query_timeout=60,
)

ds = load_mimiciv_bigquery_dataset(
    client=client,
    cohort=BigQueryCohort(CohortCriteria(cohort_size=25)),
    preset=get_feature_preset("vitals_only"),
    itemid_map=load_itemid_map("v3_1"),
    action_config=ActionConfig(
        vasopressor_bins=DoseBins(edges=(0.0, 0.1, 0.3, 0.6)),
        fluid_bins=DoseBins(edges=(0.0, 250.0, 500.0, 1000.0)),
    ),
)
```

Feature presets reference canonical clinical names rather than raw itemids:

```python
from ehr2rl.data.itemid_maps import load_itemid_map
from ehr2rl.data.presets import get_feature_preset

resolved = get_feature_preset("sepsis3_core").resolve(load_itemid_map("v3_1"))
print(sorted(resolved))
```

Medication actions are represented as two columns: vasopressor bin and fluid
bin. Synthetic datasets can opt into that shape for tests and examples:

```python
from ehr2rl import make_synthetic_dataset

ds = make_synthetic_dataset(include_medication_metadata=True)
print(ds[0].metadata["action_names"])
```

Exported real datasets can write a JSON provenance sidecar:

```python
from ehr2rl import to_d3rlpy

mdp_dataset = to_d3rlpy(ds, provenance_path="dataset.provenance.json")
```

## Rewards

Each reward implements `compute(trajectory)` and `shape(dataset)`, and
`shape()` returns a new dataset with rewards filled in.

| Reward | Signal | Needs in `metadata` |
|---|---|---|
| `MortalityReward` | `+1` survival or `-1` death at the terminal step | `died` |
| `SofaReward` | Per-step SOFA improvement, clipped to `[-5, 5]` | `sofa_scores` |
| `ReadmissionReward` | `+1` / `-1` for 30-day readmission at the terminal step; `NaN` when censored | `readmitted_within_30_days` |
| `CompositeReward` | Weighted sum of other rewards | Whatever its components need |

BigQuery trajectories include `died`, so `MortalityReward` works on them
directly. `sofa_scores` and `readmitted_within_30_days` must be added by the
researcher. Synthetic trajectories include `died` and `sofa_scores`. See
[reward semantics](https://pinktownscavenger.github.io/ehr2rl/concepts/reward-semantics.html).

```python
from ehr2rl import CompositeReward, MortalityReward, SofaReward

reward = CompositeReward([(MortalityReward(), 1.0), (SofaReward(), 0.1)])
ds = reward.shape(ds)
```

## Development Install

```bash
pip install -e ".[dev]"
pytest
ruff check .
mypy ehr2rl
```

To exercise the optional BigQuery and `d3rlpy` paths in development:

```bash
pip install -e ".[dev,all]"
pytest
```

## MIMIC-IV Access

`ehr2rl` does not ship, mirror, or provide access to MIMIC-IV. Researchers must
obtain any clinical data through the appropriate credentialed channels, such as
PhysioNet, and comply with the applicable data use agreements.

Credentialed users can run a bounded BigQuery smoke test without downloading the
dataset. The test is opt-in, applies a per-query bytes-billed cap, and disables
BigQuery retries so credential or network failures return quickly.

```bash
gcloud auth application-default login
gcloud auth application-default set-quota-project YOUR_BILLING_PROJECT_ID
EHR2RL_RUN_BIGQUERY_SMOKE=1 \
EHR2RL_BIGQUERY_BILLING_PROJECT=YOUR_BILLING_PROJECT_ID \
EHR2RL_BIGQUERY_TIMEOUT_SECONDS=60 \
pytest tests/test_bigquery_smoke.py
```

The smoke helper uses the MIMIC-IV v3.1 BigQuery datasets exposed as
`physionet-data.mimiciv_3_1_hosp` and `physionet-data.mimiciv_3_1_icu`.

## Known Limitations

The local CSV loader is retained for backward compatibility and offline/no-GCP
development, but v0.2 real-data validation is BigQuery-first.

`d3rlpy` currently prints a Gym deprecation warning under NumPy 2.x; this is an
upstream issue and does not affect functionality.

## Clinical Disclaimer

`ehr2rl` is research infrastructure. It is not clinical decision support, does
not validate treatment recommendations, and does not define a clinically
authoritative reward function. Reward design and interpretation remain the
responsibility of the researcher.

## Roadmap

Future priorities:

- Minari export.
- Standalone pre-flight validation.
- CLI tooling.
- Additional dataset families such as eICU, MIMIC-III, and OMOP-CDM.

## Contributing

Issues and pull requests are welcome. For v0.2, the most useful contributions
are BigQuery schema checks, validated itemid-map updates, synthetic-data edge
cases, documentation fixes, and export compatibility improvements.

If you have access to full MIMIC-IV v3.1 and spot a schema mismatch, opening an
issue with the table name and column is especially helpful.

## Citation

```bibtex
@software{ehr2rl2026,
  author  = {Bilal Malik},
  title   = {ehr2rl: Bridging MIMIC-IV and Offline Reinforcement Learning},
  year    = {2026},
  version = {0.2.1},
  url     = {https://github.com/pinktownscavenger/ehr2rl}
}
```
