Metadata-Version: 2.4
Name: zebra-open
Version: 0.1.0
Summary: Research-grade reference implementation of a ZeBRA-style retrospective risk modeling framework.
Author: Ishanu Chattopadhyay
License:                                  Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
        TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
        1. Definitions.
        
           "License" shall mean the terms and conditions for use, reproduction,
           and distribution as defined by Sections 1 through 9 of this document.
        
           "Licensor" shall mean the copyright owner or entity authorized by
           the copyright owner that is granting the License.
        
           "Legal Entity" shall mean the union of the acting entity and all
           other entities that control, are controlled by, or are under common
           control with that entity. For the purposes of this definition,
           "control" means (i) the power, direct or indirect, to cause the
           direction or management of such entity, whether by contract or
           otherwise, or (ii) ownership of fifty percent (50%) or more of the
           outstanding shares, or (iii) beneficial ownership of such entity.
        
           "You" (or "Your") shall mean an individual or Legal Entity
           exercising permissions granted by this License.
        
           "Source" form shall mean the preferred form for making modifications,
           including but not limited to software source code, documentation
           source, and configuration files.
        
           "Object" form shall mean any form resulting from mechanical
           transformation or translation of a Source form, including but
           not limited to compiled object code, generated documentation,
           and conversions to other media types.
        
           "Work" shall mean the work of authorship, whether in Source or
           Object form, made available under the License, as indicated by a
           copyright notice that is included in or attached to the work
           (an example is provided in the Appendix below).
        
           "Derivative Works" shall mean any work, whether in Source or Object
           form, that is based on (or derived from) the Work and for which the
           editorial revisions, annotations, elaborations, or other modifications
           represent, as a whole, an original work of authorship.
        
        2. Grant of Copyright License.
        
           Subject to the terms and conditions of this License, each Contributor
           hereby grants to You a perpetual, worldwide, non-exclusive, no-charge,
           royalty-free, irrevocable copyright license to reproduce, prepare
           Derivative Works of, publicly display, publicly perform, sublicense,
           and distribute the Work and such Derivative Works in Source or Object form.
        
        3. Grant of Patent License.
        
           Subject to the terms and conditions of this License, each Contributor
           hereby grants to You a perpetual, worldwide, non-exclusive, no-charge,
           royalty-free, irrevocable (except as stated in this section) patent
           license to make, have made, use, offer to sell, sell, import, and
           otherwise transfer the Work.
        
        4. Redistribution.
        
           You may reproduce and distribute copies of the Work or Derivative Works
           thereof in any medium, with or without modifications, and in Source or
           Object form, provided that You meet the following conditions:
        
           (a) You must give any other recipients of the Work or Derivative Works
               a copy of this License; and
        
           (b) You must cause any modified files to carry prominent notices stating
               that You changed the files; and
        
           (c) You must retain, in the Source form of any Derivative Works that You
               distribute, all copyright, patent, trademark, and attribution notices
               from the Source form of the Work.
        
        5. Disclaimer of Warranty.
        
           Unless required by applicable law or agreed to in writing, Licensor
           provides the Work (and each Contributor provides its Contributions)
           on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
           either express or implied, including, without limitation, any warranties
           or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS
           FOR A PARTICULAR PURPOSE.
        
        6. Limitation of Liability.
        
           In no event and under no legal theory, whether in tort (including negligence),
           contract, or otherwise, unless required by applicable law (such as deliberate
           and grossly negligent acts) or agreed to in writing, shall any Contributor
           be liable to You for damages, including any direct, indirect, special,
           incidental, or consequential damages of any character arising as a result
           of this License or out of the use or inability to use the Work.
        
        END OF TERMS AND CONDITIONS
        
        APPENDIX: How to apply the Apache License to your work.
        
           Copyright 2026 Ishanu Chattopadhyay
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
        
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
        
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: scikit-learn
Requires-Dist: joblib
Requires-Dist: matplotlib
Requires-Dist: lightgbm; platform_system != "Emscripten"
Dynamic: license-file

# ZeBRA_open

ZeBRA_open is a research-grade implementation of a retrospective risk modeling framework based on the ZeBRA architecture. It is designed for methodological research in longitudinal health record modeling, temporally correct cohort construction, odds-ratio embeddings, and stacked prediction.

> **Research Use Only**  
> This repository is not a medical device and must not be used for diagnosis, treatment, or clinical decision making. See `DISCLAIMER.md` and `CLINICAL_USE_NOTICE.md`.

---

# Installation

From the ZeBRA_open repository root:

```bash
python3 -m pip install -e ./python --no-cache-dir
```

Alternatively:

```bash
cd python
python3 -m pip install -e . --no-cache-dir
```

Verify CLI availability:

```bash
zebra-generate -h
zebra-cohort -h
zebra-train -h
zebra-eval -h
zebra-score -h
```

---

# Quickstart (CLI)

## 1) Generate Synthetic Cohort

Basic generation (DX-only JSONL cohort):

```bash
zebra-generate   --out_dir data/generated/j8411_seed0   --n_patients 10000   --target J84.11   --case_frac 0.10   --seed 0   --start_date 2010-01-01   --end_date 2024-12-31
```

Generator controls that mirror the standalone scripts:

```bash
zebra-generate   --out_dir data/generated/j8411_seed0_v2   --n_patients 10000   --target J84.11   --case_frac 0.10   --seed 1   --start_date 2010-01-01   --end_date 2024-12-31   --min_encounters_per_year 2   --max_encounters_per_year 14   --min_problem_list 2   --max_problem_list 18   --min_days_to_target 30   --max_days_to_target 2000   --target_min_occ 1   --target_max_occ 5   --target_last_third   --missingness_encounter_drop 0.05   --missingness_code_drop 0.10
```

Optional: enable the private LLM enrichment pack (OpenAI Responses API). This is disabled by default and must be explicitly turned on:

```bash
export OPENAI_API_KEY="..."
zebra-generate   --out_dir data/generated/j8411_seed0_llm   --n_patients 2000   --target J84.11   --case_frac 0.10   --seed 2   --start_date 2010-01-01   --end_date 2024-12-31   --enable_private_llm_pack   --openai_model gpt-5-nano   --openai_timeout_s 45   --openai_max_output_tokens 1200
```

Outputs:

- `patients.jsonl`
- `cohort_manifest.json`

Notes:

- The generator simulates utilization (encounters per year), per-patient observation span, and per-encounter code sampling.
- Missingness can be applied both at the encounter level (dropping encounters) and within encounters (dropping codes).
- The target code is inserted for cases with controls on placement and repetition.

---

## 2) Summarize Cohort

```bash
zebra-cohort   --patients data/generated/j8411_seed0/patients.jsonl   --target_prefix J84.11   --out results/cohort_summary_j8411.csv   --aggregate_json results/cohort_summary_j8411_agg.json
```

---

## 3) Train Retrospective Model

```bash
zebra-train   --patients data/generated/j8411_seed0/patients.jsonl   --target J84.11   --out results/zebra_j8411_v1   --observation_days 730   --horizon_days 28   --prediction_days 365   --confidence_days 365   --or_infer_frac 0.8   --val_frac 0.34   --random_state 0
```

Primary artifact:

```
results/zebra_j8411_v1/zebra_retro_model.joblib
```

---

## 4) Retrospective Evaluation (AUC + Calibration)

```bash
zebra-eval   --model results/zebra_j8411_v1/zebra_retro_model.joblib   --patients data/generated/j8411_seed0/patients.jsonl   --out results/zebra_j8411_v1/eval.csv   --summary results/zebra_j8411_v1/eval_summary.json   --calibration_bins 10   --calibration_csv results/zebra_j8411_v1/calibration.csv   --calibration_plot results/zebra_j8411_v1/calibration.png
```

---

## 5) Prospective Scoring

```bash
zebra-score   --model results/zebra_j8411_v1/zebra_retro_model.joblib   --patients data/generated/j8411_seed0/patients.jsonl   --out results/zebra_j8411_v1/scores.csv   --as_of 2023-12-31
```

If `--as_of` is omitted, scoring uses each patient's last observed event date.

---

# Python API Quickstart

## Generate Cohort

```python
from zebra_open.generator import (
    GenParams,
    generate_dx_cohort,
    write_patients_jsonl,
    write_manifest,
    parse_yyyy_mm_dd,
)

params = GenParams(
    n_patients=10000,
    case_frac=0.10,
    target_code="J84.11",
    start_date=parse_yyyy_mm_dd("2010-01-01"),
    end_date=parse_yyyy_mm_dd("2024-12-31"),
    seed=0,
    output="data/generated/j8411_seed0",
    jsonl=True,
    target_min_occurrences=1,
    target_max_occurrences=4,
    target_last_third=True,
    missingness_encounter_drop=0.05,
    missingness_code_drop=0.10,
)

patients, manifest = generate_dx_cohort(
    params,
    enable_openai_pack=False,
)

write_patients_jsonl(patients, "data/generated/j8411_seed0/patients.jsonl")
write_manifest(manifest, "data/generated/j8411_seed0/cohort_manifest.json")
```

If you want the OpenAI enrichment pack:

```python
patients, manifest = generate_dx_cohort(
    params,
    enable_openai_pack=True,
    openai_model="gpt-5-nano",
    openai_timeout_s=45,
    openai_max_output_tokens=1200,
)
```

---

## Summarize Cohort

```python
from zebra_open.cohort import summarize_file, summarize_aggregate

df = summarize_file(
    "data/generated/j8411_seed0/patients.jsonl",
    target_prefix="J84.11",
    out_path="results/cohort_summary_j8411.csv",
)

print(summarize_aggregate(df))
```

---

## Train Model

```python
from zebra_open import ZebraModel

model = ZebraModel.from_config(
    observation_days=730,
    horizon_days=28,
    prediction_days=365,
    confidence_days=365,
    or_infer_frac=0.8,
    val_frac=0.34,
    random_state=0,
)

model.fit(
    patients_json="data/generated/j8411_seed0/patients.jsonl",
    target_codes=["J84.11"],
    out_dir="results/zebra_j8411_v1",
)
```

---

# Input Format

Accepted formats:

- JSONL (one patient per line)
- JSON (list of patient objects)

Each patient must include one of:

```
patient_id
pid
id
```

DX records must follow:

```json
{
  "date": "MM/DD/YYYY",
  "code": "J84.11"
}
```

---

# Detailed CLI Option Reference

## zebra-generate

| Option | Description |
|--------|-------------|
| `--out_dir` | Output directory |
| `--n_patients` | Total patients |
| `--target` | ICD code |
| `--case_frac` | Fraction of cases |
| `--seed` | Random seed |
| `--start_date` | YYYY-MM-DD |
| `--end_date` | YYYY-MM-DD |
| `--min_encounters_per_year` | Utilization lower bound |
| `--max_encounters_per_year` | Utilization upper bound |
| `--min_problem_list` | Min chronic codes |
| `--max_problem_list` | Max chronic codes |
| `--min_days_to_target` | Target placement min (days from start) |
| `--max_days_to_target` | Target placement max (days from start) |
| `--target_min_occ` | Minimum number of target insertions for cases |
| `--target_max_occ` | Maximum number of target insertions for cases |
| `--target_last_third` | If set, force target occurrences into the final third of the timeline |
| `--missingness_encounter_drop` | Probability of dropping an encounter (0 to 1) |
| `--missingness_code_drop` | Probability of dropping a code within an encounter (0 to 1) |
| `--enable_private_llm_pack` | Enable OpenAI enrichment pack (disabled by default) |
| `--openai_model` | OpenAI model name (used only with private pack) |
| `--openai_timeout_s` | OpenAI request timeout seconds |
| `--openai_max_output_tokens` | Cap on OpenAI output tokens |

---

## zebra-cohort

| Option | Description |
|--------|-------------|
| `--patients` | Patient file |
| `--target_prefix` | ICD prefix match |
| `--out` | Summary CSV/Parquet |
| `--aggregate_json` | Aggregate stats JSON |

---

## zebra-train

| Option | Description |
|--------|-------------|
| `--patients` | Input patient file |
| `--target` | ICD code(s) |
| `--out` | Output directory |
| `--observation_days` | Feature lookback window |
| `--horizon_days` | Gap before prediction window |
| `--prediction_days` | Event window defining case |
| `--confidence_days` | Required control follow-up |
| `--or_infer_frac` | OR embedding split |
| `--val_frac` | Validation split |
| `--random_state` | Random seed |

---

## zebra-eval

| Option | Description |
|--------|-------------|
| `--model` | Path to model |
| `--patients` | Patient file |
| `--out` | Eval CSV |
| `--summary` | JSON summary |
| `--calibration_bins` | Number of bins |
| `--calibration_csv` | Calibration CSV |
| `--calibration_plot` | Calibration PNG |
| `--target` | Override target |
| `--max_controls` | Control cap |

---

## zebra-score

| Option | Description |
|--------|-------------|
| `--model` | Path to model |
| `--patients` | Patient file |
| `--out` | Scores CSV |
| `--as_of` | YYYY-MM-DD cut date |

---

# Notes

- Training and evaluation use identical temporal window logic.
- Prospective scoring performs feature construction at an as-of time without labeling.
- See `METHODS.md` for full algorithmic details.
