Metadata-Version: 2.4
Name: fyron
Version: 0.1.9
Summary: Fyron is an open-source Python toolkit for interoperable healthcare data and AI workflows. It provides unified access to FHIR data via REST APIs and relational (SQL-backed) FHIR servers, integrates DICOM imaging sources, and enables semantic exploration of clinical narratives using modern language models.
License: MIT
License-File: LICENSE
Keywords: FHIR,DICOM,healthcare,clinical,NLP,LLM
Author: Bits & Flames
Author-email: hello@bitsandflames.ai
Requires-Python: >=3.10,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Provides-Extra: boruta
Provides-Extra: excel
Provides-Extra: ml
Provides-Extra: survival
Requires-Dist: Boruta (>=0.3.0,<0.4.0) ; extra == "boruta"
Requires-Dist: SimpleITK (>=2.3.1,<3.0.0)
Requires-Dist: dicomweb-client (>=0.59.1,<0.60.0)
Requires-Dist: lifelines (>=0.30.0,<0.31.0) ; extra == "survival"
Requires-Dist: matplotlib (>=3.8.0,<4.0.0) ; extra == "survival" or extra == "ml"
Requires-Dist: numpy (>=1.26.0,<2.0) ; extra == "ml"
Requires-Dist: openpyxl (>=3.1.0,<4.0.0) ; extra == "excel"
Requires-Dist: pandas (>=2.2.0,<3.0.0)
Requires-Dist: psycopg (>=3.1.19,<4.0.0)
Requires-Dist: pydicom (>=2.4.4,<3.0.0)
Requires-Dist: pyjwt (>=2.9.0,<3.0.0)
Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
Requires-Dist: requests (>=2.32.0,<3.0.0)
Requires-Dist: retrying (>=1.3.3,<2.0.0)
Requires-Dist: scikit-learn (>=1.4.0,<2.0.0) ; extra == "ml"
Requires-Dist: tqdm (>=4.66.0,<5.0.0)
Requires-Dist: xgboost (>=2.0.0,<3.0.0) ; extra == "ml"
Project-URL: Homepage, https://github.com/bitsandflames/fyron
Project-URL: Repository, https://github.com/bitsandflames/fyron
Description-Content-Type: text/markdown


![Fyron Banner](images/BF-Fyron.png)

<div align="center">
  <img alt="Python" src="https://img.shields.io/badge/python-3.10%2B-blue" />
  <img alt="License" src="https://img.shields.io/badge/license-MIT-green" />
  <img alt="FHIR" src="https://img.shields.io/badge/FHIR-enabled-0b7dda" />
  <img alt="DICOM" src="https://img.shields.io/badge/DICOM-supported-2e7d32" />
  <img alt="Documents" src="https://img.shields.io/badge/Documents-supported-6a1b9a" />
</div>

Fyron is a pragmatic Python toolkit for interoperable healthcare data and AI workflows. It gives developers clean, testable primitives for FHIR (REST + SQL), DICOM imaging, document downloads, Teable integration, and LLM-assisted analysis without hiding the underlying protocols.

**Features:** FHIR REST + SQL with pagination and caching · DICOMweb with NIfTI export · Local imaging I/O (`fyron.imaging`) · Document download with auth · Teable read/write · LLM prompts over DataFrames and files · Cohort table helpers (`fyron.cohort`) · Optional Excel (`fyron[excel]`) · Optional survival analysis (`fyron[survival]`) · Optional tabular ML (`fyron[ml]`)

## Table of Contents
- [Quickstart](#quickstart)
- [Common tasks](#common-tasks)
- [Requirements](#requirements)
- [Who It's For](#who-its-for)
- [Package Layout](#package-layout)
- [Package guide](#package-guide)
- [Setup](#setup)
- [Core Workflows](#core-workflows)
  - [FHIR REST](#fhir-rest)
  - [FHIR SQL](#fhir-sql)
  - [FHIR Utilities](#fhir-utilities)
  - [DICOM](#dicom)
  - [Documents](#documents)
  - [LLM Agent](#llm-agent)
  - [Data IO](#data-io)
  - [Teable](#teable)
  - [Imaging utilities](#imaging-utilities)
  - [Survival analysis](#survival-analysis)
  - [Tabular ML](#tabular-ml)
  - [Cohort tables](#cohort-tables)
- [Caching](#caching)
- [Logging](#logging)
- [Development](#development)
- [Troubleshooting](#troubleshooting)
- [License](#license)

## Quickstart
Get a first FHIR query running in a few steps.

**1. Install** (Python 3.10+)

```bash
# pip
pip install fyron

# Poetry (add to your project)
poetry add fyron

# uv
uv pip install fyron
# or, in a project with pyproject.toml: uv add fyron
```

With Excel support: `pip install fyron[excel]`, `poetry add fyron[excel]`, or `uv pip install "fyron[excel]"`.

With survival analysis (lifelines + matplotlib): `pip install 'fyron[survival]'`, `poetry add fyron --extras survival`, or `uv pip install "fyron[survival]"`.

With tabular ML (scikit-learn, XGBoost, matplotlib): `pip install 'fyron[ml]'`. For Boruta feature selection: `pip install 'fyron[ml,boruta]'`.

**2. Configure environment**  
If you use a repo clone: `cp .example.env .env` and edit. If you installed from PyPI, copy [.example.env](https://github.com/bitsandflames/fyron/blob/main/.example.env) from the repo or set variables from [Setup](#setup) in your shell or `.env`.

**3. Load env and run a query**
```python
from fyron import load_env, FHIRRestClient

load_env()  # optional: path=".env", override=False, warn_if_missing=True

client = FHIRRestClient()
patients = client.query_df(
    resource_type="Patient",
    params={"_count": 25, "_sort": "_id"},
    max_pages=1,
)
print(patients.head())
```

To try without any server: use `base_url="https://hapi.fhir.org/baseR4"` (public HAPI server; no auth needed for read-only).

## Common tasks
| Goal | Section / hint |
|------|----------------|
| Overview of every module | [Package guide](#package-guide) |
| First FHIR query | [Quickstart](#quickstart) |
| Use a `.env` file | `load_env()` — [Quickstart](#quickstart), [Setup](#setup) |
| Auth with token endpoint | `Auth.token_env(auth_url=..., refresh_url=...)` — [FHIR REST](#fhir-rest) |
| Export to Excel | `write_excel(df, "out.xlsx")` — install `fyron[excel]` — [Data IO](#data-io) |
| FHIRPath extraction | `query_df(..., fhir_paths=[("col", "path")])` — [FHIR REST](#fhir-rest) |
| Query from a DataFrame | `query_df_from(df, ...)` (REST or SQL) — [FHIR REST](#fhir-rest), [FHIR SQL](#fhir-sql) |
| DICOM from a table | `download_from_df(df, output_dir=...)` — columns: `study_instance_uid`, `series_instance_uid` — [DICOM](#dicom) |
| LLM over documents | `agent.prompt_documents(documents=..., prompt=...)` — [LLM Agent](#llm-agent) |
| Survival endpoints | `fyron.survival` — install `fyron[survival]` — [Survival analysis](#survival-analysis) |
| Local DICOM / NIfTI | `fyron.imaging` — [Imaging utilities](#imaging-utilities) |
| Tabular classification | `fyron.ml` — install `fyron[ml]` — [Tabular ML](#tabular-ml) |
| Join FHIR feature + outcome tables | `fyron.cohort` — [Cohort tables](#cohort-tables) |

## Requirements
- **Python** 3.10 or newer.
- **Optional:** `fhirpathpy` for FHIRPath in `query_df` — `pip install fhirpathpy`.
- **Optional:** Excel (`.xlsx`) — `pip install fyron[excel]`.
- **Optional:** Survival analysis — `pip install 'fyron[survival]'` (pulls `lifelines` and `matplotlib`).
- **Optional:** Tabular ML — `pip install 'fyron[ml]'` (pulls `numpy`, `scikit-learn`, `xgboost`, `matplotlib`). Boruta: `pip install 'fyron[boruta]'` or `fyron[ml,boruta]`.
- **Optional:** `FHIRSQLClient` (Postgres) is included if `psycopg` is available; otherwise it is `None` and only REST is used.

## Who It’s For
- Data scientists and ML engineers building clinical datasets and cohorts.
- Clinical informatics and analytics teams working across FHIR, SQL, and imaging.
- Researchers who need reproducible pipelines and methods-ready documentation.
- Engineers integrating healthcare data into apps, dashboards, or LLM workflows.

## Package Layout
```
fyron/
  __init__.py   # Top-level imports (FHIR, DICOM, LLM, core I/O, …)
  fhir/         # REST + SQL clients, auth, types, utilities
  dicom/        # DICOMweb downloader (+ CLI fyron-dicom)
  imaging/      # Local DICOM + NIfTI I/O and CT/MR normalization
  documents/    # URL / DataFrame document downloads
  llm/          # LLM prompting agent
  core/         # load_env, CSV/Excel, Teable
  cohort/       # Cohort join, validate, survival column builders
  survival/     # KM, RMST, Cox, Weibull AFT (optional extra)
  ml/           # Tabular classification ML (optional extra)
```

## Package guide

Use this section to pick the right module, install any extra, and run a minimal example. Detailed workflows follow in [Core Workflows](#core-workflows).

### Install extras

| Extra | Command | Modules unlocked |
|-------|---------|------------------|
| (default) | `pip install fyron` | `fyron`, `fyron.fhir`, `fyron.dicom`, `fyron.imaging`, `fyron.documents`, `fyron.llm`, `fyron.core`, `fyron.cohort` |
| `excel` | `pip install fyron[excel]` | `read_excel` / `write_excel` |
| `survival` | `pip install 'fyron[survival]'` | `fyron.survival` |
| `ml` | `pip install 'fyron[ml]'` | `fyron.ml` |
| `boruta` | `pip install 'fyron[boruta]'` | Boruta feature selection inside `fyron.ml` (often `fyron[ml,boruta]`) |

Combined examples: `pip install 'fyron[survival,ml]'` · `poetry add fyron --extras "survival ml"`

### `fyron` (top-level)

**Purpose:** One import for the most common clients and I/O helpers used in pipelines.

**Import:**
```python
from fyron import (
    Auth,
    FHIRRestClient,
    FHIRSQLClient,      # None if psycopg unavailable
    DICOMDownloader,
    DocumentDownloader,
    LLMAgent,
    load_env,
    read_csv,
    write_csv,
    read_excel,         # needs fyron[excel]
    write_excel,
    TeableClient,
    FHIRObj,
    safe_get,
    process_patient_bundle,
    # … other process_*_bundle helpers
)
```

| Symbol | Role |
|--------|------|
| `load_env` | Load `.env` into `os.environ` |
| `FHIRRestClient` | Query FHIR REST → `DataFrame` |
| `FHIRSQLClient` | Query Postgres FHIR store → `DataFrame` |
| `DICOMDownloader` | DICOMweb download (DICOM or NIfTI) |
| `DocumentDownloader` | Download files from URLs |
| `LLMAgent` | Prompt OpenAI-compatible APIs |
| `TeableClient` | Read/write Teable tables |
| `read_csv` / `write_csv` | Pandas CSV shortcuts |

**Quick start:** [Quickstart](#quickstart) · **Details:** [FHIR REST](#fhir-rest), [DICOM](#dicom), [Documents](#documents), [LLM Agent](#llm-agent), [Data IO](#data-io), [Teable](#teable)

---

### `fyron.fhir`

**Purpose:** Authenticate, query FHIR over REST or SQL, and normalize bundles into flat rows.

**Install:** Included in default `fyron`.

**Import:**
```python
from fyron.fhir import FHIRRestClient, FHIRSQLClient, Auth, FHIRObj, safe_get
from fyron.fhir import process_patient_bundle, extract_observation
# or: from fyron import FHIRRestClient, …
```

| API | Description |
|-----|-------------|
| `FHIRRestClient.query_df` | Paginated search → `DataFrame` |
| `FHIRRestClient.query_df_from` | Parallel per-row queries from a `DataFrame` |
| `FHIRSQLClient.query_df` / `query_df_from` | SQL → `DataFrame` with chunking |
| `Auth` / `Auth.token_env` | Session and OAuth-style tokens |
| `process_*_bundle` | Standard extractors for Patient, Observation, etc. |
| `extract_*` / `safe_get` | Single-resource field extraction |

**Quick start:**
```python
from fyron import FHIRRestClient

df = FHIRRestClient(base_url="https://hapi.fhir.org/baseR4").query_df(
    "Patient", params={"_count": 10}, max_pages=1
)
```

**Details:** [FHIR REST](#fhir-rest) · [FHIR SQL](#fhir-sql) · [FHIR Utilities](#fhir-utilities)

---

### `fyron.dicom`

**Purpose:** Download studies or series from a DICOMweb server; optional NIfTI export and manifests.

**Install:** Default `fyron` (needs `pydicom`, `SimpleITK`, `dicomweb-client`).

**Import:**
```python
from fyron import DICOMDownloader
# CLI: fyron-dicom --help
```

| API | Description |
|-----|-------------|
| `download_series` | One series → files or `.nii.gz` |
| `download_study` | Full study (hierarchical or flat layout) |
| `download_from_df` | Batch from `study_instance_uid` / `series_instance_uid` columns |

**Quick start:**
```python
from fyron import DICOMDownloader

loader = DICOMDownloader(output_format="nifti")
loader.download_series("1.2.3", "1.2.3.4", output_dir="out")
```

**Details:** [DICOM](#dicom)

---

### `fyron.imaging`

**Purpose:** Work with **local** DICOM and NIfTI after download (headers, series load, HU/MR normalization).

**Install:** Default `fyron` (uses pydicom + SimpleITK).

**Import:**
```python
import fyron.imaging as fim
# or: from fyron.imaging import read_dicom_series, normalize_ct, read_nifti
```

| API | Description |
|-----|-------------|
| `read_dicom_header` | Tags from one `.dcm` file |
| `get_dicom_series_ids` | Discover series under a study folder |
| `read_dicom_series` | Load a series as SimpleITK image and/or NumPy (Z,Y,X) |
| `read_nifti` / `write_nifti` | NIfTI I/O |
| `normalize_ct` / `normalize_mr` | Intensity scaling for ML |

**Quick start:**
```python
import fyron.imaging as fim

image, arr = fim.read_dicom_series("study_dir", series_id="1.2.3.4.5", return_array=True)
arr_hu = fim.normalize_ct(arr, hu_min=-1000, hu_max=3000)
```

**Details:** [Imaging utilities](#imaging-utilities)

---

### `fyron.documents`

**Purpose:** Download clinical documents (PDF, text, etc.) from URLs with hashes and metadata.

**Install:** Default `fyron`.

**Import:**
```python
from fyron import DocumentDownloader
```

| API | Description |
|-----|-------------|
| `download_urls` | List of URLs → `DataFrame` of paths and metadata |
| `download_from_df` | Column of URLs → same |

**Quick start:**
```python
from fyron import DocumentDownloader

DocumentDownloader(output_dir="docs").download_urls(["https://example.org/report.pdf"])
```

**Details:** [Documents](#documents)

---

### `fyron.llm`

**Purpose:** Send prompts to OpenAI-compatible APIs over text, files, images, or DataFrame columns.

**Install:** Default `fyron`. Set `LLM_BASE_URL`, `LLM_API_KEY`, `LLM_MODEL` (see [Setup](#setup)).

**Import:**
```python
from fyron import LLMAgent
```

| API | Description |
|-----|-------------|
| `prompt` | Single text prompt |
| `prompt_documents` | List of paths/text → summaries |
| `prompt_dataframe` | Column-wise batch prompts |
| `describe_python_file` | Generate Methods-style markdown from code |

**Quick start:**
```python
from fyron import LLMAgent

LLMAgent(provider="openai").prompt("Summarize this cohort in three bullets.")
```

**Details:** [LLM Agent](#llm-agent)

---

### `fyron.core`

**Purpose:** Environment loading and lightweight data integrations (no FHIR/DICOM logic).

**Install:** Default `fyron`; Excel needs `fyron[excel]`.

**Import:**
```python
from fyron import load_env, read_csv, write_csv, TeableClient, DataIO
```

| API | Description |
|-----|-------------|
| `load_env` | Load `.env` (path, override, warn_if_missing) |
| `read_csv` / `write_csv` | Thin pandas wrappers |
| `read_excel` / `write_excel` | Requires `openpyxl` extra |
| `TeableClient` | Teable REST: read/write/overwrite tables |
| `DataIO` | Static class with same CSV/Excel methods |

**Quick start:**
```python
from fyron import load_env, read_csv

load_env()
df = read_csv("cohort.csv")
```

**Details:** [Data IO](#data-io) · [Teable](#teable)

---

### `fyron.cohort`

**Purpose:** Prepare **patient-level** tables for survival or ML: join features and outcomes, validate schema, build `time` / `event` from dates.

**Install:** Default `fyron` (pandas only).

**Import:**
```python
from fyron.cohort import join_cohort_tables, validate_cohort_table, build_survival_columns
```

| API | Description |
|-----|-------------|
| `join_cohort_tables` | Merge feature + outcome frames on `patient_id` |
| `validate_cohort_table` | Required columns, unique IDs |
| `build_survival_columns` | `time` (days) and `event` from start/end dates |

**Quick start:**
```python
from fyron.cohort import join_cohort_tables, validate_cohort_table

cohort = join_cohort_tables(features_df, outcomes_df, on="patient_id")
validate_cohort_table(cohort, required_columns=["time", "event"])
```

**Details:** [Cohort tables](#cohort-tables)

---

### `fyron.survival`

**Purpose:** Time-to-event analysis (Kaplan–Meier, RMST, Cox PH, Weibull AFT) on cohort tables with `time` and `event` columns.

**Install:** `pip install 'fyron[survival]'` (lifelines + matplotlib).

**Import:**
```python
from fyron import survival  # submodule
from fyron.survival import plot_kaplan_meier, fit_multivariate_cox, calculate_rmst
```

| API | Description |
|-----|-------------|
| `plot_kaplan_meier` | KM curves + log-rank |
| `calculate_rmst` / `compare_rmst` | Restricted mean survival time |
| `fit_univariate_cox` / `fit_multivariate_cox` | Cox proportional hazards |
| `check_ph_assumptions` | Proportional hazards diagnostics |
| `fit_weibull_aft` / `plot_aft_survival_curves` | Parametric survival |
| `create_survival_groups` | Median/quartile/custom risk groups |

**Quick start:**
```python
from fyron.survival import plot_kaplan_meier

plot_kaplan_meier(df, time_col="time", event_col="event", group_col="stage")
```

**Details:** [Survival analysis](#survival-analysis)

---

### `fyron.ml`

**Purpose:** Tabular **classification** for healthcare datasets: splits, imbalance, RF/XGBoost, metrics (sensitivity, specificity, AUC), CV, plots, grid search, optional Boruta.

**Install:** `pip install 'fyron[ml]'`. Boruta: `pip install 'fyron[ml,boruta]'`.

**Import:**
```python
from fyron import ml
# or: from fyron.ml import train_random_forest, calculate_classification_metrics
```

| API | Description |
|-----|-------------|
| `train_test_split_data` | Stratified train/test split |
| `compute_class_weights` / `compute_scale_pos_weight` | Imbalance helpers |
| `train_random_forest` / `train_xgboost` | Fit + metrics (+ optional plots) |
| `calculate_classification_metrics` | Accuracy, F1, sensitivity, specificity, AUC, TN/FP/FN/TP |
| `cross_validate_model` | Stratified k-fold with per-fold metrics |
| `grid_search_model` / `get_default_param_grid` | Hyperparameter search |
| `plot_roc_curve` / `plot_confusion_matrix` | Evaluation figures |
| `get_feature_importance` / `plot_feature_importance` | Model interpretation |
| `run_boruta_feature_selection` | Optional all-relevant features |
| `run_classification_pipeline` | Split → (Boruta) → train → metrics |
| `save_estimator` / `load_estimator` | joblib persistence |

**Quick start:**
```python
from fyron import ml

X_train, X_test, y_train, y_test = ml.train_test_split_data(X, y, stratify=True)
result = ml.train_random_forest(
    X_train, y_train, X_test, y_test,
    class_weight=ml.compute_class_weights(y_train),
    n_estimators=200,
    random_state=42,
)
ml.plot_roc_curve(y_test, result["y_prob"])
```

**Details:** [Tabular ML](#tabular-ml)

---

### Typical pipeline

```
FHIR / SQL / Teable  →  patient DataFrames
        ↓
fyron.cohort          →  join + validate + survival columns
        ↓
fyron.survival / fyron.ml / fyron.imaging  →  analysis
```

## Setup
Fyron uses environment variables for endpoints and credentials. You can set them in your shell or in a `.env` file (use [.example.env](https://github.com/bitsandflames/fyron/blob/main/.example.env) as a template).

**Minimum for Quickstart (FHIR REST only)**  
Set `FHIR_BASE_URL`. For token auth, also set `FHIR_AUTH_URL`, `FHIR_USER`, and `FHIR_PASSWORD`. Call `load_env()` if you use a `.env` file.

**Full reference** — all optional, depending on which features you use:
```bash
# FHIR REST
export FHIR_BASE_URL="https://example.org/fhir"
export FHIR_AUTH_URL="https://example.org/auth/token"
export FHIR_REFRESH_URL="https://example.org/auth/refresh"
export FHIR_USER="alice"
export FHIR_PASSWORD="secret"

# DICOMweb
export DICOM_WEB_URL="https://pacs.example.org/dicomweb"
export DICOM_USER="alice"
export DICOM_PASSWORD="secret"

# FHIR SQL (optional; requires psycopg)
export FHIR_DB_HOST="localhost"
export FHIR_DB_PORT="5432"
export FHIR_DB_NAME="fhir"
export FHIR_DB_USER="postgres"
export FHIR_DB_PASSWORD="secret"

# LLM
export LLM_PROVIDER="openai"
export LLM_BASE_URL="https://api.openai.com"
export LLM_API_KEY="your_api_key"
export LLM_MODEL="gpt-4.1-mini"

# Teable
export TEABLE_BASE_URL="https://app.teable.ai"
export TEABLE_TOKEN="your_teable_token"
```

**Notes**
- **FHIR auth:** Token auth uses `FHIR_USER`, `FHIR_PASSWORD`, and `FHIR_AUTH_URL`. Use `Auth.token_env(auth_url=..., refresh_url=...)` to build an auth object from env.
- **`.env` loading:** `load_env()` looks for a `.env` in the current directory. Options: `path` (explicit file), `override` (overwrite existing env vars), `warn_if_missing`.
- **FHIRPath:** For `fhir_paths` in `query_df`, install `fhirpathpy`: `pip install fhirpathpy`.

## Core Workflows

### FHIR REST
Query any FHIR resource type; results are returned as a pandas DataFrame. Supports pagination, optional FHIRPath extraction, custom bundle processors, and parallel fetch from a DataFrame.
```python
from fyron import FHIRRestClient

client = FHIRRestClient()
patients = client.query_df(
    resource_type="Patient",
    params={"_count": 25, "_sort": "_id"},
    max_pages=1,
)
print(patients.head())
```

`FHIRRestClient` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `base_url` | FHIR server base URL | Defaults to `FHIR_BASE_URL` |
| `auth` | Reuse `Auth` or `requests.Session` | Optional |
| `num_processes` | Parallel worker count | Default `4` |
| `request_timeout` | Per-request timeout in seconds | Default `30` |
| `log_requests` | Log timing and summaries | `True`/`False` |
| `log_request_urls` | Print each request URL (with query string) to the console | `True`/`False` |
| `return_fhir_obj` | Wrap bundles as `FHIRObj` | `True`/`False` |

This uses FHIRPath expressions to extract specific fields.
```python
from fyron import FHIRRestClient

client = FHIRRestClient()

df = client.query_df(
    resource_type="Observation",
    params={"_count": 25},
    fhir_paths=[
        ("patient_id", "subject.reference.replace('Patient/', '')"),
        ("code", "code.coding.code"),
        ("value", "valueQuantity.value"),
    ],
    max_pages=1,
)
```

This applies a custom bundle processor with safe FHIR extraction.
`safe_get` supports dotted paths and list indexes (for example, `"code.coding[0].code"`).
```python
from fyron import FHIRObj, FHIRRestClient, safe_get

client = FHIRRestClient(return_fhir_obj=True)


def process_bundle(bundle):
    bundle = FHIRObj(**bundle) if isinstance(bundle, dict) else bundle
    rows = []
    for entry in bundle.entry or []:
        resource = entry.resource
        rows.append({
            "resourceType": safe_get(resource, "resourceType"),
            "id": safe_get(resource, "id"),
            "subject": safe_get(resource, "subject.reference"),
            "code": safe_get(resource, "code.coding[0].code"),
        })
    return {"Resource": rows}

custom_df = client.query_df(
    resource_type="Patient",
    params={"_count": 10},
    mode="custom",
    process_function=process_bundle,
    max_pages=1,
)
```

This runs one query per row in a DataFrame and fetches in parallel.
```python
from fyron import Auth, FHIRRestClient

auth = Auth.token_env(
    auth_url="https://example.org/auth/token",
    refresh_url="https://example.org/auth/refresh",
)

client = FHIRRestClient(auth=auth, num_processes=4)

result_df = client.query_df_from(
    df=patients_df,
    resource_type="Observation",
    column_map={"subject": "patient_id"},
    params={"_count": 50},
    parallel_fetch=True,
)
```

`Auth.token_env` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `auth_url` | Token endpoint URL | Required |
| `refresh_url` | Refresh endpoint | Optional |
| `token` | Pre-issued bearer token | Optional |

### FHIR SQL
Run parameterized SQL against a Postgres-backed FHIR store and get DataFrames. Requires `psycopg`; if it is not installed, `FHIRSQLClient` is `None` (REST-only installs still work).
```python
from fyron import FHIRSQLClient

sql_client = FHIRSQLClient()

patients = sql_client.query_df(
    "SELECT id, resource_type FROM fhir_resources WHERE resource_type = %s",
    params=["Patient"],
)

sql = """
SELECT id, resource_type, subject_id
FROM observation
WHERE subject_id IN ({patient_ids})
"""

obs = sql_client.query_df_from(
    df=patients_df,
    sql=sql,
    column_map={"patient_ids": "patient_id"},
    chunk_size=500,
    parallel=True,
)
```

`FHIRSQLClient` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `dsn` | Full connection string | Optional |
| `host` | DB host | Optional |
| `port` | DB port | Optional |
| `dbname` | Database name | Optional |
| `user` | DB user | Optional |
| `password` | DB password | Optional |
| `connect_timeout` | Connect timeout (seconds) | Default `10` |
| `log_queries` | Log SQL queries | `True`/`False` |

### FHIR Utilities
This uses built-in bundle processors to standardize common resources.
```python
from fyron import (
    FHIRRestClient,
    process_patient_bundle,
    process_encounter_bundle,
    process_observation_bundle,
    process_condition_bundle,
    process_procedure_bundle,
    process_imaging_study_bundle,
    process_diagnostic_report_bundle,
)

client = FHIRRestClient(return_fhir_obj=True)

patients = client.query_df(
    resource_type="Patient",
    params={"_count": 25},
    mode="custom",
    process_function=process_patient_bundle,
)
```

### DICOM
Download DICOM series via DICOMweb; optionally convert to NIfTI. Supports single series, full studies, batch from a DataFrame, and a CLI (`fyron-dicom`).

**API:** Use `DICOMDownloader`, `download_series`, `download_study`, and `download_from_df` / `download_from_dataframe` with the same `DicomResult` shape. Full-study options include **`study_layout`** (`"hierarchical"` or `"flat"`), **`write_manifest`** / **`save_study_manifest`** (control `study_manifest.json`; if `save_study_manifest` is set it overrides `write_manifest` for that flag only), **`study_folder_name`** on `download_study`, and **`study_folder_name_col`** on DataFrame downloads (names the study subdirectory from a column, e.g. accession number; values are sanitized for the filesystem; missing/blank values fall back to the hashed id).

**Behavior:** `download_series` loads one series via WADO-RS per series. `download_study` and DataFrame downloads with `download_full_study=True` load the entire study via WADO-RS at study scope and group by `SeriesInstanceUID`. **Hierarchical** layout (default) writes `{study_dir}/{series_uid}/` per series. **Flat** layout writes all DICOM files or all `{series_uid}.nii.gz` volumes directly under `{study_dir}/`. By default `{study_dir}` is a hash of the study UID unless you pass a folder label. The manifest, when enabled, lists each series and includes a `study_layout` field.

```python
from fyron import DICOMDownloader

loader = DICOMDownloader(output_format="nifti", num_processes=2)

results = loader.download_series(
    study_uid="1.2.3.4.5",
    series_uid="1.2.3.4.5.6",
    output_dir="dicom_out",
)
print(results)

# Full study — hierarchical (default): per-series subfolders; optional manifest
study_results = loader.download_study(
    study_uid="1.2.3.4.5",
    output_dir="dicom_out",
    study_layout="hierarchical",
    write_manifest=True,
)
print(study_results)

# Full study — flat: all series outputs in one study folder; no manifest
flat_results = loader.download_study(
    study_uid="1.2.3.4.5",
    output_dir="dicom_out",
    study_layout="flat",
    save_study_manifest=False,
)
print(flat_results)

# DataFrame full-study: folder per accession, flat layout, no manifest under study dir
downloads = loader.download_from_df(
    df=imaging_df,
    output_dir="dicom_out",
    download_full_study=True,
    study_layout="flat",
    study_folder_name_col="accession_number",
    save_study_manifest=False,
)
print(downloads.head())
```
Example auth options:

```python
# Reuse an Auth object
from fyron import Auth, DICOMDownloader

auth = Auth.token_env(auth_url="https://example.org/auth/token")
loader = DICOMDownloader(auth=auth)

# Standalone basic auth
loader = DICOMDownloader(basic_auth=("user", "pass"))
```

`DICOMDownloader` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `dicom_web_url` | DICOMweb endpoint | Defaults to `DICOM_WEB_URL` |
| `auth` | Reuse `Auth` or `requests.Session` | Optional |
| `basic_auth` | Standalone basic auth tuple | `(user, password)` |
| `output_format` | Output format | `"dicom"` or `"nifti"` |
| `num_processes` | Parallel workers for DataFrame downloads | Default `1` |
| `use_compression` | Compress NIfTI output | `True`/`False` |

Download studies or series from a DataFrame. Per-series mode requires **`study_instance_uid`** and **`series_instance_uid`** (or your column names). Full-study mode only needs the study column plus **`study_folder_name_col`** if you name folders from a field (e.g. accession).

CLI (`fyron-dicom`): **`--full-study`** for study CSV jobs; **`--study-layout`**, **`--manifest`** (write `study_manifest.json`); **`--no-study-manifest`** to force no manifest; **`--study-folder-name`** for a single full-study folder label; **`--study-folder-col`** with **`--full-study`** to take the folder name from a CSV column.

```bash
fyron-dicom --dicom-web-url https://pacs.example.org/dicomweb \
  --study-uid 1.2.3.4.5 --series-uid 1.2.3.4.5.6 --output-dir dicom_out

fyron-dicom --dicom-web-url https://pacs.example.org/dicomweb \
  --study-uid 1.2.3.4.5 --output-dir dicom_out --study-layout flat --manifest

fyron-dicom --dicom-web-url https://pacs.example.org/dicomweb \
  --csv studies.csv --full-study --study-folder-col accession_number \
  --study-layout flat --no-study-manifest --output-dir dicom_out
```


### Documents
Download documents from URLs (or from a DataFrame column) into a deterministic folder layout, with optional auth and metadata/hashes.
```python
from fyron import DocumentDownloader

urls = [
    "https://example.org/reports/report1.pdf",
    "https://example.org/reports/report2.pdf",
]

loader = DocumentDownloader(output_dir="docs_out")
results = loader.download_urls(urls)
print(results.head())
```
Example auth options:

```python
# Reuse an Auth object
from fyron import Auth, DocumentDownloader

auth = Auth.token_env(auth_url="https://example.org/auth/token")
loader = DocumentDownloader(auth=auth)

# Standalone basic auth
loader = DocumentDownloader(basic_auth=("user", "pass"))
```

`DocumentDownloader` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `base_url` | Prefix for relative URLs | Defaults to `FHIR_BASE_URL` |
| `output_dir` | Download folder | Default `documents_out` |
| `timeout` | Request timeout (seconds) | Default `30` |
| `skip_existing` | Skip if file exists | `True`/`False` |
| `max_workers` | Thread pool size | Default `4` |
| `save_mode` | File mode | `"auto"`, `"txt"`, `"pdf"` |
| `force_extension` | Force file extension | Optional |
| `auth` | Reuse `Auth` or `requests.Session` | Optional |
| `basic_auth` | Standalone basic auth tuple | `(user, password)` |

This downloads documents referenced by a DataFrame column.
```python
from fyron import DocumentDownloader

loader = DocumentDownloader(output_dir="docs_out")
results = loader.download_from_df(df=docs_df, url_col="document_url")
print(results.head())
```

### LLM Agent
Send prompts to OpenAI or compatible APIs; run over a list of documents, a DataFrame column, or a single prompt. Supports text, images, and file inputs.
```python
from fyron import LLMAgent

agent = LLMAgent(provider="openai")
response = agent.prompt("Summarize the key findings in this dataset")
print(response)
```

`LLMAgent` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `provider` | Provider type | `"openai"`, `"anyllm"`, `"custom"` |
| `base_url` | API base or custom endpoint | Defaults to `LLM_BASE_URL` |
| `api_key` | API key | Optional (required by some providers) |
| `model` | Model name | Defaults to `LLM_MODEL` |
| `timeout` | Request timeout (seconds) | Default `30` |
| `verify_ssl` | Verify TLS | `True`/`False` |

This runs a prompt across a list of documents or file paths.
```python
from fyron import LLMAgent

agent = LLMAgent(provider="openai")

docs = ["doc1 text", "doc2 text", "/path/to/file.txt"]
summary_df = agent.prompt_documents(
    documents=docs,
    prompt="Summarize clinically relevant findings.",
    output_csv="doc_summaries.csv",
)
print(summary_df.head())
```

This runs a prompt across a DataFrame column and appends results.
```python
from fyron import LLMAgent

agent = LLMAgent(provider="openai")

out = agent.prompt_dataframe(
    df=notes_df,
    text_col="note_text",
    prompt="Extract key diagnoses.",
    output_col="diagnoses",
    output_csv="notes_with_diagnoses.csv",
)
print(out.head())
```

This generates a Methods-style description of a Python module.
```python
from fyron import LLMAgent

agent = LLMAgent(provider="openai")
md = agent.describe_python_file(
    file_path="src/project/pipeline.py",
    output_md="docs/methods_pipeline.md",
)
print(md[:400])
```

### Data IO
Shortcut helpers for CSV and Excel: `read_csv`, `write_csv`, `read_excel`, `write_excel`. For `.xlsx` files use the Excel extra: `pip install fyron[excel]`.
```python
from fyron import read_csv, write_excel

df = read_csv("patients.csv")
write_excel(df, "patients.xlsx")
```

### Teable
Read and write [Teable](https://teable.io) tables as DataFrames: list spaces/bases/tables, get-or-create base/table, overwrite table.
```python
from fyron import TeableClient

teable = TeableClient(base_url="https://app.teable.ai", token="YOUR_TOKEN")

df = teable.read_table("tblXXXX")
record_ids = teable.write_dataframe("tblXXXX", df)
```

`TeableClient` arguments:

| Argument | Description | Values/Defaults |
| --- | --- | --- |
| `base_url` | Teable API base | Defaults to `TEABLE_BASE_URL` |
| `token` | Teable API token | Required |
| `timeout` | Request timeout (seconds) | Default `30` |

This lists spaces, bases, and tables for discovery.
```python
from fyron import TeableClient

teable = TeableClient(base_url="https://app.teable.ai", token="YOUR_TOKEN")

spaces = teable.list_spaces()
space_id = spaces[0]["id"] if spaces else None
bases = teable.list_bases(space_id=space_id) if space_id else []
tables = teable.list_tables(bases[0]["id"]) if bases else []
```

This ensures a base and table exist (creating them if needed).
```python
from fyron import TeableClient

teable = TeableClient(base_url="https://app.teable.ai", token="YOUR_TOKEN")

base = teable.get_or_create_base(name="Clinical", space_name="My Workspace")
base_id = base["id"]

table = teable.get_or_create_table(
    base_id=base_id,
    name="Observations",
)
```

This fully replaces a table by deleting then inserting all rows.
```python
from fyron import TeableClient

teable = TeableClient(base_url="https://app.teable.ai", token="YOUR_TOKEN")
record_ids = teable.overwrite_table("tblXXXX", df)
```

### Imaging utilities

**Module:** `fyron.imaging` · **Install:** default `fyron` · **See also:** [Package guide — imaging](#fyronimaging)

Use on-disk DICOM and NIfTI after [DICOM](#dicom) download (or any local folder). NumPy arrays use SimpleITK **Z, Y, X** order. For `get_dicom_series_ids(..., recursive=True)`, pass the directory that contains the series as `study_path` to `read_dicom_series`.

| Function | Description |
|----------|-------------|
| `read_dicom_header(path, tags=...)` | Selected or full header as dict |
| `get_dicom_series_ids(study_path, recursive=..., include_metadata=...)` | Series UID table |
| `read_dicom_series(study_path, series_id, return_array=...)` | Load one series |
| `read_nifti(path, return_array=...)` | Load NIfTI |
| `write_nifti(array_or_image, path, reference_image=...)` | Save NIfTI (optional geometry copy) |
| `normalize_ct(array, hu_min, hu_max)` | Clip and scale HU to [0, 1] |
| `normalize_mr(array, mode=..., percentile=...)` | MR intensity normalization |

```python
import fyron.imaging as fim

header = fim.read_dicom_header("study/series/image001.dcm")
series_ids = fim.get_dicom_series_ids("study", include_metadata=True)
image, array = fim.read_dicom_series(
    study_path="study",
    series_id=series_ids.iloc[0]["series_id"],
    return_array=True,
)
nifti, nifti_array = fim.read_nifti("image.nii.gz", return_array=True)
ct_norm = fim.normalize_ct(nifti_array, hu_min=-1000, hu_max=3000)
fim.write_nifti(ct_norm, "outputs/ct_normalized.nii.gz", reference_image=nifti)
```

### Survival analysis

**Module:** `fyron.survival` · **Install:** `pip install 'fyron[survival]'` · **See also:** [Package guide — survival](#fyronsurvival)

Requires a cohort `DataFrame` with **`time`** (duration) and **`event`** (1 = event, 0 = censored). Build these with [Cohort tables](#cohort-tables) if needed.

| Function | Description |
|----------|-------------|
| `plot_kaplan_meier(df, time_col, event_col, group_col=...)` | KM plot + log-rank |
| `create_survival_groups(df, column, method, out_col)` | median / quartile / tertile / custom bins |
| `calculate_rmst` / `compare_rmst` | Restricted mean survival time |
| `fit_univariate_cox` / `fit_multivariate_cox` | Cox PH models |
| `check_ph_assumptions` | Schoenfeld-style PH checks |
| `fit_weibull_aft` / `plot_aft_survival_curves` | Weibull accelerated failure time |

```python
import pandas as pd
from fyron.survival import (
    calculate_rmst,
    check_ph_assumptions,
    compare_rmst,
    create_survival_groups,
    fit_multivariate_cox,
    fit_univariate_cox,
    fit_weibull_aft,
    plot_aft_survival_curves,
    plot_kaplan_meier,
)

df = pd.DataFrame(
    {
        "time": [4, 8, 12, 6, 10, 14, 3, 9],
        "event": [1, 0, 1, 1, 0, 1, 1, 0],
        "age": [54, 61, 58, 70, 66, 59, 63, 67],
        "sex": ["F", "M", "F", "M", "F", "M", "F", "M"],
        "tumor_stage": ["I", "I", "II", "II", "III", "III", "II", "I"],
        "risk_score": [0.2, -0.4, 0.8, -0.1, 1.1, 0.0, -0.6, 0.5],
    }
)

km_stage = plot_kaplan_meier(
    df,
    "time",
    "event",
    group_col="tumor_stage",
    title="Kaplan–Meier by tumor stage",
    figsize=(7, 4),
)

df_q = create_survival_groups(df, "risk_score", "quartile", "risk_quartile")
km_quartiles = plot_kaplan_meier(
    df_q,
    "time",
    "event",
    group_col="risk_quartile",
    title="KM by risk score quartiles",
)

rmst_tab = calculate_rmst(df, "time", "event", group_col="tumor_stage")
rmst_pairs = compare_rmst(df, "time", "event", "tumor_stage", reference="I")

uni = fit_univariate_cox(df, "time", "event", ["age", "risk_score"])
multi = fit_multivariate_cox(
    df,
    "time",
    "event",
    formula="age + C(sex) + C(tumor_stage)",
)

ph = check_ph_assumptions(df, multi["fitter"], show_plots=False)

aft = fit_weibull_aft(df, "time", "event", covariates=["age", "risk_score"])
aft_curves = plot_aft_survival_curves(
    aft["fitter"],
    "risk_score",
    [{"risk_score": -0.5}, {"risk_score": 1.0}],
    figsize=(6, 4),
)
```

### Tabular ML

**Module:** `fyron.ml` · **Install:** `pip install 'fyron[ml]'` · **See also:** [Package guide — ml](#fyronml)

Binary classification is the primary target; multiclass is supported for several metrics and for `compute_class_weights`. Pass pandas `DataFrame`/`Series` or NumPy arrays.

| Function | Description |
|----------|-------------|
| `train_test_split_data` | Stratified train/test split |
| `compute_class_weights` / `compute_scale_pos_weight` | RF `class_weight` / XGB `scale_pos_weight` |
| `train_random_forest` / `train_xgboost` | Returns `model`, `y_pred`, `y_prob`, `metrics` |
| `calculate_classification_metrics` | Sensitivity, specificity, AUC, confusion counts |
| `cross_validate_model` | Per-fold metrics + mean/std |
| `grid_search_model` / `get_default_param_grid` | `GridSearchCV` wrapper |
| `plot_roc_curve` / `plot_confusion_matrix` | Matplotlib figures (`save_path` optional) |
| `run_classification_pipeline` | Optional end-to-end helper |

```python
from fyron import ml

X_train, X_test, y_train, y_test = ml.train_test_split_data(
    X, y, test_size=0.2, stratify=True, random_state=42
)
class_weights = ml.compute_class_weights(y_train)
scale_pos_weight = ml.compute_scale_pos_weight(y_train)

rf_result = ml.train_random_forest(
    X_train, y_train, X_test, y_test,
    class_weight=class_weights,
    n_estimators=500,
    random_state=42,
)
xgb_result = ml.train_xgboost(
    X_train, y_train, X_test, y_test,
    scale_pos_weight=scale_pos_weight,
    n_estimators=500,
    random_state=42,
)

cv = ml.cross_validate_model(rf_result["model"], X, y, cv=5)
fig_roc, _ = ml.plot_roc_curve(y_test, rf_result["y_prob"], title="Random Forest ROC")
fig_cm, _ = ml.plot_confusion_matrix(y_test, rf_result["y_pred"], title="Confusion matrix")
```

Optional Boruta (`pip install 'fyron[ml,boruta]'`):

```python
boruta = ml.run_boruta_feature_selection(X_train, y_train, max_iter=50)
X_selected = X_train[boruta["selected_features"]]
```

### Cohort tables

**Module:** `fyron.cohort` · **Install:** default `fyron` · **See also:** [Package guide — cohort](#fyroncohort)

Bridge FHIR/feature tables and [Survival](#survival-analysis) / [ML](#tabular-ml) modules.

| Function | Description |
|----------|-------------|
| `join_cohort_tables(features, outcomes, on="patient_id")` | Inner (or other) merge on patient ID |
| `validate_cohort_table(df, required_columns=..., id_column=...)` | Fail fast on missing/duplicate keys |
| `build_survival_columns(df, start_col, end_col, event_col)` | Add `time` (days) and `event` |

```python
from fyron.cohort import join_cohort_tables, validate_cohort_table, build_survival_columns

cohort = join_cohort_tables(features_df, outcomes_df, on="patient_id")
cohort = build_survival_columns(
    cohort,
    start_col="enrollment_date",
    end_col="last_followup",
    event_col="died",
)
validate_cohort_table(cohort, required_columns=["time", "event"])
```

## Caching
FHIR REST GET responses can be cached in memory to avoid repeated requests. Control TTL and size (or disable).
```python
from fyron import FHIRRestClient

client = FHIRRestClient(
    cache_ttl_seconds=300,   # default 5 minutes
    cache_max_entries=1000,  # default 1000 responses
)

# Disable caching
client = FHIRRestClient(cache_ttl_seconds=0, cache_max_entries=0)
```

## Logging
Set `logging` to INFO and use `FHIRRestClient(log_requests=True)` (or other clients’ logging options) to log request timing and summaries.
```python
import logging
from fyron import FHIRRestClient

logging.basicConfig(level=logging.INFO)
client = FHIRRestClient(log_requests=True)
```

## Development
See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. Install dependencies with Poetry and run tests:
```bash
poetry install -E survival -E ml
poetry run pytest
```
The `survival` and `ml` extras are required for `fyron.survival` and `fyron.ml` tests. Boruta tests need `poetry install -E boruta` (or `-E ml -E boruta`).
Optional integration tests (require a reachable FHIR server) run when `FYRON_INTEGRATION=1` is set; see [tests/test_fhir_rest_integration.py](tests/test_fhir_rest_integration.py).

## Troubleshooting
| Issue | What to do |
|-------|------------|
| `No module named 'fyron'` | Install the package: `pip install fyron` or, for development, `poetry install` and run with `poetry run pytest` / `poetry run python`. |
| `FHIRSQLClient` is `None` | The SQL client is optional. Install `psycopg` (included in default deps) or use REST-only. |
| Excel: missing engine / openpyxl | Install the Excel extra: `pip install fyron[excel]`. |
| No .env found / credentials not picked up | Call `load_env()` before creating clients, or set variables in the shell. Use `load_env(path=".env")` if the file is not in the current directory. |
| FHIRPath errors or missing columns | Install the optional dependency: `pip install fhirpathpy`. |
| `ImportError` for `fyron.survival` | Install: `pip install 'fyron[survival]'`. |
| `ImportError` for `fyron.ml` | Install: `pip install 'fyron[ml]'`. |
| Boruta feature selection fails | Install: `pip install 'fyron[boruta]'` or `fyron[ml,boruta]`. |

For bugs and feature requests, open an issue on the [repository](https://github.com/bitsandflames/fyron).

## License
MIT

