Metadata-Version: 2.5
Name: caddforge-datascience
Version: 0.5.0
Summary: Data science utilities for the CADDForge toolbox (CSV plotting, profiling, etc.).
Author-email: Manuel Sellner <manuel.sellner@idorsia.com>, Christoph Sager <christoph.sager@idorsia.com>
License: MIT
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: bokeh>=3.3
Requires-Dist: caddforge-core>=0.21.0
Requires-Dist: matplotlib>=3.7
Requires-Dist: pandas>=2.1
Requires-Dist: scipy>=1.10
Description-Content-Type: text/markdown

# CADDForge Data Science Tooling

Install this plugin with `python -m pip install caddforge-datascience`. It depends directly
on `caddforge-core>=0.21.0`; Python imports and CLI commands still use `caddforge`.
For the complete public toolbox, including user-facing extras, install
`python -m pip install caddforge`. When migrating an existing environment, uninstall
the former `caddforge` core distribution before installing renamed core and updated plugins.

`caddforge-datascience` hosts lightweight data science utilities as CADDForge plugins. The first tool, **CSV Scatter**, builds an interactive Bokeh scatter plot from a CSV/TSV file with optional color mapping, molecule hover images from SMILES, and threshold sliders that drive a live confusion matrix. Axis scales can be linear/log independently, and the color map can be inverted.

The confusion matrix treats the x-threshold as the predictor and the y-threshold as the actual outcome; you can control which side of each threshold is considered positive via `x_positive_direction` and `y_positive_direction`.

## Tool: CSV Scatter
- **Section/Name:** `datascience` / `csv-scatter`
- **Purpose:** Generate an interactive scatter plot from a CSV/TSV file with optional color mapping, hover details (including rendered SMILES depictions), and threshold sliders (confusion matrix).
- **Outputs:** HTML plot under the job run directory plus a standard `result_record.json`.

### Config example (YAML)
```yaml
csv_path: data/metrics.csv
x_column: dG_tr_calc
y_column: pappab
color_column: efflux_ratio          # optional numeric column
smiles_column: smiles               # optional for hover images (requires RDKit)
id_column: idor_no                  # optional identifier shown on hover
hover_columns: [compute_time]       # any extra columns to show in hover
x_scale: linear                     # linear or log
y_scale: log                        # linear or log
invert_color_scale: false           # reverse the colormap if true
palette: viridis                    # viridis|plasma|inferno|magma|cividis|turbo|blues
thresholds_enabled: true            # show sliders + live confusion matrix
x_positive_direction: low           # which side of x threshold is positive: low|high
y_positive_direction: high          # which side of y threshold is positive: low|high
point_size: 10
alpha: 0.8
title: "Transport predictions"
run_directory: runs/demo_plot       # optional override
```

### CLI usage
```bash
# Show a template config
caddforge show-config datascience csv-scatter

# Run a plot job
caddforge run datascience csv-scatter --config plot.yaml

# Inspect job artifacts
caddforge status <job-id>
caddforge logs <job-id>
ls runs/<date>/<job-id>/results
```

### Python usage
```python
from cadd_datascience import CsvScatterConfig, CsvScatterWorkflow

config = CsvScatterConfig(
    csv_path="data/metrics.csv",
    x_column="molecular_weight",
    y_column="pIC50",
    color_column="logS",
    x_scale="log",
)
workflow = CsvScatterWorkflow(config)
job = workflow.prepare_job()
result = workflow.run(job)
print(result.paths["plot_png"])
```

## Tool: Distribution Diagnostics
- **Section/Name:** `datascience` / `distribution-diagnostics`
- **Purpose:** Diagnose whether a numeric column looks approximately normal or log-normal using summary stats, normality tests, and saved plots (histograms + Q–Q plots).
- **Outputs:** `results/results.json` report and `results/distribution_diagnostics.png` plot under the job run directory.

### Config example (YAML)
```yaml
csv_path: data.csv
column: value
bins: 50
clip_std: null
sample_size: 5000
random_seed: 0
include_log: true
plot_filename: distribution_diagnostics.png
results_filename: results.json
run_directory: runs/dist_demo
```

### CLI usage
```bash
caddforge show-config datascience distribution-diagnostics
caddforge run datascience distribution-diagnostics --config dist.yaml
ls runs/<date>/<job-id>/results
```

### Python usage
```python
from cadd_datascience import DistributionDiagnosticsConfig, DistributionDiagnosticsWorkflow

cfg = DistributionDiagnosticsConfig(csv_path="data.csv", column="value")
wf = DistributionDiagnosticsWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)
print(result.paths["report_json"])
print(result.paths["plot_png"])
```

## Development
- Plugins are registered via the `caddforge.tools` entry point group.
- Tests live under `tests/` and can be run with `pytest` once dependencies are installed.
## Compressed inputs

`csv-scatter` accepts `.csv.gz` and `.tsv.gz`, while `distribution-diagnostics` accepts `.csv.gz`,
through normal CLI, GUI, and pipeline submission. CADDForge expands them only in job-local runtime
scratch; configuration files continue to reference the original compressed file.

## Continuous integration and releases

`.github/workflows/release.yml` builds a wheel and source archive, validates
metadata, and tests the installed wheel on Linux with Python 3.10 and 3.12.
The lightweight test selection is recorded in `scripts/ci_tests.json`;
external scientific engines are validated separately.

Pushing a matching `v*` release tag publishes the tested artifacts to PyPI.
Branch pushes, pull requests and manual runs never publish. Configure a
PyPI Trusted Publisher for the `idorsia` repository, `release.yml` workflow
and `pypi` environment before pushing release tags. Runtime CADDForge
dependencies must already be available publicly; publish core first,
and publish docking/descriptors before MD.
