Metadata-Version: 2.4
Name: eosquality
Version: 0.2.1
Summary: Assess the quality of query data against a fitted reference population
Author-email: Ersilia Open Source Initiative <hello@ersilia.io>
License-Expression: GPL-3.0-or-later
Project-URL: Repository, https://github.com/ersilia-os/eosquality
Project-URL: Issues, https://github.com/ersilia-os/eosquality/issues
Project-URL: Documentation, https://github.com/ersilia-os/eosquality/tree/main/docs
Keywords: ersilia,applicability-domain,cheminformatics,quality-assessment
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Chemistry
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click<9,>=8.3.2
Requires-Dist: eosframes==1.2.0
Requires-Dist: FPSim2==0.7.4
Requires-Dist: rdkit==2026.3.1
Requires-Dist: loguru<1,>=0.7.3
Requires-Dist: numpy<3,>=2.4.4
Requires-Dist: packaging>=26.0
Requires-Dist: pandas<4,>=3.0.2
Requires-Dist: psutil<8,>=7.2.2
Requires-Dist: rich<16,>=15.0.0
Requires-Dist: scikit-learn<2,>=1.8.0
Requires-Dist: scipy<2,>=1.17.1
Requires-Dist: threadpoolctl<4,>=3.6.0
Provides-Extra: dev
Requires-Dist: pre-commit==4.3.0; extra == "dev"
Requires-Dist: pytest==9.0.3; extra == "dev"
Requires-Dist: ruff==0.15.11; extra == "dev"
Provides-Extra: viz
Requires-Dist: stylia==1.0.1; extra == "viz"
Dynamic: license-file

![Work in Progress](https://img.shields.io/badge/status-work%20in%20progress-orange)

# Quality of Ersilia predictions

Quality scoring for [Ersilia Model Hub](https://ersilia.io) predictions. `eosquality` describes how a model's output for a molecule compares with the same model's behaviour on a reference library and with its training sets. The scores are not ground truth: they do **not** estimate whether a prediction is correct. Per-column normalisation is done with [`eosframes`](https://github.com/ersilia-os/eosframes).

## Installation

Install the latest version of `eosquality` directly from GitHub:

```bash
pip install git+https://github.com/ersilia-os/eosquality.git
```

The CLI is then available as `eosquality`. Start by setting it up, which fetches the reference library:

```bash
eosquality setup
```

This downloads about 1.3 GB and stores the library under `~/.eosquality/indices/`.

## Quick start

The typical workflow is two commands: `fit` once per Ersilia model, then `run` on any query dataset against the saved artifacts. `fit` takes the model's predictions on the reference library (`-r`), its per-output-column training sets (`-t`), or both, and fits the matching scores. File and folder names carry the model id and version (`[prefix_]<eos_id>_<version>`, e.g. `reference_eos4e40_v1.csv`); a name without them is an error. See the [CLI docs](docs/cli.md).

### Fitting a reference library

The input CSV must hold the model's predictions on the **exact** molecules of the canonical Ersilia reference library for your installed `eosquality` version, in library order. It needs a `key` column, an `input` (SMILES) column, and one numeric column per model output. The artifacts folder (`-a`) stores the fitted scores.

There is one and only one reference library per major version of `eosquality`, so the molecule set is fixed by your install. If the SMILES entries in the input CSV don't match that library, `fit` refuses with an error.

```bash
eosquality fit -r reference_eos4e40_v1.csv -a artifacts_eos4e40_v1/
# optionally with training sets: -t training_eos4e40_v1/
# skip a score: --exclude ref_match
```

Please check [Isaura](https://github.com/ersilia-os/isaura) for a large store of pre-calculations across Ersilia models.

### Fitting training sets

`-t` takes a folder with one `<output_column>.csv` per model output, each
with a `smiles` column. Given both `-r` and `-t`, the scores cover only the
output columns that have a training set.

```bash
eosquality fit -r reference_eos4e40_v1.csv -t training_eos4e40_v1/ -a artifacts_eos4e40_v1/
```

### Running against new samples

At querying time, `run` loads a fitted artifacts folder and scores any query CSV containing Ersilia results for the same model. A training-only artifact needs nothing but SMILES, in an `input` or `smiles` column. The output CSV has `key`, `input`, and the columns of each fitted score (see below): percentiles in `(0, 1]` with their `*_raw` values, and 1 / 0 match flags. Reference scores are named `ref_…`, training scores `trn_…`.

```bash
eosquality run -i query_eos4e40_v1.csv -a artifacts_eos4e40_v1/ -o quality_eos4e40_v1.csv
```

## Scores

Typicality and extremity compare a query against the model's own predictions on the reference library, which is **not** ground truth; each is calibrated so that reference molecules score roughly Uniform(0, 1). The match flags are exact lookups against the library's molecules. Training-modality scores compare it against the model's training sets instead, calibrated so that training molecules score roughly Uniform(0, 1).

| Score | Columns | Question |
|---|---|---|
| **Typicality** | `ref_typicality_pct`, `ref_typicality_raw` | Are the predicted values ones the model commonly produces? |
| **Extremity** | `ref_extremity_pct`, `ref_extremity_raw` | Are the predicted values far from the centre of the model's output range? |
| **Reference match** | `ref_match`, `ref_scaffold` | Is the same structure, or the same Murcko scaffold, in the reference library? 1 or 0. |
| **Training similarity** | `trn_tanimoto_pct`, `trn_tanimoto_raw` | How close is the molecule to the model's training molecules (Morgan fingerprints), compared with how close they are to each other? Raw is the mean Tanimoto similarity to the 5 nearest; higher is closer. |
| **Training physchem** | `trn_physchem_pct`, `trn_physchem_raw` | The same question in physicochemical descriptor space. |
| **Training match** | `trn_match`, `trn_scaffold` | Is the same structure, or the same Murcko scaffold, in a training set? 1 or 0. |

The `_pct` columns are calibrated percentiles and the `_raw` columns the underlying values. `run --details` also writes the per-column values and the nearest training molecules.

## Documentation

- [Concepts](docs/concepts.md): how each score is computed and calibrated
- [CLI](docs/cli.md) and [Python API](docs/api.md)
- [Architecture](docs/diagram.md): fit/run data flow and save layout
- [Reference library](docs/reference-library.md): versioning, compatibility, maintainer release steps
- [Project status](docs/status.md): current results, known limitations, open questions

## License

GPL-3.0-or-later; see [LICENSE](LICENSE).

## About the Ersilia Open Source Initiative

The [Ersilia Open Source Initiative](https://ersilia.io) is a tech-nonprofit organization fueling sustainable research in the Global South. Ersilia's main asset is the [Ersilia Model Hub](https://github.com/ersilia-os/ersilia), an open-source repository of AI/ML models for antimicrobial drug discovery.

![Ersilia Logo](assets/Ersilia_Brand.png)
