Metadata-Version: 2.4
Name: analysis-pipeline
Version: 0.2.1
Summary: Core runtime for spectral processing pipelines
Author: Gaspard Russias
License: MIT
Project-URL: Homepage, https://gitlab.com/photonics-open-projects/pop_analysis_pipeline/pop_analysis_pipeline
Project-URL: Source, https://gitlab.com/photonics-open-projects/pop_analysis_pipeline/pop_analysis_pipeline
Requires-Python: >=3
Description-Content-Type: text/markdown
Requires-Dist: numpy

# pop_analysis_pipeline (`analysis-pipeline`, module `spectra_core`)

Core runtime of the spectral processing pipelines shared by ONE-PIX and PRO-PIX:
plugin discovery, a pipeline of plugins, and a runner for real measurements.

```bash
pip install analysis-pipeline          # or, for development: pip install -e .
```

Python 3.8 or later. Only dependency: numpy.

## Plugins

Plugins are packages declared on the `spectra.providers` entry point. Every module
of the package that defines `name` and `transform` (or `analyze`) is a plugin:

```toml
[project.entry-points."spectra.providers"]
basic_treatments = "basic_treatments"
```

```python
name = "crop"

def transform(spectrum, wavelengths, min_wl: float = 500.0, max_wl: float = 700.0):
    ...
    return new_spectrum, new_wavelengths        # transform: a spectrum on a grid

name = "indices"

def analyze(spectrum, wavelengths, indices: Optional[List[Literal["ndvi", "savi"]]] = None):
    return {"ndvi": 0.72}                       # analysis: {name: number}
```

One spectrum at a time (1D arrays). The parameters are described from the signature
(`discover_transforms()` / `discover_analyses()` fill `TRANSFORMS` / `ANALYSES`
with `{"fn", "params": {name: {"type", "default"}}}`). A `Literal` annotation adds
`"options"` (the values to pick from) and `"multiple"` (a list of them); `"type"`
and `"default"` are unchanged.

Existing providers: `basic-treatments` (crop, smoothing, SNV, derivative...),
`spectral-indices` (NDVI, PRI, Brix...).

## Pipeline

```python
from spectra_core.data import SpectralData
from spectra_core.pipeline import Pipeline
from spectra_core.plugins.discovery import discover_transforms, discover_analyses

discover_transforms(); discover_analyses()
result = Pipeline([("snv", {}), ("crop", {"min_wl": 600, "max_wl": 1800}),
                   ("indices", {"indices": ["ndvi"]})]).run(SpectralData(values, wavelengths))
# transforms only: SpectralData; with analyses: one {index: value} per spectrum
```

Transforms first, then analyses. `Pipeline.run` processes the matrix as it is: a
spectrum with a missing band (NaN) gives NaN.

## Runner for real measurements (`spectra_core.engine`)

The plugins are not NaN-safe, while real spectra have bands that were not measured
(beyond a detector, too little light...). `run` handles them, checks the pipeline,
and processes identical spectra once:

```python
from spectra_core.engine import run, validate, PipelineError

res = run(values, wavelengths, steps, missing="per_spectrum")   # (n, bands), NaN allowed
res.kind                   # "spectra" or "indices"
res.values, res.wavelengths    # transformed spectra (n, bands'), NaN where not processed
res.indices["ndvi"]        # one value per spectrum
res.to_legacy()            # what Pipeline.run returns
```

| `missing` | Use | What it does |
|---|---|---|
| `"per_spectrum"` (default) | a few spectra, exact | each spectrum alone, on its measured bands; one output grid |
| `"fill"` | thousands of spectra (hypercubes) | leaves out bands measured in < 50 % of the spectra and spectra measured in < 80 % of the other bands, interpolates the few NaN left, runs once; interpolated values are NaN again in the transformed spectra |

`unique=True` processes identical spectra once. The result is compact
(`rows` / `row_indices`: one row per distinct spectrum, `row_of`: the row of each
input spectrum) and is expanded only by `values` / `indices`.
`validate(steps, allow_analyses=True)` raises `PipelineError` for an unknown plugin,
a transform after an analysis, or analyses where they are not allowed; `res.notes`
says what was left out or interpolated.

## Tests

```bash
python -m pytest tests          # tests/test_engine.py: fake plugins, no provider needed
```

## Release

Every push to `main` builds the package and uploads it to PyPI (`.gitlab-ci.yml`):
change `version` in `pyproject.toml` first.
