Metadata-Version: 2.4
Name: conforme
Version: 0.1.0
Summary: Conformal forecasting: point forecasts, hierarchy reconciliation, and calibrated bands.
Keywords: conformal prediction,forecasting,time series,hierarchical reconciliation,prediction intervals,inventory
Author: Valentin Dusserre
Author-email: Valentin Dusserre <valentindusserre@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Typing :: Typed
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: scipy
Requires-Dist: mlforecast ; extra == 'ml'
Requires-Dist: neuralforecast ; extra == 'neural'
Requires-Dist: statsforecast ; extra == 'stats'
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/Vzlentin/conforme
Project-URL: Repository, https://github.com/Vzlentin/conforme
Project-URL: Issues, https://github.com/Vzlentin/conforme/issues
Project-URL: Documentation, https://github.com/Vzlentin/conforme/tree/main/docs
Project-URL: Changelog, https://github.com/Vzlentin/conforme/blob/main/CHANGELOG.md
Provides-Extra: ml
Provides-Extra: neural
Provides-Extra: stats
Description-Content-Type: text/markdown

# Conforme

[![CI](https://github.com/Vzlentin/conforme/actions/workflows/ci.yml/badge.svg)](https://github.com/Vzlentin/conforme/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/Vzlentin/conforme/blob/main/LICENSE)
![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue.svg)

Conforme is a conformal forecasting library for panels of time series. It makes point
forecasts at many origins, reconciles them over a hierarchy, and calibrates bands and
bounds from out-of-sample residuals.

## Install

Install `conforme` from PyPI:

```sh
pip install conforme
# or
uv add conforme
```

The vendor model adapters need an extra: `stats` (statsforecast), `ml` (mlforecast), or
`neural` (neuralforecast).

```sh
pip install "conforme[stats]"
```

## Quick start

```python
import numpy as np
import pandas as pd

from conforme import Absolute, BottomUp, Hierarchy, LeadTime, Panel, SeasonalNaive, Signed
from conforme import SplitQuantile, Step, replay, rolling_forecasts
from conforme.metrics import coverage

rng = np.random.default_rng(0)
series = np.array(["a", "b", "c"])
periods = pd.date_range("2024-01-01", periods=120, freq="D")
panel = Panel(series, periods, rng.poisson(5.0, (3, 120)), "D")
hierarchy = Hierarchy.from_attributes(
    series, pd.DataFrame({"group": ["x", "x", "y"]}, index=series)
)

origins = np.arange(60, 113)  # each origin is the index of the last observed period
run = rolling_forecasts(panel, hierarchy, SeasonalNaive(7), BottomUp(hierarchy), origins, 7)
actuals = hierarchy.aggregate(panel.values)

calibrator = SplitQuantile(0.9, window=28)
bands = replay(run, actuals, target=Step(), score=Absolute(), calibrator=calibrator)
bound = replay(run, actuals, target=LeadTime(7), score=Signed(), calibrator=calibrator)
print(coverage(bands.target, bands.lower, bands.upper))
```

## Concepts

Shapes use these axes: B bottom series, T periods, N nodes with bottoms first,
S forecast series (B for `BottomUp`, N otherwise), O origins, H forecast steps, and
C target columns (H for `Step`, 1 for `LeadTime`).

![Panel and Hierarchy go to the Forecaster, base forecasts to the Reconciler, points to the Calibrator, bounds to order_up_to and settle](https://raw.githubusercontent.com/Vzlentin/conforme/main/docs/pipeline.png)

| Stage | Names |
|---|---|
| Data | `Panel`, `Hierarchy`, `Covariate` |
| Forecast | `Forecaster`, `Reconciler` (`BottomUp`, `Identity`, `WlsStruct`), `rolling_forecasts` |
| Calibrate | `Target` (`Step`, `LeadTime`), `Score` (`Absolute`, `Signed`), `Loss` (`Miss`), `Calibrator` (`SplitQuantile`, `ACI`, `QuantileTracker`, `RiskControl`), `step`, `replay` |
| Decide | `critical_ratio`, `order_up_to`, `settle` |
| Measure | `conforme.metrics`: coverage, width, interval score, pinball, cost |

Two rules hold everywhere. A model reads only its window, so a later value cannot
change a point unless a covariate declares it known ahead. A calibrator sees a score
only once its target is known, and before the origin that knows it issues.

[Architecture](https://github.com/Vzlentin/conforme/blob/main/docs/architecture.md) has the shapes and the modules.

## Write a calibrator

A calibrator owns its target, a level or a loss, and is three functions of an explicit
state. `conforme.online` handles the origins, the delays, and the indexing. A new method
is one file in `conforme/conformal/calibrators/` that imports only `calibrators.base`,
`calibrators.ranks`, and `conformal.losses`. This is the whole of a quantile tracker:

```python
import numpy as np

from conforme.conformal.calibrators.base import Calibrator


class Tracker(Calibrator):
    def __init__(self, level, lr):
        self.level = level
        self.lr = lr

    def initial_state(self, n_nodes, n_columns):
        return {"q": np.zeros((n_nodes, n_columns))}

    def update(self, state, feedback):
        q = state["q"].copy()
        for row, column in enumerate(feedback.column):  # one row = one origin, one column
            miss = feedback.scores[row] > feedback.issued[row]
            q[:, column] += self.lr * (miss - (1 - self.level))
        return {"q": q}

    def threshold(self, state):
        return state["q"].astype(np.float32)
```

Keep the state in numpy arrays. Then a product can save it after each origin with
`conforme.online.state.flatten` and continue from it.

## Models

| Model | Kind | Install |
|---|---|---|
| `conforme.forecast.models.naive.SeasonalNaive` | local | core |
| `conforme.forecast.models.statsforecast.StatsForecastModel` | local, any `statsforecast.models` model | `conforme[stats]` |
| `conforme.forecast.models.mlforecast.MLForecast` | global regressor with lags and covariates | `conforme[ml]` |
| `conforme.forecast.models.neuralforecast.NeuralForecast` | global network from `neuralforecast.models` | `conforme[neural]` |

```python
from lightgbm import LGBMRegressor
from conforme import BottomUp, Covariate, rolling_forecasts
from conforme.forecast.models.mlforecast import MLForecast

model = MLForecast(
    LGBMRegressor(), lags=[7, 14, 28], features=["price"], fit_periods=365, lookback=84
)
price = Covariate(prices, known_ahead=True, aggregate="mean")  # [B, T + H]
run = rolling_forecasts(
    panel,
    hierarchy,
    model,
    BottomUp(hierarchy),
    origins,
    28,
    refit_every=7,
    covariates={"price": price},
)
```

## Benchmarks

`bench/` is a folder of scripts. They load benchmark data from a directory, run
benchmark protocols, and compare calibrators. They import the installed `conforme`.
Raw data is not in Git.

| Script | Content |
|---|---|
| `vn2.py` | `load(path)`: weekly VN2 sales with the out-of-stock mask, hierarchy, and starting stock. `play(data, bounds)`: six orders up to the bounds, eight settled weeks, holding 0.2 and shortage 1.0 |
| `m5.py` | `load(path)`: daily M5 sales and the 12-level hierarchy (42,840 nodes). `prices(horizon)`: the sell price covariate |
| `compare.py` | `compare(forecasts, actuals, calibrators, target, score, level)`: one row of metrics per method, on the cells where each method was ready |

`vn2_conformal.py` runs the whole VN2 path in about one second:

```sh
uv run python bench/vn2_conformal.py data/vn2
```

## Documents

- [Architecture](https://github.com/Vzlentin/conforme/blob/main/docs/architecture.md): modules, array contracts, and costs.
- [Semantics](https://github.com/Vzlentin/conforme/blob/main/docs/semantics.md): the calibration rules.

## Development

```sh
uv sync --group dev
uv run pytest
uv run ruff check .
uv run ruff format --check .
uv run ty check src/
uv build --no-sources
```

See [Contributing](https://github.com/Vzlentin/conforme/blob/main/CONTRIBUTING.md).

## License

[MIT](https://github.com/Vzlentin/conforme/blob/main/LICENSE)
