Metadata-Version: 2.4
Name: preqts
Version: 0.3.0
Summary: Prequential (test-then-update) streaming evaluation for stateful time-series forecasters, with covariate-arrival simulation, streaming MASE/WQL/SQL/coverage, compute-cost accounting, and adaptive conformal calibration.
Author: Felipe Santibáñez-Leal
License: Apache-2.0
Project-URL: Homepage, https://github.com/fsantibanezleal/CAOS_PreqTS
Project-URL: Repository, https://github.com/fsantibanezleal/CAOS_PreqTS
Keywords: time-series,forecasting,evaluation,streaming,prequential,conformal,foundation-models
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Provides-Extra: statsforecast
Requires-Dist: statsforecast>=2.0; extra == "statsforecast"
Provides-Extra: chronos
Requires-Dist: chronos-forecasting>=2.0; extra == "chronos"
Provides-Extra: fev
Requires-Dist: fev>=0.9; extra == "fev"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Dynamic: license-file

# preqts

**Prequential (test-then-update) streaming evaluation for time-series forecasters.**

[![CI](https://img.shields.io/github/actions/workflow/status/fsantibanezleal/CAOS_PreqTS/ci.yml?branch=main&label=CI)](https://github.com/fsantibanezleal/CAOS_PreqTS/actions)
[![License](https://img.shields.io/github/license/fsantibanezleal/CAOS_PreqTS)](LICENSE)

Real forecasting is streaming: observations arrive over time, covariates interact, some are known
into the future, and a model must update its prediction at each step. Public benchmarks evaluate the
*batch* case: at each cutoff they re-forecast an independent window (fev, GIFT-Eval, GluonTS/Darts
rolling backtests). None of them evaluate a **stateful** forecaster that carries hidden state and
updates at constant cost, which is exactly the regime of the new recurrent and state-space
foundation models (TiRex-2 / xLSTM, FlowState / SSM). When the TiRex-2 authors needed to measure
streaming behaviour, they had to hand-roll a bespoke protocol because nothing standard existed.

`preqts` is that missing harness: it drives any forecaster through a test-then-update loop, scores
stateful and batch models identically on one stream, simulates realistic covariate arrival, and
**measures** the per-step compute cost so the streaming-cost argument is verified, not asserted.

> Part of the **ChronoScope** time-series method atlas (extracted as a standalone, PyPI-destined
> library per CAOS ADR-0061). Core depends on numpy only (Pyodide-safe); model backends are optional
> extras.

## Install

```bash
pip install preqts                 # core (numpy only)
pip install "preqts[statsforecast]" # + a batch backend to wrap
pip install "preqts[dev]"           # + pytest, ruff
```

## Quickstart

```python
import numpy as np
from preqts import SeasonalNaive, Stream, run_prequential

rng = np.random.default_rng(0)
y = 50 + 10 * np.sin(np.arange(600) * 2 * np.pi / 12) + rng.normal(0, 2, 600)

stream = Stream(y, seasonality=12, name="demo")
result = run_prequential(SeasonalNaive(12), stream, horizon=1, quantile_levels=[0.1, 0.5, 0.9])

print(result.summary())   # MASE, WQL, SQL, coverage, total_compute_s, final_predict_ms
```

See [`examples/quickstart.py`](examples/quickstart.py) for the stateful-vs-batch cost comparison.

## Core concepts

| Object | Role |
|---|---|
| `StatefulForecaster` | The contract every evaluated model implements: `start` / `predict` / `ingest`. |
| `Stream` + `Covariate` | A target series plus covariates with **arrival policies**: aligned past, lagged (late-arriving) past, known-future; NaN for missing. |
| `SeasonalNaive` | A real pure-numpy reference forecaster with empirical, horizon-widening quantiles. |
| `ReplayAdapter` | Wraps any batch forecaster into the streaming protocol by replaying accumulated history (makes batch and stateful comparable, and exposes batch cost growth). |
| `run_prequential` | The test-then-update loop; returns a `PrequentialResult`. |
| `AdaptiveConformal` | Online (ACI) recalibration of a forecaster's outer interval to track nominal coverage. |

### Metrics

`mase`, `weighted_quantile_loss` (WQL), `scaled_quantile_loss` (SQL, the probabilistic analogue of
MASE used by fev-bench), `empirical_coverage`, and `pinball_loss`. MASE and SQL are scaled by the
in-sample seasonal-naive MAE fixed from the warmup window, so every cutoff and every model share one
denominator.

## Wrapping a real batch model

Any callable `batch_fn(context, horizon, past_cov, future_cov, quantile_levels) -> (horizon, Q)` can
be wrapped:

```python
from preqts import ReplayAdapter, run_prequential

def chronos_batch(context, horizon, past_cov, future_cov, levels):
    # call a Chronos-2 / TimesFM / StatsForecast model on `context`, return quantile columns
    ...

result = run_prequential(ReplayAdapter(chronos_batch, name="chronos-2"), stream, horizon=12)
```

The adapter re-reads the whole history each `predict`, so `result.predict_latency` shows the batch
cost growing with the stream: the phenomenon a streaming-native model avoids.

## Docs

The `docs/` wiki explains the method and the design: why prequential
([docs/00_why-prequential.md](docs/00_why-prequential.md)), the protocol and stream model
([docs/01_protocol-and-stream.md](docs/01_protocol-and-stream.md)), the metrics
([docs/02_metrics.md](docs/02_metrics.md)), and the conformal module
([docs/03_conformal.md](docs/03_conformal.md)).

## Status

`v0.01.000` (2026-07-03), alpha. API may change while `0.x`. 32 tests, ruff-clean.

## License

Apache-2.0.
