Metadata-Version: 2.4
Name: finval
Version: 0.5.0
Summary: Rigorous validation for synthetic financial time series
Project-URL: Homepage, https://github.com/sablier-ai/finval
Project-URL: Repository, https://github.com/sablier-ai/finval
Project-URL: Issues, https://github.com/sablier-ai/finval/issues
Author-email: Sablier <hello@sablier.it>
License: MIT
License-File: LICENSE
Keywords: finance,generative models,stylized facts,synthetic data,time series,validation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Office/Business :: Financial :: Investment
Classifier: Topic :: Scientific/Engineering :: Mathematics
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.11
Provides-Extra: dev
Requires-Dist: matplotlib>=3.7; extra == 'dev'
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pandas>=2.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == 'pandas'
Provides-Extra: plot
Requires-Dist: matplotlib>=3.7; extra == 'plot'
Description-Content-Type: text/markdown

# finval

**Rigorous validation for synthetic financial time series.**

`finval` is a Python library for assessing the quality of synthetic
market data against real data. It was built because no existing library
covers the financial stylized facts that matter: fat tails, volatility
clustering, leverage effect, crash co-movement, and probabilistic
forecast calibration.

`finval` is the scoring backend behind
**[FinBench](https://github.com/sablier-ai/finbench)**, the public
leaderboard for multivariate financial time-series generation.

> **Current release: `0.5.0`** (`0.2.0`/`0.3.0`/`0.4.0` preserved at their tags
> for reproducibility). 0.5.0 adds an **opt-in `subwindow` mode** for long-horizon
> path validation plus a per-metric **`effective_n`**, both purely additive — the
> default (`subwindow=None`) is **byte-identical to 0.4.0**, so existing FinBench
> scores do not move. The 0.4.0 scoring structure is unchanged: **6 weighted lenses** —
> marginal (0.15), dependence (0.20), temporal (0.13), joint (0.10),
> **conditional (0.22)** and **generative (0.20)** — plus **7 hard gates** that
> fail a model outright regardless of the weighted score (`memorization`,
> `tail_quantiles`, `tail_dependence_lower`, `drawdown_distribution`,
> `conditional_sensitivity`, `c2st`, `coverage_deficit`). New in 0.4.0: a
> regime-stratified **`conditional_sensitivity`** axis (catches a
> calibrated-but-*climatological* generator whose forecast barely moves across
> regimes), a **generative** lens (Naeem density/coverage scored as the delta vs
> a block-bootstrap *replay* of the real data — a generator must beat replay to
> justify itself), a **joint** C2ST omnibus, and **graceful degradation**
> (metrics flagged non-`applicable` on thin-data / long-horizon inputs rather
> than scored as failures). The library is in active use (FinBench v2 production
> scoring); pin an exact version if you need score-stability across releases.

## Why finval?

General-purpose synthetic data libraries (`sdmetrics`, `synthcity`,
`tsgm`) treat time series as generic sequences. They don't know what
"leverage effect" is, don't check PIT uniformity, and don't compute tail
dependence coefficients. For financial applications — risk management,
backtesting, derivatives — you need a suite that tests the things that
actually matter for market data.

We're not aware of another library that combines, for financial time series, the
full stylized-fact battery (fat tails, volatility clustering, leverage effect,
crash co-movement, **time-irreversibility**, long memory, aggregational
gaussianity) with a path-law **signature** metric, a **C2ST** omnibus,
proper-scoring calibration (CRPS / PIT / coverage), **regime-conditional**
sensitivity, and **memorization** gating against a real-vs-real baseline — in one
weighted, gated battery.

`finval` computes **35 metric functions** for synthetic financial time series:
**26 carry weight** across 6 lenses (marginal, dependence, temporal, joint,
conditional, generative); the balance are **weight-0 diagnostic localizers** —
computed and reported so they pinpoint *where* a defect is without destabilizing
the headline score, and including a **path-signature** path-law metric — and
**7 hard gates** fail a model outright regardless of the weighted score. Every
threshold is calibrated against real financial data and justified by the
statistical literature. The metrics split across three entry points by input shape:

- **11 flat metrics** — 3 distributional (`marginal_ks`,
  `energy_distance`, `tail_quantiles`) and 8 dependence (`pearson_corr`,
  `spearman_corr`, `copula_distance`, `tail_dependence_upper`,
  `tail_dependence_lower`, `correlation_breakdown`,
  `tail_dependence_asymmetry`, `covariance_calibration`) — run by
  `validate(...)` on 2D flat data.
- **7 path-level metrics** (`acf_returns`, `volatility_clustering`,
  `leverage_effect`, `cross_correlation`, `drawdown_distribution`,
  `regime_conditional`, `memorization`) — run by `validate_paths(...)`
  on 3D sample paths, which also reshapes the paths and runs the 11
  flat metrics on them (so `validate_paths` produces **18** scores
  total).
- **5 calibration metrics** (`pit_uniformity`, `crps`, `coverage_50`,
  `coverage_90`, `coverage_95`) — run by `validate_calibration(...)` on
  per-observation forecast distributions paired with realized actuals.

Implementation note: **26 metrics carry weight** in `overall_score` (across the
6 lenses), drawn from a larger surface of **35 compute functions**. The balance
are **weight-0 localizers** — e.g. `signature_distance` (level-2 truncated
path-signature distance), `time_reversal_asymmetry`, `aggregational_gaussianity`,
`long_memory`, `coskewness` — computed and reported, but kept out of the weighted
aggregate because they are lower-power, so a noisy metric can flag a defect
without swinging the score. Some compute functions emit multiple outputs
(`compute_tail_dependence` → upper + lower; `compute_coverage` → three levels).

## Installation

```bash
pip install finval
```

## Quickstart

```python
import numpy as np
import finval

# 2D data: (n_samples, n_features) returns
real = np.random.randn(1000, 3) * 0.01
synthetic = np.random.randn(1000, 3) * 0.01

report = finval.validate(synthetic, real)
print(report.summary())
print(f"Overall quality: {report.overall_quality}")
print(f"Pass rate: {report.pass_rate:.0%}")

# 3D data: (n_paths, horizon, n_features) for path-level validation
real_paths = np.random.randn(100, 60, 3) * 0.01
syn_paths = np.random.randn(100, 60, 3) * 0.01

report = finval.validate_paths(syn_paths, real_paths)
print(report.summary())
```

## Metrics

As of 0.4.0 the weighted score is a **six-lens** mean (weights below); a set of
**diagnostic localizers** is computed + reported at weight 0; and **7 hard gates**
fail a model outright. `validate_full(...)` runs every lens it has inputs for.

### Marginal (15%)
- **marginal_ks** — Kolmogorov-Smirnov test on each feature's marginal
- **energy_distance** — multivariate distribution difference
- **tail_quantiles** *(gate)* — 1st/5th/95th/99th percentile comparison (robust alternative to kurtosis)
- **tail_heaviness** — tail-index fidelity

### Dependence (20%)
- **pearson_corr** / **spearman_corr** — linear / rank correlation matrix error
- **copula_distance** — Cramér-von Mises distance between empirical copulas
- **tail_dependence_upper** (λ_U) / **tail_dependence_lower** *(gate)* (λ_L) — rally / crash co-movement
- **correlation_breakdown** — stress vs calm regime correlation shift
- **tail_dependence_asymmetry** — fidelity of the crash-vs-rally gap A = λ_L − λ_U (0 by construction for any elliptical/Gaussian model, so it catches what the individual λ levels miss)
- **covariance_calibration** — variance/correlation dispersion ratio (catches a covariance right on average but wrong in spread)

### Temporal (13%)
- **acf_returns** — autocorrelation of returns (should be ~0)
- **volatility_clustering** — autocorrelation of squared returns
- **leverage_effect** — corr(r_t, |r_{t+k}|) (negative for equities)
- **cross_correlation** — contemporaneous cross-asset correlation

### Joint (10%)
- **c2st** *(gate)* — Classifier Two-Sample Test: train a classifier to tell synthetic from real; ~0.5 accuracy = indistinguishable. An omnibus that catches joint-distribution defects the per-axis metrics miss.

### Conditional (22%)
- **regime_conditional** — regime-conditional distributional fidelity: paths bucketed into low/mid/high realized-vol regimes (tercile edges from the real paths) and scored on stress-path frequency + within-regime shape.
- **conditional_sensitivity** *(gate)* — regime-stratified energy-distance ratio: does the forecast distribution actually *move* across vol/trend regimes, or is the model a calibrated **climatology** that ignores the conditioning? The pooled lenses are blind to this.
- **pit_uniformity** / **crps** / **coverage_50/90/95** *(coverage_deficit is a gate)* — per-observation forecast-distribution calibration.

### Generative (20%)
- **Naeem density / coverage** — manifold realism scored as the **delta vs a block-bootstrap replay** of the real data (a generator must beat replay to justify itself, not merely tie it). Surfaces `coverage_deficit` *(gate)* + `plausibility_deficit`.

### Hard gates (fail outright, regardless of the weighted score)
`memorization` (data-copying: synth→real vs real→real NN distances — pass the *training* set as `real`), `tail_quantiles`, `tail_dependence_lower`, `drawdown_distribution`, `conditional_sensitivity`, `c2st`, `coverage_deficit`.

## Baselines

Compare your model against simple reference generators to calibrate what
"good" means for your data:

```python
from finval.baselines import gaussian_baseline, historical_bootstrap, block_bootstrap

# Gaussian: matches mean+cov, no temporal structure
gauss = gaussian_baseline(real, n_samples=1000)

# i.i.d. bootstrap: matches joint distribution exactly, zero temporal
boot = historical_bootstrap(real, n_samples=1000)

# Block bootstrap: preserves short-range temporal structure
blocks = block_bootstrap(real, n_paths=100, path_length=60, block_size=20)

# Validate each
for name, syn in [("gaussian", gauss), ("iid", boot)]:
    r = finval.validate(syn, real)
    print(f"{name}: {r.overall_quality} ({r.overall_score:.0%})")
```

## Design principles

1. **Reliable over comprehensive.** Each metric is chosen because it's
   robust and informative, not because it's impressive.

2. **Mean over max for pairwise metrics.** Max over n(n-1)/2 feature
   pairs is dominated by sampling noise. `finval` uses mean error, which
   is harder to fool and more stable run-to-run.

3. **Lower is always better.** Every metric is normalized so that zero
   is perfect and higher is worse. No flipped signs to remember.

4. **Financial stylized facts first.** Leverage effect, vol clustering,
   fat tails, crash co-movement — these aren't optional for financial
   data.

5. **Proper scoring rules.** CRPS and PIT uniformity are proper scoring
   rules, not just rank-order checks. Your model is evaluated against
   the ground truth the statistics literature actually endorses.

## Changelog

- **0.5.0** — Opt-in **sub-window validation** + per-metric **`effective_n`**, both
  purely additive: the default (`subwindow=None`) is **byte-identical to 0.4.0**, so
  existing scores do not move. `validate_paths(...)` and `validate_full(...)` gain a
  keyword-only `subwindow=W` argument. When set (and `path_length > W`), the
  short/medium-scale path metrics are scored on **non-overlapping W-length sub-windows**
  of the long paths (`(n, H, f) → (n*(H//W), W, f)`, dropping the `H % W` remainder),
  giving many more independent samples — and so more statistical **power** — at long
  horizons (e.g. H=252) where a full-horizon path functional otherwise has only a
  handful of independent episodes. Genuinely full-horizon metrics
  (`long_memory`, `variance_term_structure`, `signature_distance`, `memorization`) stay
  on the full paths; flat metrics are unchanged (flattening is invariant to
  sub-windowing). Every `MetricResult` now carries **`effective_n`** — the number of
  independent **real** (sub)windows it was computed on (`n_real_paths` for
  full-horizon/default path metrics, `n_real_paths * (H // W)` for sub-windowed ones,
  the flattened real row count for flat metrics) — the reference sample size that bounds
  the metric's power. NOTE: `effective_n` is the count finval was *given*; if the caller
  passes **overlapping** real windows the truly-independent N is lower (independence is
  the caller's responsibility).
- **0.4.0** — Scoring reorganized into **6 weighted lenses** (marginal 0.15,
  dependence 0.20, temporal 0.13, joint 0.10, conditional 0.22, generative 0.20)
  + **7 hard gates**. New: `conditional_sensitivity` (regime-stratified — catches a
  climatological generator whose forecast ignores the conditioning), a **generative**
  lens (Naeem density/coverage vs a block-bootstrap *replay* baseline), a **joint**
  `c2st` omnibus, and **graceful degradation** (metrics flagged non-`applicable` on
  thin-data / long-horizon inputs instead of scored as failures). `validate_full(...)`
  is the all-lenses entry point.
- **0.3.0** — Two new dependence metrics: `tail_dependence_asymmetry`
  (scores whether synthetic paths reproduce the real lower-vs-upper
  tail-dependence asymmetry `A = λ_L − λ_U` that elliptical/Gaussian
  baselines get as 0) and `covariance_calibration` (scores the
  variance/correlation dispersion ratios of synthetic vs real, catching a
  covariance that is right on average but wrong in spread). Two new
  category axes: `regime_conditional` (12% — regime-conditional fidelity,
  the measured option-pricing gap that no pooled metric sees) and
  `memorization` (5% — nearest-neighbor data-copying diagnostic). Scoring
  is now de-quantized (continuous bands rather than discrete tiers).
  Category weights rebalanced: distribution 0.15→0.20, temporal
  0.20→0.15, calibration 0.30→0.15, path 0.10→0.08, with the new
  conditional/memorization axes carved out. The dependence and path
  metrics run under `validate_paths(...)`; the two dependence metrics
  also run under `validate(...)`.

## License

MIT
