Metadata-Version: 2.5
Name: acteval-insurance
Version: 0.3.0
Summary: Model-agnostic evaluation of actuarial predictive models
Project-URL: Homepage, https://github.com/aminemanai2003/acteval
Project-URL: Repository, https://github.com/aminemanai2003/acteval
Project-URL: Issues, https://github.com/aminemanai2003/acteval/issues
Author-email: Amine Manai <amine.manai@esprit.tn>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: actuarial,calibration,insurance,model-evaluation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: numpy>=1.26
Requires-Dist: pandas>=2.1
Requires-Dist: scikit-learn>=1.4
Requires-Dist: scipy>=1.11
Requires-Dist: tweedie>=0.0.9
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: matplotlib>=3.8; extra == 'dev'
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: pandas-stubs>=2.1; extra == 'dev'
Requires-Dist: pre-commit>=4.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Provides-Extra: plot
Requires-Dist: matplotlib>=3.8; extra == 'plot'
Provides-Extra: test
Requires-Dist: build>=1.2; extra == 'test'
Requires-Dist: mypy>=1.11; extra == 'test'
Requires-Dist: pandas-stubs>=2.1; extra == 'test'
Requires-Dist: pytest-cov>=5.0; extra == 'test'
Requires-Dist: pytest>=8.0; extra == 'test'
Requires-Dist: ruff>=0.9; extra == 'test'
Description-Content-Type: text/markdown

# ActEval

Evaluate actuarial predictive models beyond predictive accuracy.

ActEval is a model-agnostic Python framework for non-life insurance model
evaluation. It accepts prediction arrays instead of fitted model objects, so it
works after GLMs, scikit-learn pipelines, XGBoost, CatBoost, or neural networks.

ActEval reports accuracy, calibration, discrimination, and observed-tail
behavior separately. It does not create an arbitrary overall score or claim
that one model is universally best.

Version 0.2 also evaluates full predictive distributions using proper scoring
rules and keeps uncertainty diagnostics separate from model-quality claims.
Version 0.3 adds explicit, benchmarked financial decision diagnostics without
collapsing pricing, reserving, capital, and reinsurance into one score.

## Installation

Install the package from PyPI:

```bash
python -m pip install acteval-insurance
```

For plots:

```bash
python -m pip install "acteval-insurance[plot]"
```

The distribution is named `acteval-insurance` because `acteval` is occupied by
an unrelated project on PyPI. The import name remains `acteval`.

## Quick start

```python
import acteval as ae

result = ae.evaluate(
    y_true=[0.0, 0.4, 1.0, 2.0, 4.0, 7.0],
    y_pred=[0.1, 0.5, 0.9, 1.8, 3.6, 6.4],
    exposure=[1.0, 0.5, 1.2, 0.8, 1.5, 2.0],
    task="claim_frequency",
    metrics=["rmse", "poisson_deviance", "ae_ratio", "normalized_gini"],
)

print(result.summary())
```

Task defaults provide a broader report, including 95% observed-tail metrics.

## Model comparison

```python
comparison = ae.compare(
    y_true=y,
    predictions={
        "GLM": glm_predictions,
        "CatBoost": catboost_predictions,
        "XGBoost": xgb_predictions,
    },
    exposure=exposure,
    task="claim_frequency",
)

print(comparison.to_dataframe())
print(comparison.rank(metric="poisson_deviance"))
```

`rank()` uses a metric's documented direction. Target metrics such as A/E are
ranked by distance from 1. Rankings remain metric-specific.

## Accuracy can disagree with tail calibration

The example below deliberately creates two models:

- Model A makes moderate errors on many ordinary risks but predicts large
  observed outcomes accurately.
- Model B improves ordinary-risk predictions and overall RMSE while
  underpredicting the observed tail.

```python
import numpy as np
import acteval as ae

y = np.r_[np.tile([0.5, 1.0, 1.5, 1.0, 0.5], 19), np.repeat(10.0, 5)]
model_a = np.r_[y[:95] + 0.5, np.repeat(10.0, 5)]
model_b = np.r_[y[:95], np.repeat(9.0, 5)]

tradeoff = ae.compare(
    y,
    {"Model A": model_a, "Model B": model_b},
    task="claim_frequency",
    metrics=["rmse", "poisson_deviance", "tail_ae_95"],
)
print(tradeoff.to_dataframe())
```

Model B has lower overall RMSE and deviance, while Model A has tail A/E equal
to 1. The appropriate choice depends on the actuarial objective.

## Input and exposure contract

`y_true` and `y_pred` must be finite, one-dimensional, nonnegative arrays on
the same scale.

- For claim frequency, use frequency rates for both arrays and provide policy
  exposure as `exposure`.
- For pure premium, use pure-premium rates for both arrays and provide exposure
  when portfolio-volume weighting is desired.
- For severity, use claim severities. Exposure is optional and usually
  unnecessary; claim-level `sample_weight` is normally more meaningful.
- If both are supplied, effective weight is `sample_weight * exposure`.

ActEval does not silently convert raw claim counts into rates.

## Parameterized metrics

Use `MetricSpec` whenever a parameter should be explicit and reproducible:

```python
result = ae.evaluate(
    y,
    predictions,
    task="pure_premium",
    metrics=[
        ae.MetricSpec("tweedie_deviance", {"power": 1.7}),
        ae.MetricSpec("tail_mae", {"quantile": 0.99}, label="tail_mae_99"),
    ],
)
```

Tail aliases such as `tail_mae_95`, `tail_rmse_99`, and `tail_ae_95` are also
accepted. Parameter values are retained in result metadata.

## Calibration, discrimination, and tail diagnostics

```python
calibration = ae.calibration_by_quantile(y, predictions, n_bins=10)
lift = ae.lift_by_quantile(y, predictions, n_bins=10)

print(calibration.to_dataframe())
print(lift.to_dataframe())

ae.plot_calibration(y, predictions)
ae.plot_lift(y, predictions)
ae.plot_residuals(y, predictions)
ae.plot_tail_diagnostics(y, predictions, quantile=0.95)
```

## Predictive distributions

Built-in vectorized adapters provide one predictive distribution per
observation:

- `PoissonDistribution(mu)`;
- `NegativeBinomialDistribution(mean, dispersion)`;
- `GammaDistribution(mean, shape)`;
- `LognormalDistribution(meanlog, sdlog)`;
- `EmpiricalDistribution(samples)`;
- `TweedieDistribution(mean, power, dispersion)` for compound
  Poisson-Gamma `1 < power < 2`.

```python
poisson = ae.PoissonDistribution(mu=poisson_means)
negative_binomial = ae.NegativeBinomialDistribution(
    mean=nb_means,
    dispersion=nb_dispersion,
)

distribution_comparison = ae.compare_distributions(
    y_true=claim_counts,
    distributions={
        "Poisson": poisson,
        "Negative Binomial": negative_binomial,
    },
    exposure=exposure,
    task="claim_frequency",
    metrics=[
        ae.MetricSpec("crps", {"n_samples": 5000, "random_state": 42}),
        "log_score",
        ae.MetricSpec("brier_score", {"threshold": 0}),
        ae.MetricSpec("interval_score", {"coverage": 0.9}),
    ],
)

print(distribution_comparison.to_dataframe())
```

Samples have shape `(n_samples, n_observations)`. Scalar quantiles have shape
`(n_observations,)`; vector quantiles have shape
`(n_quantiles, n_observations)`. CRPS randomness is explicitly seeded and
recorded in result metadata.

Tweedie sampling uses the exact compound representation. CDF and log-density
evaluation use a numerical series implementation; quantiles use deterministic
Monte Carlo. Entropy is a seeded Monte Carlo estimate of `-E[log_prob(X)]` and
is only comparable under the same mixed distribution measure. Empirical draws
are treated as a discrete distribution: repeated values determine probability
mass, and unseen values have log probability `-inf`.

## Decision-aware evaluation

Decision functions always expose their financial loss and benchmark. Regret is
`model financial loss - benchmark financial loss` in the loss function's unit.
It may be negative when the model decision outperforms the benchmark. Relative
regret is omitted when benchmark loss is zero.

```python
premiums = ae.premium_from_distribution(
    severity_distribution,
    profit_loading=0.08,
    expense_ratio=0.20,
)

pricing = ae.pricing_regret(
    y_true=realized_loss,
    premium=premiums,
    benchmark_premium=current_tariff,
    underpricing_cost=2.0,
    overpricing_cost=1.0,
    benchmark_name="current tariff",
)

loss_ratio = ae.loss_ratio_impact(
    realized_loss,
    premiums,
    target_loss_ratio=0.70,
)

reserve = ae.reserve_shortfall(realized_loss, held_reserve)
capital = ae.capital_shortfall(realized_loss, available_capital)
```

Stop-loss reinsurance selection compares quoted options under one explicit
rule: premium plus expected retained aggregate loss plus a user-selected cost
of VaR or expected-shortfall capital.

```python
options = [
    ae.ReinsuranceOption("No cover", retention=1_000_000, premium=0),
    ae.ReinsuranceOption("100k retention", retention=100_000, premium=25_000),
]

selection = ae.select_reinsurance_option(
    aggregate_loss_distribution,
    options,
    risk_measure="expected_shortfall",
    risk_quantile=0.995,
    capital_cost_rate=0.10,
    random_state=42,
)

realized = ae.reinsurance_decision_regret(
    aggregate_loss=realized_annual_losses,
    selected=selection.selected,
    benchmark=options[0],
)
```

For reinsurance selection, each sampled row is a scenario and columns are
summed into portfolio aggregate loss. Dependence must therefore already be
represented by the supplied distribution's joint samples. Built-in parametric
adapters sample observation columns independently; use `EmpiricalDistribution`
with joint scenario draws when portfolio dependence matters.

## Supported MVP metrics

| Metric | Category | Interpretation |
|---|---|---|
| `mae` | accuracy | Lower is better |
| `rmse` | accuracy | Lower is better |
| `poisson_deviance` | accuracy | Lower; frequency only |
| `gamma_deviance` | accuracy | Lower; positive severity only |
| `tweedie_deviance` | accuracy | Lower; explicit power required |
| `ae_ratio` | calibration | Target is 1 |
| `weighted_calibration_error` | calibration | Lower is better |
| `gini` | discrimination | Higher is better |
| `normalized_gini` | discrimination | Perfect ordering is 1 |
| `lift` | discrimination | Higher means stronger top-group concentration |
| `tail_mae` | tail risk | Lower is better |
| `tail_rmse` | tail risk | Lower is better |
| `tail_ae_ratio` | tail risk | Target is 1 |
| `crps` | probabilistic | Lower is better |
| `log_score` | probabilistic | Lower is better |
| `brier_score` | probabilistic | Lower is better for an explicit event |
| `quantile_score` | probabilistic | Lower is better |
| `interval_score` | probabilistic | Lower is better |
| `interval_coverage` | uncertainty | Compare with requested coverage |
| `interval_width` | uncertainty | Sharpness; no universal direction |
| `predictive_variance` | uncertainty | No universal direction |
| `predictive_entropy` | uncertainty | No universal direction |

Use `ae.list_metrics()` for machine-readable registry metadata. Exact formulas
and limitations are in [the metric reference](docs/metric-reference.md).

## Development

```bash
git clone https://github.com/aminemanai2003/acteval.git
cd acteval
python -m venv .venv
python -m pip install -e ".[dev]"
ruff check .
mypy src/acteval
pytest
python -m build
```

See [CONTRIBUTING.md](CONTRIBUTING.md) and the
[implementation audit](docs/plan-audit.md).

## Implemented releases

- v0.1: point-prediction accuracy, calibration, discrimination, tail
  diagnostics, comparisons, and plotting.
- v0.2: predictive-distribution scores and uncertainty diagnostics.
- v0.3: explicit benchmarked pricing, loss-ratio, reserve, capital, and
  reinsurance financial consequences.

The original v0.1-v0.3 implementation plan is complete. Remaining work is
release operations and future scope, not missing behavior from that plan. See
the [completion audit](docs/plan-audit.md) for boundaries and evidence.

## License

Apache-2.0.
