Metadata-Version: 2.4
Name: insurance-scmoe
Version: 0.1.0
Summary: Spatially Clustered Mixture of Experts for joint frequency-severity insurance pricing
Project-URL: Homepage, https://github.com/burning-cost/insurance-scmoe
Project-URL: Repository, https://github.com/burning-cost/insurance-scmoe
Project-URL: Issues, https://github.com/burning-cost/insurance-scmoe/issues
Author-email: Burning Cost <pricing.frontier@gmail.com>
License: MIT
Keywords: actuarial,frequency-severity,insurance,mixture-of-experts,pricing,spatial
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Office/Business :: Financial
Classifier: Topic :: Scientific/Engineering :: Mathematics
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scipy>=1.11
Provides-Extra: dev
Requires-Dist: numpy>=1.24; extra == 'dev'
Requires-Dist: pandas>=2.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: scipy>=1.11; extra == 'dev'
Provides-Extra: plot
Requires-Dist: matplotlib>=3.7; extra == 'plot'
Provides-Extra: spatial
Requires-Dist: geopandas>=0.14; extra == 'spatial'
Requires-Dist: libpysal>=4.9; extra == 'spatial'
Description-Content-Type: text/markdown

# insurance-scmoe

**Spatially Clustered Mixture of Experts for joint frequency-severity insurance pricing.**

## The problem

Standard insurance pricing models treat frequency and severity as separate GLMs. This misses the joint structure: high-frequency policyholders are often also high-severity. On top of that, a typical two-GLM approach applies a territorial factor as a post-hoc correction — either a manual zone scheme or a spatial GLM — but this is independent of the risk segmentation.

SC-MoE (Spatially Clustered Mixture of Experts, NAAJ 2025) solves both problems simultaneously. It discovers K latent risk types — low-frequency/low-severity, medium, high — and enforces geographic continuity so that nearby postcodes are encouraged to belong to the same risk class. The territory structure emerges from the model rather than being imposed on top of it.

## How it works

Each latent class k has:
- **Poisson(lambda_k)** claim frequency
- **Gamma(alpha_k, beta_k)** severity per claim

A gating network assigns each policyholder a probability of belonging to each class:

```
pi_k(x_i) = exp(alpha_k' x_i) / sum_j exp(alpha_j' x_j)
```

The spatial penalty `(lambda/2) * trace(alpha_area' L alpha_area)` applied to the graph Laplacian L of the geographic adjacency graph encourages neighbouring areas to have similar class memberships.

Estimation uses MM-ADMM: a quadratic MM surrogate (Böhning 1992) handles the non-linear logistic objective, and ADMM (Boyd et al. 2010) solves the penalised quadratic with the sparse Laplacian structure.

## Installation

```bash
pip install insurance-scmoe
```

For building spatial graphs from shapefiles (requires geopandas and libpysal):

```bash
pip install insurance-scmoe[spatial]
```

## Quick start

```python
from insurance_scmoe import SCMoE, simulate_scmoe

# Simulate 2,000 policyholders across 100 postcode sectors (10x10 grid)
X, y_freq, y_sev, area_ids, graph, truth = simulate_scmoe(
    n=2000, K=3, n_areas=100, n_rows=10, n_cols=10, seed=42
)

# Fit the model
model = SCMoE(n_components=3, lam=1.0, max_iter=100, random_state=42)
model.fit(X, y_freq, y_sev, graph, area_ids)

# Pure premium predictions
pp = model.predict_pure_premium(X, area_ids)
print(f"Mean pure premium: {pp.mean():.4f}")

# Fitted expert parameters (ordered by ascending frequency rate)
print(f"Poisson rates lambda_k:  {model.expert_.lambda_}")
print(f"Gamma shapes alpha_k:    {model.expert_.alpha_}")
print(f"Gamma rates beta_k:      {model.expert_.beta_}")

# Class membership probabilities
pi = model.predict_proba(X, area_ids)  # shape (n, K)
```

## Building from a real spatial dataset

If you have a GeoDataFrame of postcode sectors:

```python
import geopandas as gpd
from insurance_scmoe import SpatialGraph

geo = gpd.read_file("postcode_sectors.shp")
graph = SpatialGraph.from_geodataframe(geo, id_col="sector_code", method="queen")
```

Or from a pre-computed adjacency edge list:

```python
graph = SpatialGraph.from_adjacency_csv("adjacency.csv")
# CSV must have columns: area_i, area_j (zero-based integer indices)
```

## Model selection

```python
from insurance_scmoe import ModelSelector

sel = ModelSelector(
    k_range=(2, 6),
    lam_grid=[0.0, 0.5, 1.0, 2.0, 5.0],
    max_iter=100,
    random_state=0,
    verbose=True,
)
sel.fit(X, y_freq, y_sev, graph, area_ids)

print(f"Best K: {sel.best_k_},  Best lambda: {sel.best_lam_}")
print(sel.summary())

best_model = sel.best_model_
```

## API reference

### `SCMoE(n_components, lam, rho, max_iter, tol, admm_max_iter, random_state, verbose)`

Main model class.

| Method | Description |
|--------|-------------|
| `fit(X, y_freq, y_sev, graph, area_ids)` | Fit via MM-ADMM ECM |
| `predict_proba(X, area_ids)` | Class membership probabilities, shape (n, K) |
| `predict_frequency(X, area_ids)` | E[N \| x, area], shape (n,) |
| `predict_severity(X, area_ids)` | E[X \| x, area], shape (n,) |
| `predict_pure_premium(X, area_ids)` | E[N*S \| x, area], shape (n,) |
| `log_likelihood(X, y_freq, y_sev, area_ids)` | Observed-data log-likelihood |
| `bic(...)` / `aic(...)` | Information criteria |

### `SpatialGraph`

| Method | Description |
|--------|-------------|
| `from_adjacency_matrix(W)` | From dense or sparse binary matrix |
| `from_adjacency_csv(path)` | From edge-list CSV |
| `from_geodataframe(geo_df, ...)` | From GeoDataFrame (requires `[spatial]`) |
| `adjacency()` | Returns sparse W |
| `laplacian()` | Returns sparse L = D - W |

### `ModelSelector(k_range, lam_grid, ...)`

Grid search over K and lambda by BIC.  `sel.fit(...)` then `sel.best_model_`, `sel.summary()`.

### `simulate_scmoe(n, K, n_areas, n_rows, n_cols, lam_spatial, seed, ...)`

Generates synthetic portfolio data from a ground-truth K-class SC-MoE. Returns `(X, y_freq, y_sev, area_ids, graph, truth)`.

## Design notes

**Why not PyTorch?** All operations are closed-form ADMM linear systems and weighted MLEs. For K up to 8 components and n up to 500k policyholders, numpy/scipy is faster and has no GPU dependency. The Laplacian linear system is sparse and well-conditioned — `spsolve` handles it cleanly.

**Why not Bayesian (PyMC/Stan)?** SC-MoE is frequentist penalised likelihood. The spatial term is regularisation, not a prior. MCMC would be 10-100x slower for no gain given the MM-ADMM algorithm's convergence properties.

**Gamma parameterisation**: `f(x; alpha, beta) = beta^alpha / Gamma(alpha) * x^(alpha-1) * exp(-beta*x)`. Mean = alpha/beta, Var = alpha/beta^2. Newton-Raphson on the profile log-likelihood gives fast, stable shape estimation.

**Label switching**: Components are re-ordered by ascending lambda_k after fitting. This is not perfect — if two classes have identical rates, order is arbitrary — but it gives reproducible output for most practical cases.

**Gating architecture**: The ADMM operates at the area level (one alpha vector per postcode sector per class), averaging down to policy level for predictions. This means the spatial penalty acts on the area-level assignment, not on individual policies.

## Reference

NAAJ 2025, DOI: [10.1080/10920277.2025.2567283](https://doi.org/10.1080/10920277.2025.2567283).

LRMoE: Fung, Badescu, Lin (2019). [GitHub](https://github.com/UofTActuarial/LRMoE).

## Related packages

- [`insurance-spatial`](https://github.com/burning-cost/insurance-spatial): BYM2 spatial random effects in a GLM — territorial smoothing within a single risk model.
- [`insurance-nested-glm`](https://github.com/burning-cost/insurance-nested-glm): Neural embeddings + contiguity-constrained clustering for territory factor construction.
- [`insurance-glm-cluster`](https://github.com/burning-cost/insurance-glm-cluster): Fused lasso clustering of factor levels within a GLM.

SC-MoE uniquely combines latent risk type discovery, joint frequency-severity modelling, and geographic continuity enforcement in a single penalised mixture model.

## Licence

MIT
