Metadata-Version: 2.5
Name: ambric
Version: 0.0.8
Summary: Augmented Mixed-frequency Bayesian Regional Inference with Constraints
Author-email: Arthur Turrell <anon@anon.gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.12
Requires-Dist: arviz>=0.23.1
Requires-Dist: graphviz>=0.21
Requires-Dist: loguru>=0.7.3
Requires-Dist: matplotlib>=3.10.8
Requires-Dist: nbstripout>=0.9.0
Requires-Dist: pandas>=2.3.3
Requires-Dist: pre-commit-hooks>=6.0.0
Requires-Dist: pyarrow>=23.0.0
Requires-Dist: pydemetra
Requires-Dist: pydoclint>=0.8.3
Requires-Dist: pymc>=5.25.1
Requires-Dist: ruff>=0.15.2
Requires-Dist: scikit-learn>=1.7.2
Requires-Dist: ty>=0.0.17
Requires-Dist: xgboost>=3.2.0
Description-Content-Type: text/markdown


# AMBRIC

[![Status](https://img.shields.io/pypi/status/ambric.svg)](https://pypi.org/project/ambric/)
[![Python Version](https://img.shields.io/pypi/pyversions/ambric)](https://pypi.org/project/ambric)
[![Read the documentation at https://aeturrell.github.io/ambric/](https://img.shields.io/badge/docs-passing-brightgreen)](https://aeturrell.github.io/ambric/)
[![Tests](https://github.com/aeturrell/ambric/workflows/Tests/badge.svg)](https://github.com/aeturrell/ambric/actions?workflow=Tests)
[![Codecov](https://codecov.io/gh/aeturrell/ambric/branch/main/graph/badge.svg)](https://codecov.io/gh/aeturrell/ambric)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Source](https://img.shields.io/badge/source%20code-github-lightgrey?style=for-the-badge)](https://github.com/aeturrell/ambric)


![](https://img.shields.io/badge/mac%20os-000000?style=for-the-badge&logo=macos&logoColor=F0F0F0)
![Linux](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black)

## **Augmented Mixed-frequency Bayesian Regional Inference with Constraints**

AMBRIC is a Bayesian state-space model for estimating latent regional growth in a given variable from sparse and temporally misaligned observations. It combines factor analysis, autoregressive dynamics at both the factor and regional level, and observation constraints from aggregate (country-wide) and annual regional data.

**When would I need this model?**

- When you have a highly lagged annual release of, say, economic growth at the regional level but quarterly growth at the national level with minimal lag.
- When you want to nowcast regional growth—both quarterly and annual—from published national quarterly growth.

It was originally designed to nowcast UK gross value-added (GVA) and gross household disposable income (GHDI) for regions following the publication of the equivalent quarterly national growth rate.

The core model features are:

- Hierarchical structure: Factor and macro loadings are partially pooled across regions, enabling information sharing while allowing regional heterogeneity.

- Dual AR(1) dynamics: Persistence is modelled at both the factor level (common shocks) and regional level (idiosyncratic dynamics).

- Robust observations: Student-t likelihoods for UK and annual constraints provide robustness to outliers.

- Mixed frequency: Regional growth is observed annually, while latent quarterly growth is estimated for every region and quarter.

- Published quarterly regional data (optional): Where quarterly regional growth is published, for some regions or some quarters, the model's quarterly growth for those cells is fixed to the published values, which then inform the other regions and quarters through the constraints and dynamics.

- Cross-sectional constraint: National and regional growth rates are constrained to be consistent.

- Temporal constraints: Annual regional growth rates are constrained to be consistent with latent quarterly growth rates.

- Soft implementation of constraints: Regional growth rates are not always *exactly* consistent with the national growth rate, for measurement, extra-regional, and rounding reasons. Regional weights, $w$, are positive and sum to one. Their prior is centred on shares you supply (for example each region's share of national output) or, failing that, on shares estimated from the growth data, and the model can move away from that centre.

- Dimensional reduction: With a rich set of data on each region, it would be extremely difficult to solve the model due to the increasing number of parameters. Factor analysis is applied to the panel of regional indicators first to reduce the dimensionality to a manageable level.

- XGBoost bridge signal: An XGBoost model is trained on annually-aggregated regional indicators and macro variables to predict annual regional growth. These annual predictions are then disaggregated to quarterly frequency via a MIDAS bridge equation, producing a quarterly signal $s_{t,r}$ that enters the state-space model with a hierarchical loading $\delta_r$. Predictions for years with a published outturn are made out-of-fold (each year is predicted by a model trained on the other years), so the bridge equation is not fitted to XGBoost's in-sample fit. This allows the model to incorporate non-linear relationships captured by XGBoost while retaining the Bayesian uncertainty quantification of the state-space framework.

AMBRIC is currently supported on macOS and Linux only; Windows users should run AMBRIC under WSL.

**Units.** Growth rates must be supplied as decimals: `0.005` for 0.5%. The model's priors are sized for decimal growth rates, and the posterior (`amb.trace`) and `amb.populate_results()` are in the same units. The summary tables `point_estimates_q_on_q()` and `point_estimates_q_on_4q()` report percentages instead (`0.5` for 0.5%), rounded to two decimal places. Macro series and regional indicators can be in any units; they are standardised internally.

## Model Details

### Variable Definitions

Let $Y$ be the variable of interest in levels.

- $t = 1, \ldots, T$ at quarterly frequency.
- $r = 1, \ldots, R$ denotes the $R$ regions of the nation.
- $Y^\text{UK}_t$ is the level in quarter $t$ for the whole nation (observed).
- $y^\text{UK}_t = \log(Y^\text{UK}_t) - \log(Y^\text{UK}_{t-1})$ is the quarterly growth rate for the whole nation (observed).
- $Y_{t, r}$ is the level for region $r$ in quarter $t$ (never observed)
- $Y_{t, r}^A = Y_{t, r} + Y_{t-1, r} + Y_{t-2, r} + Y_{t-3, r}$ is the annual level for region $r$. Observed for Q4 only, and with a lag.
- $y_{t, r}^A = \log(Y_{t, r}^A) - \log(Y^{A}_{t-4,r})$ is the annual growth in region $r$; observed Q4 only. $y^A_t = (y^{A}_{t,1}, \ldots, y_{t, R}^A)'$ is the vector of these.
- $y_{t, r} = \log(Y_{t, r}) - \log(Y_{t-1, r})$ is the quarterly growth rate in region $r$ (unobserved, unless published quarterly regional data are supplied for that region and quarter). $y^Q_t = (y_{t,1}, \ldots, y_{t, R})'$ is the vector of these.
- $\boldsymbol{Z}_t$ is a panel of regional indicators, with elements $Z_{j,r,t}$.
- $s_{t,r}$ is a quarterly bridge signal for region $r$, derived from XGBoost annual predictions disaggregated via a MIDAS bridge equation. $\mathbf{s}_t = (s_{t,1}, \ldots, s_{t,R})'$ is the vector of these.


The key dimensions of the problem are:

| Symbol | Description |
|--------|-------------|
| $T$ | Number of time periods (quarters) |
| $R$ | Number of regions |
| $J$ | Number of indicators per region in the regional data |
| $K$ | Number of latent factors drawn from the regional panel data |
| $M$ | Number of national macroeconomic covariates |


### Model Equations

The core equation of AMBRIC is:

$$
\begin{equation}
\mathbf{y}_t = \boldsymbol{\Phi}_r \mathbf{y}_{t-1} + (\mathbf{I} - \boldsymbol{\Phi}_r)(\boldsymbol{\alpha} + \boldsymbol{\Lambda} \mathbf{F}_t + \boldsymbol{\Gamma} \mathbf{X}_t + \boldsymbol{\delta} \odot \mathbf{s}_t) + \boldsymbol{\epsilon}_t
\end{equation}
$$

where $\mathbf{y}_t$ is the vector of regional quarterly growth rates, $\mathbf{F}_t$ is a vector of factors based on a regional panel of indicators, $\mathbf{X}_t$ is a vector of national statistics (each standardised to zero mean and unit variance inside the model), $\boldsymbol{\alpha} = (\alpha_1, \ldots, \alpha_R)'$ are regional mean growth rates, and $\mathbf{s}_t$ is a quarterly bridge signal derived from XGBoost predictions via a MIDAS bridge equation. The auto-regressive term in equation (1) is a diagonal matrix, $\boldsymbol{\Phi}_r = \text{diag}(\boldsymbol{\phi}_r)$. The factors $\mathbf{F}_t$ themselves follow an AR(1) process governed by $\boldsymbol{\Phi}_f = \text{diag}(\boldsymbol{\phi}_f)$ (see below). $\boldsymbol{\Lambda}$ are *factor loadings*, $\boldsymbol{\Gamma}$ are *macro loadings*, and $\boldsymbol{\delta} = (\delta_1, \ldots, \delta_R)'$ are *bridge signal loadings* with $\odot$ denoting element-wise multiplication (the Hadamard product.)

#### Observed vs estimated data

We observe $\boldsymbol{Z}_t$, $y_t^\text{UK}$, and, with a significant lag, $y_{t, r}^A$ for $t\mod 4 \equiv 0$ (ie 4th quarter only.) Optionally, we also observe $y_{t,r}$ for some regions and quarters (see [Published quarterly regional growth](#published-quarterly-regional-growth)).

The model estimates many parameters, but those that are "outputs" are $y_{t,r}$ and $y_{t,r}^A$, the latter only for $t\mod 4 \neq 0$.

#### Cross-sectional and temporal constraints

$\mathbf{y}_t$ are latent variables; we only observe the left-hand sides of the following assumed relationships:

$$
y_t^{\text{UK}} = \mathbf{w}^\top \mathbf{y}_t \quad \text{ and }\quad \boldsymbol{y}_t^{A} = \boldsymbol{\Omega}(L) \mathbf{y}_t
$$

although the latter with a lag. The weights $\mathbf{w}$ are positive and sum to one. The annual relationship needs six earlier quarters, so an annual observation in the first six quarters of the sample cannot be matched to a full sum and is left out of the annual constraint. In this, the first equation is the cross-sectional constraint and the latter is the temporal constraint, which uses a lag polynomial $\boldsymbol{\Omega}(L) = \sum_{j=0}^{6} \Omega_j L^j$. The cross-sectional constraint ensures that quarterly regional growth is consistent with quarterly national growth, while the temporal constraint ensures that quarterly regional growth is consistent with published annual regional growth.

Because these are soft constraints, they enter the model as:

$$
y_t^{\text{UK}} \sim \mathcal{T}(\nu_{\text{UK}}, \mathbf{w}^\top \mathbf{y}_t, \sigma_{\text{UK}}), \quad \mathbf{y}_t^{\text{A}} \sim \mathcal{T}\left(\nu_{\text{A}}, \sum_{j=0}^{6} \Omega_j \mathbf{y}_{t-j}, \boldsymbol{\sigma}_{\text{A}}\right)
$$

where $\mathcal{T}$ is the Student's T-distribution.

#### Auto-regressive behaviours

The (Bayesian) auto-regressive behaviour of the quarterly regional growth rates is given by

$$y_{t,r} \mid y_{t-1,r} \sim \mathcal{N}\left(\phi_r \, y_{t-1,r} + (1 - \phi_r) \, \mu_{t,r}^{\text{exog}}, \, \sigma_{\varepsilon,r}\right)$$

where

$$
\mu_{t,r}^{\text{exog}} = \alpha_r + \boldsymbol{\Lambda}_r \mathbf{F}_t + \boldsymbol{\Gamma}_r \mathbf{X}_t + \delta_r \, s_{t,r}
$$

Note that the $(1 - \phi_r)$ scaling ensures that $\mathbb{E}[y_{t,r}] = \mu_{t,r}^{\text{exog}}$.

To reduce the dimensionality of $\boldsymbol{Z}_t$, the panel of regional indicators, we use *factor analysis*, which finds an $\boldsymbol{F}_t^{\text{obs}}$ with dimension $K$ such that

$$
\mathbf{Z}_t = \mathbf{W} \mathbf{F}_t^{\text{obs}} + \boldsymbol{\mu} + \boldsymbol{\varepsilon}_t, \quad \mathbf{F}_t^{\text{obs}} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}_K), \quad \boldsymbol{\varepsilon}_t \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Psi}), \quad \text{Cov}(\mathbf{Z}_t) = \mathbf{W} \mathbf{W}^\top + \boldsymbol{\Psi}
$$

where $\mathbf{W}$ is the matrix of factor loadings and $\boldsymbol{\Psi}$ is a diagonal matrix of indicator-specific noise variances. These extracted factors are then treated as noisy observations of an underlying autoregressive latent factor process $\mathbf{F}_t$ such that

$$
\mathbf{F}_t = \text{diag}(\boldsymbol{\phi}_f) \mathbf{F}_{t-1} + \boldsymbol{\eta}_t, \quad \mathbf{F}_t^{\text{obs}} = \mathbf{F}_t + \boldsymbol{\epsilon}_t^f
$$

#### Published quarterly regional growth

Some regions publish quarterly growth (for example Scotland's quarterly GDP), and that is more timely than the annual regional release. If these values are supplied through `region_q_on_q_measure`, then for every region $r$ and quarter $t$ with a published value $y^{\text{pub}}_{t,r}$,

$$
y_{t,r} = y^{\text{pub}}_{t,r}.
$$

This is a hard constraint, not a noisy observation. The published values still enter the national constraint, the annual constraint and the regional AR(1) dynamics, so they inform the estimates for the other quarters and regions. Coverage can be partial: any region and quarter without a published value is estimated as usual.

#### Bayesian priors

Scales below are in decimal growth units (0.01 is one percentage point). In the code they live in `ambric.utilities.PRIOR_SCALES`; each growth-unit parameter is sampled on a unit scale and multiplied by its prior scale, which leaves the priors below unchanged but helps the optimiser.

##### Regional panel and factors

$$\Lambda_\mu \sim \mathcal{N}(0, 0.01) \quad [K]$$

$$\Lambda_\sigma \sim \text{HalfNormal}(0.004) \quad [K]$$

$$\Lambda_{r,k} \sim \mathcal{N}(\Lambda_{\mu,k}, \Lambda_{\sigma,k}) \quad [R \times K]$$

$$\sigma_{\text{exog}} \sim \text{HalfNormal}(0.2) \quad [K]$$

$$F_{t,k}^{\text{obs}} \sim \mathcal{N}(F_{t,k}, \sigma_{\text{exog},k})$$

$$\phi_f \sim \mathcal{N}(0.7, 0.1) \quad [K]$$

$$\sigma_f \sim \text{HalfNormal}(0.1) \quad [K]$$

##### Macro indicators

$\Gamma$ is the effect on growth of a one standard deviation move in each macro series.

$$\Gamma_\mu \sim \mathcal{N}(0, 0.01) \quad [M]$$

$$\Gamma_\sigma \sim \text{HalfNormal}(0.003) \quad [M]$$

$$\Gamma_{r,m} \sim \mathcal{N}(\Gamma_{\mu,m}, \Gamma_{\sigma,m}) \quad [R \times M]$$

##### Bridge signal loadings

$$\delta_\mu \sim \mathcal{N}(0, 0.3)$$

$$\delta_\sigma \sim \text{HalfNormal}(0.15)$$

$$\delta_r \sim \mathcal{N}(\delta_\mu, \delta_\sigma) \quad [R]$$

##### Weights and degrees of freedom

$$w \sim \text{Dirichlet}(\kappa \, \bar{w}) \quad [R]$$

where $\bar{w}$ is the prior centre: the shares passed as `region_weights`, normalised to sum to one, or otherwise shares estimated from the data by regressing annualised national growth on regional annual growth (non-negative, summing to one, shrunk towards $1/R$). Because a Dirichlet needs positive parameters, any share below $0.1/R$ is raised to $0.1/R$ and the shares renormalised. $\kappa$ sets how tightly $w$ is held to $\bar{w}$: the prior standard deviation of $w_r$ is $\sqrt{\bar{w}_r (1 - \bar{w}_r) / (\kappa + 1)}$. By default $\kappa = 500$ for supplied shares and $\kappa = 50$ for estimated ones; set it with `region_weights_concentration`. The data-based estimate is imprecise, because regional growth rates move together and annual observations are few, so supply known shares where you can.

$$\nu_{\text{UK}} \sim \text{Gamma}(6, 1)$$

$$\nu_{\text{A}} \sim \text{Gamma}(3, 0.5)$$

##### Growth

$$\alpha_r \sim \mathcal{N}(0, 0.01) \quad [R]$$

$$\sigma_\varepsilon \sim \text{HalfNormal}(0.005) \quad [R]$$

$$\sigma_{\text{A}} \sim \text{HalfNormal}(0.01) \quad [R]$$

$$\phi_r \sim \mathcal{N}(0.5, 0.15) \quad [R]$$

$$\sigma_{\text{UK}} \sim \text{HalfNormal}(0.001)$$

The latent $y_{t,r}$ also carry a diffuse $\mathcal{N}(0, 0.05)$ prior.

#### Parameters

| Parameter | Shape |
| ----------:|:---------|
| $\Lambda_\mu$ | $K$ |
| $\Lambda_\sigma$ | $K$ |
| $\Lambda$ | $R \times K$ |
| $\Gamma_\mu$ | $M$ |
| $\Gamma_\sigma$ | $M$ |
| $\Gamma$ | $R \times M$ |
| $\delta_\mu$ | $1$ |
| $\delta_\sigma$ | $1$ |
| $\delta_r$ | $R$ |
| $\alpha_r$ | $R$ |
| $\phi_f$ | $K$ |
| $\sigma_f$ | $K$ |
| $F$ | $T \times K$ |
| $\sigma_{\text{exog}}$ | $K$ |
| $\phi_r$ | $R$ |
| $\sigma_\varepsilon$ | $R$ |
| $y_r$ | $T \times R$ |
| $w$ | $R$ |
| $\sigma_{\text{UK}}$ | $1$ |
| $\sigma_{\text{A}}$ | $R$ |
| $\nu_{\text{UK}}$ | $1$ |
| $\nu_{\text{A}}$ | $1$ |

Total: $5K + 2M + RK + RM + TK + TR + 6R + 5$ (of which $w$, summing to one, has $R - 1$ free). Each published quarterly regional value fixes one element of $y_r$, removing one free parameter.

### Model solution

Missing values in the regional indicators are first filled in by iterative imputation (scikit-learn's `IterativeImputer` with a Bayesian ridge estimator). The imputed panel is used for the factor analysis and the bridge equation. XGBoost uses the raw indicators, since it handles missing values itself.

Prior to estimating the Bayesian model, an XGBoost model is trained on annually-aggregated regional indicators and macro variables to predict annual regional growth. For each year with a published outturn, the prediction comes from a model trained on all the other years, so the predictions passed on are out-of-fold. To keep their regularisation independent of units, XGBoost is fitted to a standardised target and the bridge regression to standardised features and target.

The annual predictions are disaggregated to quarterly frequency using a MIDAS bridge equation. This is a ridge regression, pooled across regions, of published annual regional growth on the XGBoost prediction and on the MIDAS-weighted quarters of national growth and of each regional indicator. The quarterly weights follow an exponential Almon polynomial chosen to maximise the bridge's fit, or are equal if `bridge_use_almon=False`. Applying the fitted coefficients quarter by quarter gives the bridge signal $\mathbf{s}_t$.

The Bayesian state-space model is then estimated using [PyMC](https://www.pymc.io/welcome.html) and [pytensor](https://pytensor.readthedocs.io/en/latest/) via ADVI (Automatic Differentiation Variational Inference).

During fitting, the ADVI loss and its improvement are logged at regular intervals. The run stops early if the parameters stop changing, which counts as converged. Otherwise, a warning is logged after fitting if the loss was still falling at the end of the run, in which case more iterations are needed. `amb.plot_advi_convergence()` shows the loss path.

### Evaluating nowcasts out of sample

`ambric.run_out_of_sample_exercise` replays history: at each step it hides the annual regional data that would not yet have been published, fits the model, and records the nowcasts alongside the outturns that were later published. The [usage guide](https://aeturrell.github.io/ambric/) shows how to summarise the results.

### Checking the model against simulated data

`ambric.benchmark.run_recovery_benchmark` fits the model to simulated data and reports how well it recovers the true quarterly regional growth, the unpublished annual values, the regional weights and the noise scales. Two full-length versions run as tests with `pytest -m slow`.
