Metadata-Version: 2.4
Name: calibrated-bo
Version: 0.2.2
Summary: Calibrated Bayesian optimization on top of bayesian-gp-cvloss: composable conformal prediction (Weighted + Localized + Adaptive), single- and multi-objective acquisition (cUCB / cEI / cEHVI), q-batch greedy with fantasies, and a one-call BO loop with persistent ACI state.
Author-email: Shifa Zhong <sfzhong@tongji.edu.cn>
License-Expression: MIT
Project-URL: Repository, https://github.com/Shifa-Zhong/calibrated-bo
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: bayesian-gp-cvloss>=0.3.1
Requires-Dist: numpy>=1.18.0
Requires-Dist: scipy>=1.5.0
Requires-Dist: scikit-learn>=0.23.0
Dynamic: license-file

# calibrated-bo

Calibrated Bayesian optimization with composable conformal prediction, built
on top of [`bayesian-gp-cvloss`](https://github.com/Shifa-Zhong/bayesian-gp-cvloss).

## What's in the box

**Calibration**
- `CompositeConformalCalibrator` with three orthogonal switches
  - `localized=True` — restrict to x*'s k-NN of the calibration set
  - `weighted=True`  — RBF weights against x* on the surviving subset
  - `adaptive=True`  — Gibbs-Candes ACI controller updates α from rolling coverage
- `MultiOutputCalibrator` — one calibrator per objective with a uniform API
- `WeightedConformalCalibrator` / `LocalizedConformalCalibrator` /
  `AdaptiveConformalCalibrator` — ablation presets that lock specific flags
- `AdaptiveAlphaController` — ACI controller with JSON `export_state` /
  `load_state` for cross-restart persistence
- Turning all three switches off is numerically identical to standard split CP

**Acquisition**
- `CalibratedUCB`, `CalibratedEI` — single-objective, plug calibrated sigma
  into UCB / EI; minimise via the `maximise=False` flag
- `CalibratedEHVI` — MC expected-hypervolume improvement using per-objective
  calibrated uncertainty; supports any direction per objective
- `greedy_q_batch` — sequential greedy with kriging-believer fantasies for
  q-batch acquisition (used by EHVI; available to user code too)

**Loop**
- `CalibratedBOLoop` — single-call BO with `suggest()` / `observe()`
  - Single- *and* multi-objective (`n_objectives`, `objectives_direction`)
  - Auto-refits GP every `retrain_every`, recalibrates every `recalibrate_every`
  - ACI controllers persist across recalibrations and (via export/load)
    across process restarts
  - q-batch via `batch_size > 1`

**Diagnostics**
- `CoverageTracker` — rolling and cumulative empirical coverage
- `reliability_curve` — promised vs empirical coverage across a level grid
- `pit_histogram` — PIT diagnostic under the Gaussian-sigma approximation

## Install

```bash
pip install calibrated-bo
```

Requires `bayesian-gp-cvloss>=0.3.1`.

## Quickstart — single objective

```python
import numpy as np
from calibrated_bo import CalibratedBOLoop

def f(x):
    return float(np.sum((np.asarray(x) - np.array([0.3, 0.7])) ** 2))

bo = CalibratedBOLoop(
    bounds=[(0.0, 1.0), (0.0, 1.0)],
    objective="minimize",
    calibrator_config={
        "alpha": 0.1,
        "localized": True, "k": 20,
        "weighted": True,
        "adaptive": True, "gamma": 0.05,
    },
    acquisition="cUCB",                  # or "cEI"
    acquisition_kwargs={"beta": 2.0},
    batch_size=1,
    initial_random=5,
    gp_max_evals=30,
)

for _ in range(20):
    X_next = bo.suggest()
    y_next = np.array([f(x) for x in X_next])
    bo.observe(X_next, y_next)

print("best so far:", bo.best)        # (x, y)
print("diagnostics:", bo.diagnostics())
```

## Quickstart — multi-objective

```python
import numpy as np
from calibrated_bo import CalibratedBOLoop

def f1(x): return float((x[0] - 0.3) ** 2)
def f2(x): return float((x[0] - 0.7) ** 2)

bo = CalibratedBOLoop(
    bounds=[(0.0, 1.0)],
    n_objectives=2,
    objectives_direction=["minimize", "minimize"],
    calibrator_config={
        "alpha": 0.1,
        "localized": True, "k": 12,
        "weighted": True,
        "adaptive": True, "gamma": 0.05,
    },
    acquisition="cEHVI",
    acquisition_kwargs={"n_samples": 64},
    batch_size=1,
    initial_random=6,
    gp_max_evals=20,
)

for _ in range(20):
    X_next = bo.suggest()
    Y_next = np.array([[f1(x), f2(x)] for x in X_next])
    bo.observe(X_next, Y_next)

diag = bo.diagnostics()
print("|Pareto| =", len(diag["best"]["pareto_Y"]))
print("calibrator 0:", diag["calibrators"][0])  # alpha_current, rolling_coverage_20, ...
```

## Calibrator-only usage

If you only want the calibrator (no BO loop), drop it on top of any fitted
`GPCrossValidatedOptimizer`:

```python
from bayesian_gp_cvloss import GPCrossValidatedOptimizer
from calibrated_bo import CompositeConformalCalibrator

opt = GPCrossValidatedOptimizer(X_train, y_train, scoring="cv_rmse")
opt.optimize(max_evals=50)

cal = CompositeConformalCalibrator(
    alpha=0.1, localized=True, weighted=True, adaptive=False
).fit_cv(opt)

mean, lower, upper = cal.predict_interval(X_new)
```

## ACI persistence across restarts

```python
# Export at shutdown
snapshot = bo.export_aci_state()         # plain dict; JSON-serialisable

# At startup, on a fresh CalibratedBOLoop:
bo2.load_aci_state(snapshot)             # rolling coverage history is back
```

## Design notes

Three calibration flags (Localized / Weighted / Adaptive) are **orthogonal**:

| Localized | Weighted | Adaptive | Equivalent to |
|:---:|:---:|:---:|:---|
| off | off | off | standard split CP |
| on  | off | off | Localized CP |
| off | on  | off | Tibshirani 2019 Weighted CP |
| off | off | on  | Gibbs-Candes 2021 ACI |
| on  | on  | off | Localized weighted CP |
| on  | on  | on  | This package's recommended setting |

Sign conventions: the BO loop trains *every* GP in max-oriented space
(any `minimize` column is negated at GP-training time) so EHVI / Pareto /
hypervolume / cEI / cUCB stay sign-agnostic internally. `Y_history` and
`bo.best` stay in raw user space at the API boundary.

### ACI caveat under active learning

The standard conformal coverage guarantee (`P[y* in interval] >= 1-alpha`
marginal) holds under exchangeability of calibration and test points. In
a BO loop this is violated by construction -- the next `suggest()` point
is biased toward high-acquisition regions. Empirically the ACI controller
still tracks nominal coverage well (the test suite verifies this), but no
finite-sample theorem applies; treat ACI's coverage as approximate inside
the BO loop. If exact coverage matters more than sample efficiency,
calibrate on an independent held-out set via `fit(...)` rather than
`fit_cv(...)`, and refresh it periodically.
