Metadata-Version: 2.4
Name: hoptimal
Version: 0.2.0
Summary: High-performance hyperparameter optimisation — C++ core, Python bindings
Keywords: hyperparameter,optimization,bayesian-optimization,gaussian-process,machine-learning,automl
Author-Email: Daniel Dema <danieldema42@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: C++
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: OS Independent
Project-URL: Homepage, https://github.com/danieldema/hoptimal
Project-URL: Repository, https://github.com/danieldema/hoptimal
Project-URL: Issues, https://github.com/danieldema/hoptimal/issues
Requires-Python: >=3.9
Requires-Dist: numpy>=1.24
Provides-Extra: sklearn
Requires-Dist: scikit-learn>=1.3; extra == "sklearn"
Provides-Extra: torch
Requires-Dist: torch>=2.0; extra == "torch"
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.13; extra == "tensorflow"
Provides-Extra: jax
Requires-Dist: jax>=0.4; extra == "jax"
Requires-Dist: flax>=0.7; extra == "jax"
Provides-Extra: xgboost
Requires-Dist: xgboost>=2.0; extra == "xgboost"
Provides-Extra: lightgbm
Requires-Dist: lightgbm>=4.0; extra == "lightgbm"
Provides-Extra: catboost
Requires-Dist: catboost>=1.2; extra == "catboost"
Provides-Extra: viz
Requires-Dist: matplotlib>=3.5; extra == "viz"
Requires-Dist: pandas>=1.5; extra == "viz"
Requires-Dist: scikit-learn>=1.3; extra == "viz"
Provides-Extra: benchmark
Requires-Dist: optuna; extra == "benchmark"
Requires-Dist: hyperopt; extra == "benchmark"
Requires-Dist: botorch; extra == "benchmark"
Requires-Dist: scikit-optimize; extra == "benchmark"
Requires-Dist: ray[tune]; extra == "benchmark"
Requires-Dist: flaml; extra == "benchmark"
Requires-Dist: nevergrad; extra == "benchmark"
Requires-Dist: plotly; extra == "benchmark"
Requires-Dist: pandas; extra == "benchmark"
Requires-Dist: scipy; extra == "benchmark"
Provides-Extra: all
Requires-Dist: hoptimal[benchmark,catboost,jax,lightgbm,sklearn,tensorflow,torch,viz,xgboost]; extra == "all"
Description-Content-Type: text/markdown

# hoptimal

[![CI](https://github.com/danieldema/hoptimal/actions/workflows/ci.yml/badge.svg)](https://github.com/danieldema/hoptimal/actions/workflows/ci.yml)
![C++20](https://img.shields.io/badge/C%2B%2B-20-blue)
![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue)
![License: MIT](https://img.shields.io/badge/license-MIT-green)

**A from-scratch C++ Gaussian-Process Bayesian optimizer with Python bindings; competitive with Optuna and scikit-optimize on sample efficiency.**

hoptimal implements Bayesian optimization for ML model hyperparameters (a Gaussian-Process surrogate + acquisition
functions) in modern C++20, exposed to Python through pybind11.

## Benchmark

Median regret vs trial number on 8 standard optimization test functions, 8 seeds each: hoptimal against Optuna and scikit-optimize:

![regret curves](benchmarks/python/results/regret_curves.png)

Across the 8 functions, hoptimal's GP wins **3 outright** (Hartmann-6, Ackley,
Rosenbrock) and places a close 2nd on the rest. It **beats every Optuna sampler
on 6 of 8** — losing only levy/rastrigin to CMA-ES — and is edged on the two
smooth low-dimensional functions by scikit-optimize (by decimals). It is
strongest on the higher-dimensional problems.

| Function | hoptimal (best) | Optuna (best) | scikit-optimize |
|----------|------------:|--------------:|----------------:|
| branin       | 0.0013 | 0.70 | **0.0011** |
| hartmann3    | 0.0004 | 0.15 | **0.0002** |
| hartmann6    | **0.044** | 1.06 | 0.14 |
| ackley5      | **2.21** | 3.30 | 2.88 |
| rosenbrock4  | **4.37** | 10.4 | 11.9 |
| levy5        | 2.37   | **2.07** | 3.55 |
| rastrigin5   | 29.5   | **27.7** | 33.6 |
| styblinski5  | 47.9   | 55.5 | **26.5** |

Reproduce: `python benchmarks/python/competitors/bench_competitors.py && python benchmarks/python/visualize.py`

Against Optuna's **BoTorch-backed GP sampler** specifically — the toughest
GP-vs-GP comparison — hoptimal goes **4–4** on final quality, winning the harder
higher-dimensional and multimodal functions (hartmann6, rosenbrock, rastrigin,
styblinski). Reproduce (reports how many trials hoptimal saves to match Optuna):
`python benchmarks/python/bench_vs_optuna.py --optuna-sampler gp --save && python benchmarks/python/visualize_vs_optuna.py`

## Installation

```bash
pip install hoptimal
```

Prebuilt wheels are published for Linux, macOS (Intel + Apple Silicon), and
Windows on CPython 3.9–3.13, so no C++ toolchain is required.

The ML-framework integrations pull in their own dependencies and are installed
via extras:

```bash
pip install hoptimal[sklearn]     # scikit-learn
pip install hoptimal[xgboost]     # XGBoost
pip install hoptimal[lightgbm]    # LightGBM
pip install hoptimal[catboost]    # CatBoost
pip install hoptimal[torch]       # PyTorch
pip install hoptimal[jax]         # JAX + Flax
pip install hoptimal[tensorflow]  # TensorFlow / Keras
pip install hoptimal[viz]         # matplotlib plots + pandas + importances
```

## Quickstart

```python
import hoptimal

study = hoptimal.create_study("minimize")   # GP + Expected Improvement by default

def objective(trial):
    x = trial.suggest_float("x", -5.0, 5.0)
    y = trial.suggest_float("y", -5.0, 5.0)
    return (x - 2.0) ** 2 + (y + 1.0) ** 2

study.optimize(objective, n_trials=50)
print(study.best_value, study.best_params)
```

The `Trial` API mirrors Optuna's, so migration is mostly mechanical. For a
runnable tour (toy problem → Branin convergence plot → real sklearn model →
head-to-head vs Optuna) see [`examples/demo.ipynb`](examples/demo.ipynb).

### scikit-learn example

```python
from sklearn.svm import SVC
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from hoptimal.integrations.sklearn import HoptimalSearchCV

pipe = Pipeline([("scaler", StandardScaler()), ("svc", SVC())])
search = HoptimalSearchCV(
    pipe,
    {"svc__C":     ("float", 1e-3, 1e3, {"log": True}),
     "svc__gamma": ("float", 1e-4, 1e0, {"log": True}),
     "svc__kernel":("categorical", ["rbf", "sigmoid"])},
    n_trials=40, cv=5, scoring="accuracy",
)
search.fit(X, y)
print(search.best_score_, search.best_params_)
```

## Visualization & analysis

Install the plotting extras (`pip install hoptimal[viz]`) to inspect a study with
a matplotlib API that mirrors Optuna's:

![visualization gallery](assets/visualization.png)

```python
from hoptimal import visualization as viz, get_param_importances

viz.plot_optimization_history(study)   # best-so-far convergence
viz.plot_param_importances(study)      # random-forest importances
viz.plot_slice(study)                  # objective vs each parameter
viz.plot_contour(study, params=["x", "y"])
viz.plot_pareto_front(mo_study)        # multi-objective

get_param_importances(study)           # -> {"x": 0.62, "lr": 0.24, ...}
```

Export every trial to a pandas `DataFrame` for custom analysis or logging:

```python
df = study.trials_dataframe()
# columns: number, value, datetime_start/complete, duration, state,
#          params_<name>..., user_attrs_<key>...
```

See the [changelog](CHANGELOG.md) for the full list of changes.

## Build from source

**With Docker (recommended — builds + tests + validates integrations on clean Linux):**

```bash
docker build --target core -t hoptimal .          # C++ core + tests + bindings
docker build --target integrations -t hoptimal .  # + sklearn/xgboost/lightgbm/catboost
docker build --target dl -t hoptimal .            # + pytorch/jax/tensorflow (heavy)
```

**Locally** (needs a C++20 compiler, CMake, Eigen3; GoogleTest/pybind11 are
fetched automatically if absent):

```bash
# C++ library + tests
cmake -B build -DHOPTIMAL_BUILD_TESTS=ON
cmake --build build --parallel
ctest --test-dir build --output-on-failure

# Python package
pip install .
```

## How it works

Each trial, hoptimal fits a **Gaussian Process** to the observations so far, then
picks the next point by optimizing an **acquisition function** over the GP's
posterior. The implementation is hand-written C++:

- **GP regression** with RBF / Matérn-5/2 kernels using **ARD** (a separate
  length-scale per dimension, so anisotropic objectives are modelled correctly);
  inference via a **Cholesky factorization** of `K + σ²I` (with jitter for
  stability) rather than an explicit inverse.
- **Kernel hyperparameters** fitted by maximizing the log marginal likelihood
  (**MAP**, with weak log-normal priors that regularize the per-dimension
  length-scales) via L-BFGS with random restarts.
- **Acquisition** — **LogEI** by default (a numerically-stable log Expected
  Improvement that doesn't underflow far from data), also UCB / PI — optimized
  over the normalized `[0,1]ᵈ` space via a low-discrepancy (Halton) candidate set
  refined with L-BFGS. The first few trials use a space-filling Halton design
  before the GP takes over.
- **Categorical** parameters are **one-hot encoded** so unordered categories
  aren't given a false ordering.

## License

MIT — see [LICENSE](LICENSE).
