Metadata-Version: 2.4
Name: hoptimal
Version: 0.1.0
Summary: High-performance hyperparameter optimisation — C++ core, Python bindings
Keywords: hyperparameter,optimization,bayesian-optimization,gaussian-process,tpe,cma-es,machine-learning,automl
Author-Email: Daniel Dema <danieldema42@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: C++
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: OS Independent
Project-URL: Homepage, https://github.com/danieldema/hoptimal
Project-URL: Repository, https://github.com/danieldema/hoptimal
Project-URL: Issues, https://github.com/danieldema/hoptimal/issues
Requires-Python: >=3.9
Requires-Dist: numpy>=1.24
Provides-Extra: sklearn
Requires-Dist: scikit-learn>=1.3; extra == "sklearn"
Provides-Extra: torch
Requires-Dist: torch>=2.0; extra == "torch"
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.13; extra == "tensorflow"
Provides-Extra: jax
Requires-Dist: jax>=0.4; extra == "jax"
Requires-Dist: flax>=0.7; extra == "jax"
Provides-Extra: xgboost
Requires-Dist: xgboost>=2.0; extra == "xgboost"
Provides-Extra: lightgbm
Requires-Dist: lightgbm>=4.0; extra == "lightgbm"
Provides-Extra: catboost
Requires-Dist: catboost>=1.2; extra == "catboost"
Provides-Extra: benchmark
Requires-Dist: optuna; extra == "benchmark"
Requires-Dist: hyperopt; extra == "benchmark"
Requires-Dist: botorch; extra == "benchmark"
Requires-Dist: scikit-optimize; extra == "benchmark"
Requires-Dist: ray[tune]; extra == "benchmark"
Requires-Dist: flaml; extra == "benchmark"
Requires-Dist: nevergrad; extra == "benchmark"
Requires-Dist: plotly; extra == "benchmark"
Requires-Dist: pandas; extra == "benchmark"
Requires-Dist: scipy; extra == "benchmark"
Provides-Extra: all
Requires-Dist: hoptimal[benchmark,catboost,jax,lightgbm,sklearn,tensorflow,torch,xgboost]; extra == "all"
Description-Content-Type: text/markdown

# hoptimal

[![CI](https://github.com/danieldema/hoptimal/actions/workflows/ci.yml/badge.svg)](https://github.com/danieldema/hoptimal/actions/workflows/ci.yml)
![C++20](https://img.shields.io/badge/C%2B%2B-20-blue)
![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue)
![License: MIT](https://img.shields.io/badge/license-MIT-green)

**A from-scratch C++ Gaussian-Process Bayesian optimizer with Python bindings — competitive with Optuna and scikit-optimize on sample efficiency.**

hoptimal implements Bayesian optimization (a Gaussian-Process surrogate + acquisition
functions) in modern C++20, exposed to Python through pybind11. The GP, the
Cholesky-based inference, the kernel-hyperparameter fitting, and the acquisition
optimization are all hand-written — no GPyTorch, no BoTorch, no scikit-learn under
the hood.

## Benchmark

Median regret (best-found − known optimum, lower is better) vs trial number on 8
standard optimization test functions, 8 seeds each — hoptimal against Optuna and
scikit-optimize:

![regret curves](benchmarks/python/results/regret_curves.png)

hoptimal's GP methods are **best-in-class**: across the 8 functions they win 4
outright and place 2nd on the other 4, beating Optuna's TPE and CMA-ES and
trading wins with scikit-optimize's GP — especially strong on the harder
higher-dimensional problems (Hartmann-6, Ackley, Rosenbrock, Rastrigin).

| Function | hoptimal (best) | Optuna (best) | scikit-optimize |
|----------|------------:|--------------:|----------------:|
| branin       | 0.0018 | 0.70 | **0.0011** |
| hartmann3    | 0.013  | 0.15 | **0.0002** |
| hartmann6    | **0.14** | 1.06 | 0.23 |
| ackley5      | **2.48** | 3.30 | 2.78 |
| rosenbrock4  | **5.29** | 10.4 | 9.28 |
| levy5        | 2.17   | **2.07** | 3.98 |
| rastrigin5   | **18.3** | 27.7 | 41.1 |
| styblinski5  | 48.8   | 55.5 | **27.7** |

Reproduce: `python benchmarks/python/competitors/bench_competitors.py && python benchmarks/python/visualize.py`

## Quickstart

```python
import hoptimal

study = hoptimal.create_study("minimize")   # GP + Expected Improvement by default

def objective(trial):
    x = trial.suggest_float("x", -5.0, 5.0)
    y = trial.suggest_float("y", -5.0, 5.0)
    return (x - 2.0) ** 2 + (y + 1.0) ** 2

study.optimize(objective, n_trials=50)
print(study.best_value, study.best_params)
```

The `Trial` API mirrors Optuna's, so migration is mostly mechanical. For a
runnable tour (toy problem → Branin convergence plot → real sklearn model →
head-to-head vs Optuna) see [`examples/demo.ipynb`](examples/demo.ipynb).

## Features

| Capability | Status |
|------------|--------|
| GP Bayesian optimization — **EI**, **UCB**, **PI** acquisitions | ✅ |
| RBF and Matérn-5/2 kernels, MLE hyperparameter fitting (L-BFGS) | ✅ |
| Float / int / log-scale / categorical parameters | ✅ |
| Multi-objective optimization (Pareto front) | ✅ |
| Median and Hyperband pruners (early stopping) | ✅ |
| Parallel trials (`n_jobs`) with constant-liar | ✅ |
| ML integrations: scikit-learn, XGBoost, LightGBM, CatBoost, PyTorch, JAX/Flax, TensorFlow/Keras | ✅ |

All integrations are validated on clean Linux in CI (see the Dockerfile).

### scikit-learn example

```python
from sklearn.svm import SVC
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from hoptimal.integrations.sklearn import HoptimalSearchCV

pipe = Pipeline([("scaler", StandardScaler()), ("svc", SVC())])
search = HoptimalSearchCV(
    pipe,
    {"svc__C":     ("float", 1e-3, 1e3, {"log": True}),
     "svc__gamma": ("float", 1e-4, 1e0, {"log": True}),
     "svc__kernel":("categorical", ["rbf", "sigmoid"])},
    n_trials=40, cv=5, scoring="accuracy",
)
search.fit(X, y)
print(search.best_score_, search.best_params_)
```

## Build from source

**With Docker (recommended — builds + tests + validates integrations on clean Linux):**

```bash
docker build --target core -t hoptimal .          # C++ core + tests + bindings
docker build --target integrations -t hoptimal .  # + sklearn/xgboost/lightgbm/catboost
docker build --target dl -t hoptimal .            # + pytorch/jax/tensorflow (heavy)
```

**Locally** (needs a C++20 compiler, CMake, Eigen3; GoogleTest/pybind11 are
fetched automatically if absent):

```bash
# C++ library + tests
cmake -B build -DHOPTIMAL_BUILD_TESTS=ON
cmake --build build --parallel
ctest --test-dir build --output-on-failure

# Python package
pip install .
```

## How it works

Each trial, hoptimal fits a **Gaussian Process** to the observations so far, then
picks the next point by optimizing an **acquisition function** over the GP's
posterior. The implementation is hand-written C++:

- **GP regression** with RBF / Matérn-5/2 kernels; inference via a
  **Cholesky factorization** of `K + σ²I` (with jitter for stability) rather
  than an explicit inverse.
- **Kernel hyperparameters** (amplitude, length-scale) fitted by maximizing the
  log marginal likelihood with L-BFGS and random restarts.
- **Acquisition** (EI / UCB / PI) optimized over the normalized `[0,1]ᵈ` space
  via a quasi-random candidate set refined with L-BFGS.

The cost center — repeated Cholesky factorizations and acquisition
optimization — is exactly why the core is in C++.

## License

MIT — see [LICENSE](LICENSE).
