Metadata-Version: 2.4
Name: intelloptim
Version: 0.1.1
Summary: Distributed black-box optimization on Optuna's ask/tell API, with classical and quantum samplers.
Author-email: Matteo Barbetti <matteo.barbetti.94@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/mbarbetti/intelloptim
Project-URL: Documentation, https://intelloptim.readthedocs.io
Project-URL: Repository, https://github.com/mbarbetti/intelloptim
Project-URL: Issues, https://github.com/mbarbetti/intelloptim/issues
Project-URL: Changelog, https://github.com/mbarbetti/intelloptim/blob/main/CHANGELOG.md
Project-URL: Hopaas paper, https://arxiv.org/abs/2301.05522
Keywords: optimization,black-box,hyperparameter,optuna,bayesian,quantum,distributed
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: Scientific/Engineering :: Quantum Computing
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pydantic<3,>=2.6
Requires-Dist: optuna>=4.0
Requires-Dist: numpy>=1.24
Requires-Dist: httpx>=0.27
Requires-Dist: click>=8.1
Provides-Extra: gp
Requires-Dist: torch>=2.1; extra == "gp"
Requires-Dist: scipy>=1.10; extra == "gp"
Provides-Extra: qmc
Requires-Dist: scipy>=1.10; extra == "qmc"
Provides-Extra: quantum
Requires-Dist: qiskit>=1.2; extra == "quantum"
Provides-Extra: quantum-ibm
Requires-Dist: intelloptim[quantum]; extra == "quantum-ibm"
Requires-Dist: qiskit-ibm-runtime>=0.30; extra == "quantum-ibm"
Provides-Extra: samplers
Requires-Dist: intelloptim[gp,qmc,quantum]; extra == "samplers"
Provides-Extra: dev
Requires-Dist: intelloptim[quantum]; extra == "dev"
Requires-Dist: intelloptim[docs]; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Requires-Dist: respx>=0.21; extra == "dev"
Requires-Dist: ruff<0.18,>=0.17; extra == "dev"
Requires-Dist: black<27,>=26.10; extra == "dev"
Requires-Dist: mypy<3,>=2.4; extra == "dev"
Requires-Dist: pre-commit>=4.0; extra == "dev"
Requires-Dist: nox>=2024.10; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.1; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx<9,>=8.2; extra == "docs"
Requires-Dist: furo>=2024.8.6; extra == "docs"
Dynamic: license-file

<!--
<div align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/mbarbetti/intelloptim/main/docs/images/intelloptim-logo-w-bkg.png">
    <img alt="intelloptim" src="https://raw.githubusercontent.com/mbarbetti/intelloptim/main/docs/images/intelloptim-logo-black.png" width="600">
  </picture>
</div>
-->

# intelloptim

[![PyPI](https://img.shields.io/pypi/v/intelloptim.svg)](https://pypi.org/project/intelloptim/)
[![Python](https://img.shields.io/pypi/pyversions/intelloptim.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/pypi/l/intelloptim.svg)](https://github.com/mbarbetti/intelloptim/blob/main/LICENSE)
[![CI](https://github.com/mbarbetti/intelloptim/actions/workflows/ci.yml/badge.svg)](https://github.com/mbarbetti/intelloptim/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/mbarbetti/intelloptim/graph/badge.svg)](https://codecov.io/gh/mbarbetti/intelloptim)
[![Documentation](https://readthedocs.org/projects/intelloptim/badge/?version=latest)](https://intelloptim.readthedocs.io)

Distributed black-box optimization built on [Optuna](https://optuna.org/)'s
ask/tell API. Evaluators on any infrastructure (HPC, cloud, on-prem)
collaborate on a single study through a coordinating server, and the sampler
layer treats classical and quantum samplers uniformly.

This package is the client SDK and the sampler layer. It talks to an
intelloptim server over HTTP; the server itself is not part of this package.
Full documentation: **[intelloptim.readthedocs.io](https://intelloptim.readthedocs.io)**.

## Where it comes from

`intelloptim` is the evolution of **Hopaas**, *Hyperparameter OPtimization As A
Service* (Barbetti & Anderlini, [arXiv:2301.05522](https://arxiv.org/abs/2301.05522)).
Hopaas was built to coordinate many trainings of a machine-learning model on
opportunistic resources from different providers, which share nothing but a
network connection to a server that holds the study. It tuned the generative
models behind the LHCb ultra-fast simulation, with studies fed by more than
twenty concurrent nodes.

`intelloptim` keeps that philosophy and generalizes it to any
small-to-moderate-dimensional black-box problem (system calibration, model
tuning, ...), with more robustness and security:

- bearer-token authentication (the token never appears in a URL or in a study's
  identity) and a client that refuses unsafe token files;
- studies identified by their content and scoped to a user or group, so
  independent processes join the same study, and a mismatching configuration is
  refused instead of silently mixed;
- more samplers (Optuna, built-in and quantum policy samplers) behind one API,
  with background rebuilds for the ones that refit;
- `enqueue`, `stop`, call quotas and a worker for SLURM.

## Installation

```bash
pip install intelloptim
```

Samplers with heavier requirements are optional extras:

| Extra | Adds | For |
|---|---|---|
| `gp` | `torch`, `scipy` | the Gaussian-process sampler |
| `qmc` | `scipy` | the quasi-Monte Carlo sampler |
| `quantum` | `qiskit` | the simulated quantum samplers |
| `quantum-ibm` | `qiskit-ibm-runtime` | quantum samplers on IBM hardware |
| `samplers` | `gp`, `qmc`, `quantum` | everything a server needs to resolve any sampler |

Python 3.11 or newer is required. You also need the URL of a server and an API
token.

## An Optuna wrapper, with a server

If you know Optuna's ask/tell API, you know the model: a study, trials, a
sampler and a pruner, with the study living on a server so that many machines
can contribute to it.

```python
import intelloptim as intopt

with intopt.client.Client(url="https://intelloptim.example.org") as client:
    study = client.create_study(
        study_name="pump-cal-2026-Q2",
        direction="minimize",
        sampler_spec=intopt.samplers.TPESamplerSpec(seed=42),
        parameters=[
            intopt.params.Float("k_friction", 1e-3, 1.0, log=True),
            intopt.params.Float("bypass_ratio", 0.0, 0.3),
        ],
    )

    def objective(trial: intopt.client.Trial) -> float:
        k = trial.sampled_float("k_friction")
        b = trial.sampled_float("bypass_ratio")
        return rms_error_against_telemetry(k, b)

    study.optimize(objective, n_trials=200)
    print(study.best_trial)
```

**Coming from Optuna?** The search space is declared once, in `create_study`,
and not inside the objective with `suggest_*`. The objective reads the sampled
values by name (`sampled_float`, `sampled_int`, `sampled_categorical`).
[More](https://intelloptim.readthedocs.io/en/latest/guides/optuna-and-intelloptim.html)

- **ask / tell:** `optimize` runs the loop; `study.ask()` and `study.tell()` give
  you control of it. `create_study` is idempotent, so any process that declares
  the same study joins it.
  [More](https://intelloptim.readthedocs.io/en/latest/guides/ask-tell-enqueue.html)
- **enqueue:** `study.enqueue({...})` queues a point to evaluate next, and
  `study.enqueue({...}, value=...)` records one that was evaluated elsewhere.
- **prune:** report intermediate values and let the study's pruner stop a
  hopeless trial; it does not count against your call quota.
  [More](https://intelloptim.readthedocs.io/en/latest/guides/pruning.html)

  ```python
  def objective(trial: intopt.client.Trial) -> float:
      for epoch in range(20):
          loss = train_one_epoch(trial.sampled_float("lr"))
          trial.report(loss, epoch)
          if trial.should_prune():
              raise intopt.client.errors.TrialPruned
      return loss
  ```

- **samplers:** three families behind `sampler_spec=`: **Optuna** (Random, QMC,
  TPE, GP, NSGA-II), **built-in** (simulated annealing `SA`, objective-weighted
  Gaussian `OWG`) and **quantum** policy samplers, simulated or on hardware.
  [More](https://intelloptim.readthedocs.io/en/latest/guides/sampling.html)

## Authentication

Every request carries an API token. The client takes it from the `token=`
argument, then from the `INTELLOPTIM_TOKEN` environment variable, then from
`~/.config/intelloptim/token`, which it only reads if it is a regular file
owned by you with no group or other permissions. Treat the token as a password:
never put it in code, command lines, study specifications or logs, and use
HTTPS for any non-local server.
[More](https://intelloptim.readthedocs.io/en/latest/guides/authentication.html)

## Running on a cluster

`intelloptim-worker` runs an objective on a batch system without a driver
script. Each task of a SLURM array job imports the objective, joins the study
and runs its trials; worker and site ids come from the SLURM environment.

```bash
intelloptim-worker evaluator \
    --server https://intelloptim.example.org \
    --import objective.py:my_objective \
    --n-trials 50
```

[More](https://intelloptim.readthedocs.io/en/latest/guides/cluster.html), with a
complete array-job script.

## Status

Early development (alpha): the API may still change between minor versions.
See [CHANGELOG.md](https://github.com/mbarbetti/intelloptim/blob/main/CHANGELOG.md)
for what changed in each release and
[CONTRIBUTING.md](https://github.com/mbarbetti/intelloptim/blob/main/CONTRIBUTING.md)
to get involved.

## License

Apache-2.0, see [LICENSE](https://github.com/mbarbetti/intelloptim/blob/main/LICENSE).

## Citation

If you use `intelloptim` in academic work, please cite the Hopaas paper:
M. Barbetti and L. Anderlini, *Hyperparameter Optimization as a Service on INFN
Cloud*, [arXiv:2301.05522](https://arxiv.org/abs/2301.05522).
