Metadata-Version: 2.4
Name: optfuncs
Version: 0.0.6
Summary: Optimization benchmark functions and pytest-oriented evaluation helpers.
Author-email: hazy <hazy@hazysite.com>
License-Expression: AGPL-3.0-or-later
Project-URL: Repository, https://cnb.cool/uestc/optimization-test
Keywords: optimization,benchmark,pytest,pytorch
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Framework :: Pytest
Classifier: Topic :: Scientific/Engineering :: Mathematics
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.4.4
Requires-Dist: pytest>=9.0.3
Requires-Dist: pytest-xdist>=3.8.0
Provides-Extra: convex
Requires-Dist: cvxpy>=1.8.0; extra == "convex"
Requires-Dist: mosek>=11.0.29; extra == "convex"
Provides-Extra: torch-cpu
Requires-Dist: torch>=2.11.0; extra == "torch-cpu"
Provides-Extra: torch-cu118
Requires-Dist: torch<2.8,>=2.7.1; (sys_platform == "linux" or sys_platform == "win32") and extra == "torch-cu118"
Provides-Extra: torch-cu126
Requires-Dist: torch>=2.11.0; (sys_platform == "linux" or sys_platform == "win32") and extra == "torch-cu126"
Provides-Extra: torch-cu128
Requires-Dist: torch>=2.11.0; (sys_platform == "linux" or sys_platform == "win32") and extra == "torch-cu128"
Provides-Extra: torch-cu130
Requires-Dist: torch>=2.11.0; (sys_platform == "linux" or sys_platform == "win32") and extra == "torch-cu130"
Provides-Extra: torch-rocm
Requires-Dist: torch>=2.11.0; sys_platform == "linux" and extra == "torch-rocm"
Requires-Dist: triton-rocm>=3.6.0; sys_platform == "linux" and extra == "torch-rocm"
Provides-Extra: torch-xpu
Requires-Dist: torch>=2.11.0; (sys_platform == "linux" or sys_platform == "win32") and extra == "torch-xpu"
Requires-Dist: triton-xpu>=3.7.0; (sys_platform == "linux" or sys_platform == "win32") and extra == "torch-xpu"
Dynamic: license-file

# optfunc

`optfunc` provides differentiable PyTorch optfuncs, CVXPY-backed convex
benchmarks, and pytest-oriented helpers for evaluating first-order and
second-order optimizers.

The package is published as `optfuncs` and imported as `optfunc`.

## Install

For CPU-only use:

```bash
pip install "optfuncs[torch-cpu]"
```

For convex-only CVXPY benchmarks:

```bash
pip install "optfuncs[convex]"
```

For CPU-only use plus convex benchmarks:

```bash
pip install "optfuncs[torch-cpu,convex]"
```

With `uv` in this repository:

```bash
uv sync --extra torch-cpu
```

To enable the CVXPY convex benchmark suite:

```bash
uv sync --extra convex
```

Supported extras:

| Extra | Backend | Typical command |
| --- | --- | --- |
| `convex` | CVXPY convex benchmarks, default MOSEK reference solve with SCS fallback | `uv sync --extra convex` |
| `torch-cpu` | CPU, Linux/macOS/Windows | `uv sync --extra torch-cpu` |
| `torch-cu118` | CUDA 11.8, Linux/Windows, PyTorch 2.7.x | `uv sync --extra torch-cu118` |
| `torch-cu126` | CUDA 12.6, Linux/Windows | `uv sync --extra torch-cu126` |
| `torch-cu128` | CUDA 12.8, Linux/Windows | `uv sync --extra torch-cu128` |
| `torch-cu130` | CUDA 13.0, Linux/Windows | `uv sync --extra torch-cu130` |
| `torch-rocm` | ROCm 7.1, Linux | `uv sync --extra torch-rocm` |
| `torch-xpu` | Intel XPU, Linux/Windows | `uv sync --extra torch-xpu` |

Only choose one Torch extra at a time. The Torch extras are configured as
mutually exclusive in `pyproject.toml`. The `convex` extra can be combined with
any Torch extra or used on its own.

### Using optfuncs from another uv project

When another `uv` project wants to expose multiple optional dependency sets,
add the convex/base benchmark support and the Torch backend support separately.
This keeps the target project's extras explicit and lets users choose the Torch
variant independently. `optfuncs[convex]` does not install or require Torch;
add a `torch-*` extra only when the target project also uses unconstrained
PyTorch optfuncs.

Recommended target-project extra layout:

| Target extra | Include |
| --- | --- |
| `test` | `optfuncs[convex]>=0.0.5` for pytest helpers and CVXPY convex benchmarks |
| `cpu` | `optfuncs[torch-cpu]>=0.0.5` for CPU Torch optfuncs |
| `cu130` | `optfuncs[torch-cu130]>=0.0.5` for CUDA 13.0 Torch optfuncs |

Example:

```powershell
uv add --optional test "optfuncs[convex]>=0.0.5"
uv add --optional cu130 "optfuncs[torch-cu130]>=0.0.5"
```

For a CPU project, use:

```powershell
uv add --optional test "optfuncs[convex]>=0.0.5"
uv add --optional cpu "optfuncs[torch-cpu]>=0.0.5"
```

Then install the combination you need from the target project:

```powershell
uv sync --extra test --extra cu130
```

Avoid putting more than one `optfuncs[torch-*]` variant in the same resolved
environment. Keep the `convex` extra separate from the Torch backend extra
because `convex` brings CVXPY/MOSEK support, while `torch-*` selects the Torch
runtime.

Helper scripts:

```bash
# bash/zsh on Linux or macOS, and Git Bash/MSYS/Cygwin on Windows
./scripts/sync_torch_variant.sh cpu
./scripts/sync_torch_variant.sh cu130
./scripts/sync_torch_variant.sh rocm
./scripts/sync_torch_variant.sh xpu
```

```powershell
# Windows PowerShell or PowerShell 7 on Windows/Linux/macOS
.\scripts\sync_torch_variant.ps1 cpu
.\scripts\sync_torch_variant.ps1 cu130
.\scripts\sync_torch_variant.ps1 rocm
.\scripts\sync_torch_variant.ps1 xpu
```

If Windows blocks local PowerShell scripts, run:

```powershell
powershell -ExecutionPolicy Bypass -File .\scripts\sync_torch_variant.ps1 cu130
```

Platform notes:

- CPU works on Linux, macOS, and Windows.
- CUDA extras work on Linux and Windows.
- ROCm works on Linux.
- XPU works on Linux and Windows.
- macOS should use `torch-cpu` in this project configuration.

The backend indexes follow the official uv PyTorch guide and PyTorch install
selector:

- https://docs.astral.sh/uv/guides/integration/pytorch/
- https://pytorch.org/get-started/locally/

## Optfunc Usage

The repository provides these built-in optfuncs:

| Registry name | Class | Known minimizer |
| --- | --- | --- |
| `ackley` | `Ackley` | all zeros |
| `dixonprice` | `DixonPrice` | recursive Dixon-Price optimum |
| `griewank` | `Griewank` | all zeros |
| `levy` | `Levy` | all ones |
| `rastrigin` | `Rastrigin` | all zeros |
| `rosenbrock` | `Rosenbrock` | all ones |
| `rotatedhyperellipsoid` | `RotatedHyperEllipsoid` | all zeros |
| `schwefel` | `Schwefel` | near `420.968746` in every coordinate |
| `sphere` | `Sphere` | all zeros |
| `styblinskitang` | `StyblinskiTang` | near `-2.903534` in every coordinate |
| `sumsquares` | `SumSquares` | all zeros |
| `trid` | `Trid` | `x_i = i * (d + 1 - i)` with 1-based indexing |
| `zakharov` | `Zakharov` | all zeros |

Use a class directly when you know which function you want:

```python
import torch
from optfunc import Sphere

opt_func = Sphere(dim=8, dtype=torch.float64)
x = torch.zeros(8, dtype=torch.float64)

value = opt_func(x)
grad = opt_func.grad(x)
hessian = opt_func.hessian(x)
hvp = opt_func.hvp(x, torch.ones_like(x))
x_star = opt_func.global_minimizer()
distance = opt_func.distance_to_optimum(x)
```

Use `OptFuncRegistry` when a test should select an optfunc by name:

```python
from optfunc import OptFuncRegistry

opt_func = OptFuncRegistry.create("rosenbrock", dim=4)
print(OptFuncRegistry.available())
```

Use `BenchmarkRegistry` when the caller may switch between unconstrained
functions and one concrete convex benchmark instance:

```python
from optfunc import BenchmarkRegistry

convex_problem = BenchmarkRegistry.create(
    "gw_maxcut",
    constraints="convex",
    dim=8,
    seed=19,
)

optimal_value = convex_problem.meta.optimal_value
X_star = convex_problem.known_solution("X")
cvxpy_problem = convex_problem.problem
```

Use `CvxRegistry` when you want a family of parameterised convex programs:

```python
import numpy as np
from optfunc import CvxRegistry

family = CvxRegistry.create("gw_maxcut", dim=8)
rng = np.random.default_rng(0)

theta = family.sample_parameter(rng)
problem = family.build_instance(theta)

theta_sequence = family.generate_sequence(n=5, magnitude=10.0, rng=rng)
problem_sequence = [family.build_instance(theta) for theta in theta_sequence]
```

Each optfunc uses the same conventions:

- input `x` is a 1-D PyTorch tensor with shape `(dim,)`;
- output is a scalar tensor;
- `grad`, `hessian`, and `hvp` use PyTorch autograd unless a subclass provides a better implementation;
- `global_minimizer()` returns a known theoretical minimizer when available;
- `distance_to_optimum(x)` defaults to Euclidean distance to `global_minimizer()`;
- `project_to_bounds(x)` clamps a point into the optfunc's documented search box.

## Convex Benchmark Usage

The repository also provides CVXPY convex benchmarks behind
`constraints="convex"`:

| Registry name | Cone | Shape parameter | Validation path |
| --- | --- | --- | --- |
| `zero`, `zero_cone_qp` | zero cone / equality constraints | vector `dim` | known optimum |
| `nonneg`, `nonnegative_cone_qp` | nonnegative cone | vector `dim` | known optimum |
| `psd`, `psd_cone_projection` | PSD cone | matrix side length `dim` | known optimum |
| `gw_maxcut`, `maxcut_sdp` | PSD cone | graph vertex count `dim` | analytic optimum |

Each convex benchmark family is generated from `CvxRegistry` and exposes
`ProblemFamily` methods:

- `sample_parameter(rng)`: sample a Python/NumPy parameter dictionary;
- `build_instance(theta)`: turn that parameter into a concrete `CvxpyProblem`;
- `perturb(theta, magnitude, rng)`: produce a nearby parameter;
- `generate_sequence(n, magnitude, rng)`: produce a parameter sequence for
  batch testing.

Each concrete `CvxpyProblem` exposes:

- `problem.problem`: the underlying `cvxpy.Problem`;
- `problem.data`: deterministic random instance data used to build the problem;
- `problem.solve(...)`: solve with CVXPY, defaulting to MOSEK and falling back to
  SCS if MOSEK is unavailable in the current environment;
- `problem.known_solution()` and `problem.distance_to_optimum()` when the
  theoretical solution is encoded in the benchmark definition.

The `gw_maxcut` benchmark is a seeded Goemans-Williamson Max-Cut SDP relaxation
on a complete weighted bipartite graph. The seed fixes both the planted cut and
the positive edge weights. Its SDP relaxation optimum is known without calling a
solver: `problem.meta.optimal_value` is the total planted cut weight and
`problem.known_solution("X")` is the rank-one planted cut matrix.

If a convex benchmark does not have a closed-form theoretical solution yet, you
can leave that solution unspecified and use `problem.solve()` as the reference
value/solution path. The default reference solver is MOSEK.

The sequence API uses `numpy.random.Generator` for reproducible parameter
sampling, so `CvxRegistry` and CVXPY problem generation work with only the
`convex` extra. Torch is required only for `constraints="none"` PyTorch
optfuncs.

## Pytest Optimizer Evaluation

The optimizer evaluation API lives in `optfunc.testing`. A standard test file
has three parts:

1. Define or import an optimizer function.
2. Configure one or more `OptimizerCase` objects.
3. Assign `make_optimizer_tests(...)` to a pytest-visible name such as
   `test_my_optimizer`.

Run the file with:

```bash
uv run pytest tests/test_my_optimizer.py -q --tb=short --optfunc-report
```

Each `OptimizerCase` becomes an independent pytest item. If one case fails or
raises an exception, pytest continues running the remaining cases.
`--optfunc-report` prints a final serial summary with function gap, distance to
the theoretical optimum, gradient norm, Hessian information, step count, and
error messages. Do not combine `--optfunc-report` with pytest-xdist `-n` in v1.

### `OptimizerBudget`

`OptimizerBudget` describes the budget passed to the optimizer.

```python
from optfunc.testing import OptimizerBudget

budget = OptimizerBudget(max_steps=500, lr=0.05)
```

Fields:

- `max_steps`: positive integer iteration limit.
- `lr`: positive learning-rate-like scalar. The test harness does not enforce
  how the optimizer uses it; your optimizer reads it from `problem.budget.lr`.

For `torch.optim.Adam`, a typical use is:

```python
optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
for step in range(problem.budget.max_steps):
    ...
```

### `ConvergenceTolerances`

`ConvergenceTolerances` decides whether a finished optimizer run passes.

```python
from optfunc.testing import ConvergenceTolerances

tolerances = ConvergenceTolerances(
    value_gap=1e-8,
    x_distance=1e-4,
    grad_norm=1e-4,
    hessian_min_eig=0.0,
)
```

Fields:

- `value_gap`: maximum allowed absolute gap between final value and the known
  theoretical optimum value. Set to `None` to skip this check.
- `x_distance`: maximum allowed distance from final `x` to the known theoretical
  minimizer. Set to `None` to skip this check.
- `grad_norm`: maximum allowed Euclidean norm of the final gradient. Set to
  `None` to skip this check.
- `hessian_min_eig`: optional lower bound on the final Hessian's smallest
  eigenvalue. Set to `None` to skip this check.

The report still computes available metrics even when a tolerance is `None`;
`None` only disables that pass/fail check.

### `OptimizerCase`

`OptimizerCase` describes one pytest item.

```python
from optfunc.testing import OptimizerCase
from optfunc.testing import OptimizerBudget, ConvergenceTolerances

case = OptimizerCase(
  opt_func="sphere",
  constraints="none",
  dim=8,
  budget=OptimizerBudget(max_steps=350, lr=0.05),
  tolerances=ConvergenceTolerances(value_gap=1e-8, x_distance=1e-4, grad_norm=1e-4),
  start="near_minimizer",
  start_radius=0.5,
  seed=0,
  hessian_max_dim=32,
)
```

Important fields:

- `opt_func`: registry name such as `"sphere"` or an already-created
  `TorchOptFunction` instance.
- `constraints`: `"none"` for the existing Torch optfunc suite, or `"convex"`
  for the CVXPY convex benchmark suite.
- `dim`: required when `opt_func` is a string.
- `batch_size`: for `constraints="convex"`, generate this many same-family
  problem instances.
- `perturb_magnitude`: for convex batches, controls successive parameter
  perturbations in `ProblemFamily.generate_sequence(...)`.
- `budget`: `OptimizerBudget` passed to the optimizer through
  `problem.budget`.
- `tolerances`: `ConvergenceTolerances` used after the optimizer returns.
- `case_id`: optional pytest id; by default this is like `sphere[8]`.
- `x0`: optional explicit initial point. If omitted, the harness builds one
  from `start`.
- `start`: `"near_minimizer"`, `"random"`, or `"zeros"`.
- `start_radius`: offset size used by `"near_minimizer"`.
- `seed`: random seed used by `"random"`.
- `device`: optional torch device string, for example `"cuda"` or `"cpu"`.
- `dtype`: torch dtype, default `torch.float64`.
- `hessian_max_dim`: largest dimension for dense Hessian report metrics. Larger
  cases skip dense Hessian metrics to avoid slow tests.

For `constraints="convex"`, the optimizer receives a
`ConvexOptimizationProblem` with `.instances`, `.parameters`, and `.problem`
for single-instance cases. If the optimizer solves the CVXPY instances in
place and returns `None`, the harness checks objective gaps and known-solution
distances when those references are available.

### `make_optimizer_tests`

`make_optimizer_tests` converts an optimizer plus cases into a pytest test
function.

```python
from optfunc.testing import make_optimizer_tests

test_my_optimizer = make_optimizer_tests(
    optimizer=my_optimizer,
    cases=[case1, case2],
    name="test_my_optimizer",
)
```

Rules:

- Assign the returned function to a module-level variable whose name starts with
  `test_`, otherwise pytest will not collect it.
- For `constraints="none"`, `optimizer` must accept one `OptimizationProblem`
  argument.
- For `constraints="none"`, the optimizer may return either a final
  `torch.Tensor` or an `OptimizerResult`.
- Every case becomes an independent parametrized pytest item.

For convex cases, the optimizer accepts `ConvexOptimizationProblem` instead:

```python
from optfunc.testing import (
  ConvergenceTolerances,
  ConvexOptimizationProblem,
  OptimizerCase,
  make_optimizer_tests,
)


def cvxpy_solver(problem: ConvexOptimizationProblem):
  for instance in problem:
    instance.solve()


test_convex_batch = make_optimizer_tests(
  optimizer=cvxpy_solver,
  cases=[
    OptimizerCase(
      opt_func="gw_maxcut",
      constraints="convex",
      dim=8,
      batch_size=4,
      perturb_magnitude=10.0,
      seed=0,
      tolerances=ConvergenceTolerances(value_gap=1e-5, x_distance=1e-3, grad_norm=None),
    )
  ],
)
```

## Standard Adam Test Example

This example shows the recommended shape for a user-owned optimizer wrapper.
The test harness does not hide `torch.optim.Adam`; the user function decides how
to initialize Adam, how to use the budget, and what history to expose.

Create `tests/test_torch_adam.py`:

```python
import torch

from optfunc.testing import (
  ConvergenceTolerances,
  OptimizerBudget,
  OptimizerCase,
  OptimizerResult,
  make_optimizer_tests,
  optimizer_adapter,
)


def scalar_item(value):
  return float(value.detach().cpu().item())


@optimizer_adapter(name="torch_adam")
def torch_adam(problem):
  x = problem.x0.detach().clone().requires_grad_(True)
  optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
  history = []

  for step in range(1, problem.budget.max_steps + 1):
    optimizer.zero_grad(set_to_none=True)
    loss = problem.value(x)
    loss.backward()
    optimizer.step()

    with torch.no_grad():
      x.copy_(problem.project_to_bounds(x.detach()))

    if step % 25 == 0 or step == problem.budget.max_steps:
      x_now = x.detach()
      grad = problem.grad(x_now)
      history.append(
        {
          "step": step,
          "value": scalar_item(problem.value(x_now)),
          "grad_norm": scalar_item(torch.linalg.vector_norm(grad)),
          "x_norm": scalar_item(torch.linalg.vector_norm(x_now)),
        }
      )

  return OptimizerResult(
    final_x=x.detach().clone(),
    steps=problem.budget.max_steps,
    history=history,
  )


test_torch_adam = make_optimizer_tests(
  optimizer=torch_adam,
  cases=[
    OptimizerCase(
      opt_func="sphere",
      dim=8,
      budget=OptimizerBudget(max_steps=350, lr=0.05),
      tolerances=ConvergenceTolerances(
        value_gap=1e-8,
        x_distance=1e-4,
        grad_norm=1e-4,
      ),
      start="near_minimizer",
      start_radius=0.5,
    ),
    OptimizerCase(
      opt_func="rosenbrock",
      dim=4,
      budget=OptimizerBudget(max_steps=1200, lr=0.02),
      tolerances=ConvergenceTolerances(
        value_gap=5e-4,
        x_distance=5e-2,
        grad_norm=1e-2,
      ),
      start="near_minimizer",
      start_radius=0.25,
    ),
  ],
)
```

Run:

```bash
uv run pytest tests/test_torch_adam.py -q --tb=short --optfunc-report
```

The `OptimizationProblem` object passed into `torch_adam` exposes:

- `problem.opt_func`: the selected `TorchOptFunction`;
- `problem.x0`: the initial point for this case;
- `problem.budget`: the `OptimizerBudget`;
- `problem.value(x)`: scalar objective value;
- `problem.grad(x)`: gradient;
- `problem.value_and_grad(x)`: objective and gradient;
- `problem.hessian(x)`: dense Hessian;
- `problem.hvp(x, v)`: Hessian-vector product;
- `problem.project_to_bounds(x)`: clamp into the optfunc's bounds.

For second-order methods, call `problem.hessian(x)` or `problem.hvp(x, v)`
inside the same optimizer wrapper and return the same `OptimizerResult` shape.

## Built-In Adam Helper

For smoke tests or examples, `optfunc.testing.make_torch_adam()` provides the
same Adam loop as a convenience:

```python
from optfunc.testing import make_torch_adam, make_optimizer_tests

test_adam = make_optimizer_tests(
    optimizer=make_torch_adam(),
    cases=[...],
)
```

For production optimizer tests, prefer writing the small wrapper yourself so the
learning rate, projection, stopping rule, and history format are explicit in
your test file.

## Development

Developer workflows, release steps, PyPI verification, project structure, and
API design notes live in [README.dev.md](README.dev.md).

Optfunc definitions are adapted from SFU's optimization benchmark collection:

https://www.sfu.ca/~ssurjano/optimization.html
