Metadata-Version: 2.3
Name: ZoSolvers
Version: 0.1.5
Summary: Zeroth-order optimisation solvers with Gaussian and sphere random oracles
Author: amirali
Author-email: amirali77farzin@gmail.com
Requires-Python: >=3.10
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: matplotlib (>=3.10.8,<4.0.0)
Requires-Dist: numpy (>=2.2.6,<3.0.0)
Description-Content-Type: text/markdown

# ZoSolvers

**Zeroth-order optimisation solvers with Gaussian and sphere random oracles.**

ZoSolvers provides gradient-free solvers for **minimisation** and **minimax** problems. When the gradient of the objective is unavailable — because the function is non-differentiable, comes from a black-box simulator, or is too expensive to differentiate — ZoSolvers estimates it using random directional perturbations.

📖 Full documentation: [ZoSolvers Manual (PDF)](https://github.com/amirali78frz/ZoSolvers/blob/main/Docs/ZoSolvers_Manual.pdf)

## Install

```bash
pip install ZoSolvers
```

**Requirements:** Python ≥ 3.10, NumPy ≥ 2.2, Matplotlib ≥ 3.7

---

## Features

- **Two oracle types** — Gaussian (`u ~ N(0, B⁻¹)`) and sphere (uniform on the unit sphere), both with optional preconditioning via a precision matrix `B`
- **Four solvers** — ZOGD, ZOEGm (minimisation), ZOGDA, ZOEGmm (minimax)
- **Three finite-difference methods** — forward, backward, centered
- **Flexible mini-batching** — fixed number of oracle samples per step, or a growing schedule (`t="iteration"`)
- **Constrained problems** — pass any projection function, and choose whether the initial guess is projected too
- **Early stopping** — based on relative function decrease or oracle norm
- **Parallel mini-batches** — the `t` oracle samples per step are evaluated across `n_jobs` workers (thread or process backend)
- **Efficient sampling** — Cholesky factorisation cached at construction; diagonal `B` handled without matrix inversion

---

## Quick Start

### Minimisation

```python
import numpy as np
from ZoSolvers.minimisation import ZO_gauss_min

def f(x):
    return x[0]**2 + x[1]**2 + x[0]*x[1]

x0  = np.array([5.0, -5.0])
opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10)

# Zeroth-order gradient descent
x_traj = opt.ZOGD(method="center")

# Zeroth-order extra-gradient
x_traj = opt.ZOEGm(method="center", gamma=1.0)
```

### Minimax

```python
from ZoSolvers.minimax import ZO_gauss_minmax

def f(x, y):
    return x[0]**2 + x[1]**2 - y[0]**2 - y[1]**2 + x[0]*y[1]

x0 = np.array([5.0, -5.0])
y0 = np.array([3.0, -3.0])

opt = ZO_gauss_minmax(f, x0, y0, h=1e-3, tau=1, mu=1e-8, N=10000, t=5)

# Gradient descent-ascent
x_traj, y_traj = opt.ZOGDA(method="center")

# Extra-gradient
x_traj, y_traj = opt.ZOEGmm(method="center", gamma=0.8)
```

### Sphere Oracle and Precision Matrix

```python
B = np.array([[10.0, 0.5],
              [0.5,  2.0]])

opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10,
                   B=B, oracle_type="sphere")
x_traj = opt.ZOGD(method="center")
```

### Parallel Function Evaluations

The `t` oracle samples that make up one step are independent, so they can be
evaluated concurrently. Pass `n_jobs` (`-1` = all cores):

```python
opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=32, n_jobs=-1)
x_traj = opt.ZOGD(method="center")
opt.close()          # or use the solver as a context manager
```

The worker pool is created once and reused across every iteration. Use the
solver as a context manager to shut it down automatically:

```python
with ZO_gauss_minmax(f, x0, y0, h=1e-3, mu=1e-8, N=10000, t=16, n_jobs=8) as opt:
    x_traj, y_traj = opt.ZOGDA(method="center")
```

Two backends are available:

| `backend` | Use when | Notes |
|-----------|----------|-------|
| `"thread"` *(default)* | `func` releases the GIL — NumPy-heavy models, compiled extensions, subprocess or network-backed simulators | Accepts any callable, including lambdas and closures |
| `"process"` | `func` is pure-Python and CPU-bound | `func` must be picklable (module-level, not a lambda); `np.random.seed` no longer makes runs reproducible, since each worker seeds itself |

Parallelism pays off when the cost function is expensive relative to the
scheduling overhead, the usual case for black-box simulators.

### Box-Constrained Problem

```python
proj = lambda x: np.clip(x, -3.0, 3.0)

opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10, proj=proj)
x_traj = opt.ZOGD(method="center")
```

Every iterate the solver computes is feasible, but the first row of the
returned trajectory is the `x0` you passed in, returned untouched. If `x0` lies
outside the feasible set, that first row is infeasible. Set `project_init=True`
to project it as well, so the whole trajectory is feasible:

```python
opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10, proj=proj,
                   project_init=True)
x_traj = opt.ZOGD(method="center")   # x_traj[0] == proj(x0)
```

The default is `False`, which keeps `x0` visible exactly as given. For minimax,
the same flag covers both `x0` and `y0`, each through its own projection.

---

## Solvers

| Solver | Class | Problem | Description |
|--------|-------|---------|-------------|
| `ZOGD` | `ZO_gauss_min` | Minimisation | Zeroth-order gradient descent |
| `ZOEGm` | `ZO_gauss_min` | Minimisation | Zeroth-order extra-gradient |
| `ZOGDA` | `ZO_gauss_minmax` | Minimax | Zeroth-order gradient descent-ascent |
| `ZOEGmm` | `ZO_gauss_minmax` | Minimax | Zeroth-order extra-gradient minimax |

---

## Key Parameters

| Parameter | Description |
|-----------|-------------|
| `h` | Step size |
| `mu` | Smoothing parameter for finite differences |
| `N` | Maximum number of iterations |
| `t` | Oracle samples per step (`int` or `"iteration"` for growing schedule) |
| `B` | Precision matrix shaping the perturbation distribution (`None` = identity) |
| `oracle_type` | `"gaussian"` or `"sphere"` |
| `proj` | Projection onto the feasible set (`None` = unconstrained) |
| `project_init` | Project the initial guess onto the feasible set too (default `False`) |
| `n_jobs` | Workers used to evaluate the `t` samples per step (`1` = sequential, `-1` = all cores) |
| `backend` | `"thread"` (default) or `"process"` |
| `tau` | *(Minimax)* Step-size ratio: x-step = `h/tau`, y-step = `h` |
| `gamma` | *(Extra-gradient)* Second-stride scale factor |

---

## Package Structure

```
ZoSolvers/
├── src/ZoSolvers/
│   ├── minimisation.py   # ZO_gauss_min
│   ├── minimax.py        # ZO_gauss_minmax
│   └── utils.py          # shared utilities
├── tests/
│   ├── min_test.py       # pytest suite for minimisation
│   ├── minimax_test.py   # pytest suite for minimax
│   ├── demo_min.py       # minimisation demo with plots
│   └── demo_minimax.py   # minimax demo with plots
├── Docs/
│   └── ZoSolvers_Manual.pdf
└── pyproject.toml
```

---


## License

MIT

