Metadata-Version: 2.4
Name: mizoGrad
Version: 0.1.0
Summary: Very Fast Gradient-Based Optimization Algorithm
Author-email: Mazen Alamir <mazen.alamir@grenoble-inp.fr>
Project-URL: HomePage, https://github.com/mazenalamir/mizoGrad
Project-URL: Issues, https://github.com/mazenalamir/mizoGrad/issues
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: numpy>=2.4.6

# mizoGrad: Search & Accelerate Gradient-Based Algorithm

## Description 

This module is dedicated to the implementation of the Gradient-based box constrained optimization
proposed in [this paper](https://arxiv.org/abs/2607.14600).

This algorithm combines the following features: 

- Line search along the gradient line 
- Line search along the acceleration path 
- Provable asymptotic convergence to a solution meeting the KKT-optimality conditions.

The abobe mentioned [paper](https://arxiv.org/abs/2607.14600) shows that the algorithm outperfoms 
almost all existing gradient-based methods on a set of benchmark problems proposed in the
 [kaggle repository](https://www.kaggle.com/datasets/mazenalamir/gradient-based-optimization-collection-of-problems).

## Installation 

```terminal 
pip install mizoGrad
```

## Input arguments 

The user needs to provide the following **mandatory input arguments**:

- The first, say `cost`, represents the cost functiont to be minimized 
- The second, say `cost_gradient`,  is the gradient of the same cost funciton
- The number of decision variable `nx`

> **Note:** The names `cost` and `cost_gradient` are example, only their key fields 
> are detremined which are `f` and `g` resptively (see the example below).

The complete list of **input parameters** (including all those with defaults values) is given on the following table:

| Parameter   | Type     |    Default    | Description                          |
|-------------|----------|:-------------:|--------------------------------------|
| `f`         | callable |       —       | Cost function to minimize            |
| `g`         | callable |       —       | Gradient of the cost function        |
| `nx`        | int      |       —       | Number of decision variables         |
| `xmin`      | ndarray  | `[-inf] * nx` | Lower bound on the decision variable |
| `xmax`      | ndarray  | `[+inf] * nx` | Upper bound on the decision variable |
| `ng`        | int      |      `5`      | Number of exploration points         |
| `alpha_min` | float    |     `-8`      | lower initial exponent on the step   |
| `alpha_max` | float    |      `0`      | Higher initial exponent on the step  |
| `ng`        | int      |      `5`      | Number of exploration points         |
| `fast_min`  | float    |    `-0.2`     | Maximum number of iterations         |
| `fast_max`  | float    |      `1`      | Maximum number of iterations         |
| `rho_adapt` | float    |    `0.05`     | Adaptation ratio                     |
| `eta`       | float    |  `10^{-16}`   | Small step (<1/L)                    |


## Returned solution 

The returned solution is a **dictionary** with the following format:

| Dictionary key | Type of value | Description                                              |
|----------------|---------------|----------------------------------------------------------|
| `xopt`         | ndarray       | The best solution found                                  |
| `fopt`         | float         | The corresponding best cost function value               |
| `normG`        | float         | The norm of the gradient at solution                     |
| `lesalpha_min` | ndarray       | The sequence of values taken by `alpha_min`              |
| `lesalpha_max` | ndarray       | The sequence of values taken by `alpha_max`              |
| `traj`         | ndarray       | The sequence of values of the cost                       |
| `traj_c`       | ndarray       | The sequence of step sizes on the acceleration direction |


## Example of use

```python
import numpy as np
from mizoGrad import GradOptimizer
from time import time

#from SaA import GradOptimizer

np.set_printoptions(formatter={'float': lambda x: "{0:0.4f}".format(x)})

# Define the cost function and the gradient

def cost(x, a=10, b=2, m=1):

    f = np.power((x[0]-a)*x[1] + b * x[2] ** 3, 2*m)
    return f

def cost_gradient(x, a=10, b=2, m=1):
    term = 4 * np.power((x[0]-a)*x[1] + b * x[2] ** 3, 2*m-1)
    g = np.array([
        term * x[1],
        term * (x[0]-a),
        term * (3 * b * x[2] ** 2)
    ])
    return g


x0 = 2*np.array([1,1,1])

xmin = np.array([-5] * len(x0))
xmax = np.array([+5] * len(x0))

s2a = GradOptimizer(
    f=cost,
    g=cost_gradient,
    nx = 3,
    xmin=xmin,
    xmax=xmax,
    ng=5,
    alpha_min=-8,
    alpha_max=0,
    fast_min=-0.2,
    fast_max=1.0,
    rho_adapt=0.05,
    eta=1e-16,
)

# set the dictionary argument used the cost function and gradient 

kwargs = dict(
    a=3,
    b=1,
    m=1,
)

# solve the problem

t1 = time()
R = s2a.solve(x0=x0, kwargs=kwargs, maxIter=20, epsG=1e-8, display=True)
cpu = time()-t1

print('solution: ', np.array(R['xopt'], dtype=float))
print('best cost : ', cost(s2a.y, **kwargs))
print('cpu = ', cpu)

```

which gives the following results:

```
value 9.374756801 normg=292.957 | alpha_min=-8.0 | alpha_max=-0.4
value 3.931283917 normg=31.930 | alpha_min=-8.0 | alpha_max=-0.78
value 1.109572100 normg=17.759 | alpha_min=-7.962 | alpha_max=-0.419
value 0.000013588 normg=6.499 | alpha_min=-7.922 | alpha_max=-0.04185
value 0.000000666 normg=0.066 | alpha_min=-7.922 | alpha_max=-0.4359
value 0.000000029 normg=0.015 | alpha_min=-7.922 | alpha_max=-0.8102
value 0.000000000 normg=0.003 | alpha_min=-7.922 | alpha_max=-1.166
value 0.000000000 normg=0.000 | alpha_min=-7.922 | alpha_max=-1.504
value 0.000000000 normg=0.000 | alpha_min=-7.922 | alpha_max=-1.825
value 0.000000000 normg=0.000 | alpha_min=-7.89 | alpha_max=-1.52
value 0.000000000 normg=0.000 | alpha_min=-7.89 | alpha_max=-1.838
value 0.000000000 normg=0.000 | alpha_min=-7.859 | alpha_max=-1.536
value 0.000000000 normg=0.000 | alpha_min=-7.859 | alpha_max=-1.852
solution:  [1.7020 -1.2326 -1.1696]
best cost :  8.860672896017431e-21
cpu =  0.0008881092071533203
```

## Citing mizoGrad

```
@misc{alamir2026nonlinearmodelpredictivecontrol,
      title={A Nonlinear Model Predictive Control Perspective on Gradient-Based Optimization: A New Efficient, Parameter-Free and Provably Stable Algorithm}, 
      author={Mazen Alamir},
      year={2026},
      eprint={2607.14600},
      archivePrefix={arXiv},
      primaryClass={cs.CE},
      url={https://arxiv.org/abs/2607.14600}, 
}
```
