Metadata-Version: 2.4
Name: numoptlib
Version: 0.1.1
Summary: An optimization library with gradient descent and Newton methods
Author-email: Kripa Vyas <kripa@rogers.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/Kripa-Vyas03/numoptlib
Project-URL: Issues, https://github.com/Kripa-Vyas03/numoptlib/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Requires-Dist: numpy>=1.21
Dynamic: license-file

# numoptlib

A Python library implementing classical numerical optimization algorithms from scratch. This project was developed alongside graduate coursework in computational geophysics with the goal of providing clean, well-tested implementations of gradient-based optimization methods, convergence diagnostics, and benchmarking tools.

## Motivation

Most scientific Python code treats optimization as a black box: you pass a function to `scipy.optimize.minimize` and receive a solution. This library is an attempt to look inside that box by implementing optimization algorithms from first principles and exploring the tradeoffs between different methods.

The implementations are based primarily on:

> Nocedal, J. & Wright, S. (2006). *Numerical Optimization* (2nd ed.)

## Implemented Methods

| Method                     | Type          | Line Search     | Convergence |
| -------------------------- | ------------- | --------------- | ----------- |
| Gradient Descent           | Unconstrained | Strong Wolfe    | O(1/k)      |
| Momentum                   | Unconstrained | Strong Wolfe    | O(1/k)      |
| Adam                       | Unconstrained | None            | Adaptive    |
| Newton                     | Unconstrained | Strong Wolfe    | Quadratic   |
| BFGS                       | Unconstrained | Strong Wolfe    | Superlinear |
| Projected Gradient Descent | Constrained   | Strong Wolfe    | O(1/k)      |
| Augmented Lagrangian       | Constrained   | BFGS Subproblem | Linear      |

## Installation

Install directly from PyPI:

```bash
pip install numoptlib
```

Or install the development version:

```bash
git clone https://github.com/Kripa-Vyas03/numoptlib
cd numoptlib
pip install -e .
```

## Example Usage

The example below minimizes the Rosenbrock function using BFGS.

```python
import numpy as np
from numoptlib.unconstrained.bfgs import bfgs

def rosenbrock(x):
    return 100 * (x[1] - x[0]**2)**2 + (1 - x[0])**2

def rosenbrock_grad(x):
    return np.array([
        -400 * x[0] * (x[1] - x[0]**2) - 2 * (1 - x[0]),
        200 * (x[1] - x[0]**2)
    ])

x0 = np.array([-1.1, 1.1])

result = bfgs(
    rosenbrock,
    rosenbrock_grad,
    x0,
    max_iter=2000
)

print(result.x)
print(result.fun)
```

Typical output:

```text
Solution: [0.99999998 0.99999996]
Function value: 0.000000
Converged: True
Iterations: 34
```

Additional examples, including visualization and benchmarking scripts, can be found in the `examples/` directory.

## Benchmarks

The algorithms were benchmarked on the Rosenbrock function and several quadratic optimization problems.

### Rosenbrock Function

| Start Point | Gradient Descent | Newton | BFGS | Momentum | Adam  |
| ----------- | ---------------- | ------ | ---- | -------- | ----- |
| [-1, 1]     | 847              | 20     | 1    | 1        | 1034  |
| [0, 0]      | >2000            | 13     | 21   | 198      | >2000 |
| [-1.1, 1.1] | >2000            | 21     | 16   | 339      | 1195  |

Additional benchmark results are available in the repository documentation.

## Running Tests

Run the test suite with:

```bash
python -m pytest tests/ -v
```

The tests include:

1. Correctness tests
2. Result object validation
3. Method-specific convergence tests

## Dependencies

* Python >= 3.9
* NumPy
* pytest (testing)
* Matplotlib (examples and benchmarking)

## References

Nocedal, J., & Wright, S. (2006). *Numerical Optimization*. Springer.

Boyd, S., & Vandenberghe, L. (2004). *Convex Optimization*. Cambridge University Press.
