Metadata-Version: 2.1
Name: MomSPS
Version: 1.0.0
Summary: Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
Home-page: https://github.com/dimitris-oik/MomSPS
Author: Dimitris Oikonomou
Author-email: doikono1@jh.edu
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8.0
Description-Content-Type: text/markdown
License-File: LICENSE

# MomSPS
This is the official repository for the code used to run the experiments in the paper that proposed the MomSPS optimizer. The optimizer is implemented in PyTorch. 


**Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance**\
*D. Oikonomou, N. Loizou*


## Installation

You can install the package with

```
pip install momsps
```


## Usage

Import the optimizers in Python with

``` python
from momsps import MomSPS
opt = MomSPS(model.parameters(), lr=1)
```
or

``` python
from momsps import MomSPS_smooth
opt = MomSPS_smooth(model.parameters(), lr=1)
```

**Important: you only need to adapt one line of your training script, described below.** 
MomSPS needs access to the value of the (mini-batch) loss in the ``.step()`` method. The easiest way to do this is

``` python
loss = YOUR_LOSS_FUNCTION()
loss.backward()
opt.step(loss=loss)           # the line you need to change
```


## Citation

If you use MomSPS, please cite

```
@article{Oikonomou2024,
  author        = {Dimitris Oikonomou and Nicolas Loizou},
  title         = {Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance},
  year          = {2024},
}
```

