Metadata-Version: 2.4
Name: afor-optimizer
Version: 0.1.0
Summary: Adaptive Forgetting Optimizer for PyTorch
Author: AFOR Contributors
License-Expression: MIT
Keywords: pytorch,optimizer,deep-learning,adaptive-optimization,machine-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0
Dynamic: license-file

# AFOR

AFOR is a tensor-wise adaptive forgetting optimizer for PyTorch. It adapts
the second-moment decay coefficient from local gradient residuals, direction
consistency, and online Z-score normalization.

## Installation

```bash
pip install afor-optimizer
```

## Usage

```python
import torch
from AFOR import afor

model = torch.nn.Linear(10, 2)
optimizer = afor(
    model.parameters(),
    lr=1e-3,
    betas=(0.9, 0.999),
    beta2_min=0.99,
    dir_weight=1.0,
    weight_decay=1e-4,
)

inputs = torch.randn(16, 10)
targets = torch.randint(0, 2, (16,))
loss = torch.nn.functional.cross_entropy(model(inputs), targets)
loss.backward()
optimizer.step()
optimizer.zero_grad()
```

## Main Parameters

- `lr`: learning rate.
- `betas`: first-moment and initial second-moment coefficients.
- `beta2_min`: lower bound for the adaptive second-moment coefficient.
- `dir_weight`: weight of gradient-direction consistency.
- `eps`: numerical stability term.
- `weight_decay`: decoupled weight decay.
- `fast_ref`: use the larger fast/slow noise estimate as the noise reference.

AFOR currently supports dense gradients only.

## License

Released under the MIT License. See `LICENSE`.
