Metadata-Version: 2.4
Name: alrelu-keras
Version: 0.1.1
Summary: ALReLU activation package for Keras with fixed and trainable alpha variants.
Author: Stamatis
License-Expression: MIT
Project-URL: Repository, https://github.com/stamatis/ALReLU
Project-URL: Issues, https://github.com/stamatis/ALReLU/issues
Project-URL: Homepage, https://github.com/stamatis/ALReLU/tree/main
Keywords: keras,activation,deep-learning,alrelu
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: keras>=3.0.0
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.15.0; extra == "tensorflow"
Provides-Extra: dev
Requires-Dist: build>=1.2.2; extra == "dev"
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: twine>=5.1.0; extra == "dev"
Dynamic: license-file

# alrelu-keras

PyPI-installable Keras package for the ALReLU activation with 2 variants:

1. `ALReLU` (default): current behavior from this project, with fixed `alpha=0.01`.
2. `TrainableALReLU`: same formula with trainable `alpha`.

Formula used by both variants:

`ALReLU(x, alpha) = max(abs(alpha * x), x)`

## Reference

ALReLU paper:

`ALReLU: A different approach on Leaky ReLU activation function to improve Neural Networks Performance`

https://arxiv.org/abs/2012.07564

## Installation

```bash
pip install alrelu-keras
```

Keras requires a backend. For the common TensorFlow backend:

```bash
pip install alrelu-keras[tensorflow]
```

From source:

```bash
pip install .
```

## Training Examples (MNIST)

Scripts:

- Keras + TensorFlow: `scripts/train_mnist_keras_tf.py`
- PyTorch: `scripts/train_mnist_torch.py`

Direct links:

- https://github.com/stamatis/ALReLU/blob/main/scripts/train_mnist_keras_tf.py
- https://github.com/stamatis/ALReLU/blob/main/scripts/train_mnist_torch.py

Run commands (from repo root):

```bash
# Keras + TensorFlow
python scripts/train_mnist_keras_tf.py --epochs 5 --variant fixed --alpha 0.01
python scripts/train_mnist_keras_tf.py --epochs 5 --variant learnable --alpha 0.01

# PyTorch
python scripts/train_mnist_torch.py --epochs 5 --variant fixed --alpha 0.01
python scripts/train_mnist_torch.py --epochs 5 --variant learnable --alpha 0.01
```

## Usage

### 1) Default ALReLU (current implementation behavior)

```python
import keras
from alrelu_keras import ALReLU, alrelu

# As activation function
x = keras.ops.array([-2.0, -1.0, 0.0, 1.0], dtype="float32")
y = alrelu(x)  # alpha=0.01 by default

# As Keras layer
model = keras.Sequential(
    [
        keras.layers.Input(shape=(32,)),
        keras.layers.Dense(64),
        ALReLU(),  # fixed alpha
        keras.layers.Dense(10),
    ]
)
```

### 2) Trainable alpha variant

```python
import keras
from alrelu_keras import TrainableALReLU

model = keras.Sequential(
    [
        keras.layers.Input(shape=(32,)),
        keras.layers.Dense(64),
        TrainableALReLU(alpha_initializer=0.01),  # alpha is trainable
        keras.layers.Dense(10),
    ]
)
```

## Serialization / model loading

Both `ALReLU`, `TrainableALReLU`, and `alrelu` are registered as Keras-serializable objects.
You can save/load models without manually passing `custom_objects` in standard Keras workflows.

## Development

Install dev tools:

```bash
pip install -e .[dev]
```

Run tests:

```bash
pytest
```

Build package:

```bash
python -m build
```

## License

MIT License. See `LICENSE`.
