Metadata-Version: 2.4
Name: rani
Version: 1.0.1
Summary: RANI: Rational Adaptive Normalizing Initialization — a novel self-normalizing activation function with learnable parameters
Author-email: Nikki Rani <nikkirani861@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/Star-nikki/rani-activation
Project-URL: Repository, https://github.com/Star-nikki/rani-activation
Project-URL: Bug Tracker, https://github.com/Star-nikki/rani-activation/issues
Keywords: activation function,self-normalizing,deep learning,neural networks,rational approximation,adaptive,RANI,machine learning,PyTorch
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=1.9.0
Requires-Dist: numpy>=1.21.0
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.6.0; extra == "tensorflow"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Dynamic: license-file

# RANI — Rational Adaptive Normalizing Initialization

**A novel self-normalizing activation function with learnable per-layer parameters.**

> **Author:** Nikki Rani

---

## What is RANI?

RANI is a new activation function that combines three properties that no
existing activation function has simultaneously:

| Property | RANI | SELU | GELU | Swish | PReLU |
|---|:---:|:---:|:---:|:---:|:---:|
| **Self-normalizing** (no BatchNorm needed) | ✓ | ✓ | ✗ | ✗ | ✗ |
| **Learnable** per-layer parameters | ✓ | ✗ | ✗ | ✗ | ✓ |
| **Zero transcendentals** on positive branch | ✓ | ✓ | ✗ | ✗ | ✓ |

---

## Formula

```
RANI(x) = λ · x · (1 + α·|x|) / (1 + |x|)    for x ≥ 0   ← rational, no exp()
RANI(x) = λ · α · (eˣ − 1)                     for x < 0   ← exponential
```

**Default parameters** — derived from self-normalizing fixed-point equations:

| Parameter | Symbol | Default value | Role |
|---|---|---|---|
| Asymmetry | α | **1.6733** | Controls negative saturation depth and positive slope |
| Scale | λ | **1.0507** | Controls overall function amplitude |

Both parameters are **learned per layer** during training via backpropagation.

---

## Install

```bash
pip install rani
```

---

## Quick Start — PyTorch

```python
import torch.nn as nn
from rani import RANI, init_rani_network

# Build a network with RANI activations
model = nn.Sequential(
    nn.Linear(784, 512),
    RANI(),                    # alpha=1.6733, lam=1.0507 — both learnable
    nn.Linear(512, 256),
    RANI(),
    nn.Linear(256, 128),
    RANI(),
    nn.Linear(128, 10),
)

# Apply the mathematically correct weight initialization for RANI
# (LeCun normal: std = 1/sqrt(fan_in) — derived from the fixed-point equations)
init_rani_network(model)

# Train normally with any optimizer
import torch
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
```

Functional version with fixed parameters:

```python
from rani import rani
import torch

x = torch.randn(32, 256)
y = rani(x)                    # alpha=1.6733, lam=1.0507 fixed
```

Check what the network learned after training:

```python
for name, module in model.named_modules():
    if isinstance(module, RANI):
        print(f"{name}: alpha={module.alpha.item():.4f}, lam={module.lam.item():.4f}")
# Each layer learns its own optimal alpha and lambda values
```

---

## Why does it work without BatchNorm?

RANI is mathematically self-normalizing. If the input to a layer has
mean 0 and variance 1, the output also has mean 0 and variance 1 —
regardless of how many layers deep you go.

This is proved via the fixed-point equations:

```
E[ RANI(Z) ]   = 0    ← mean is preserved as zero
E[ RANI(Z)² ]  = 1    ← variance is preserved as one
```

where Z ~ N(0, 1). The initialization values α = 1.6733, λ = 1.0507
are the exact numerical solution to this system.

After initialization, α and λ are free to be learned — each layer
adapts to its own optimal values. This is the key advantage over SELU,
where these values are permanently fixed.

---

## FLOP Count — Why the Positive Branch Is Cheap

The positive branch uses only:

| Operation | Count |
|---|---|
| Multiply | 3 |
| Add | 2 |
| Divide | 1 |
| **Transcendentals (exp, erf, sigmoid)** | **0** |

Compare with GELU (needs `erf`), Swish (needs `exp` via sigmoid),
and Mish (needs 2 × `exp` + `tanh`). RANI's positive branch is
faster on both CPU and GPU for this reason.

---

## Running the Tests

```bash
cd rani-activation
pytest tests/test_torch.py -v
```

Expected output: **14 passed**

---

## Mathematical Foundation

The complete derivation covers 6 parts: formal definition, regularity
proofs, self-normalizing fixed-point equations, stability analysis via
the Banach contraction theorem, and FLOP count comparison.

**Key results:**

| Result | Description |
|---|---|
| Theorem 3.1 | RANI is continuous on all of ℝ |
| Theorem 3.3 | RANI is Lipschitz with L = λ·max(1, α) |
| Theorem 3.4 | RANI saturates at −λα as x → −∞ |
| Result 4.1  | α* = 1.6733, λ* = 1.0507 (self-normalizing init) |
| Theorem 5.1 | Fixed point (0, 1) is stable attractor, ρ(J) < 1 |

Full mathematical details and paper citation coming soon.

---

## Package Structure

```
rani/
├── __init__.py         ← package entry point
├── torch_rani.py       ← PyTorch implementation
└── tf_rani.py          ← TensorFlow implementation (pending Python 3.14 support)
```

---

## Citation

Citation details will be added after paper publication.

---

## License

Apache License 2.0 — see [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) for details.

## Author

**Nikki Rani**
[GitHub](https://github.com/Star-nikki) | [PyPI](https://pypi.org/project/rani/)
