Metadata-Version: 2.4
Name: kan-mlp-hybrid
Version: 0.1.1
Summary: Hybrid Kolmogorov-Arnold Multi-Layer Perceptrons for PyTorch
Author: Khalfoun Mohamed El Mehdi
License: MIT
Project-URL: Homepage, https://github.com/useCallback/kan-mlp-hybrid
Project-URL: Bug Tracker, https://github.com/useCallback/kan-mlp-hybrid/issues
Project-URL: Documentation, https://github.com/useCallback/kan-mlp-hybrid#readme
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: numpy>=1.21.0
Requires-Dist: scikit-learn>=1.0.0
Requires-Dist: torchvision>=0.15.0
Requires-Dist: datasets>=2.0.0
Requires-Dist: transformers>=4.0.0
Requires-Dist: matplotlib>=3.5.0
Requires-Dist: tqdm>=4.60.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: twine>=4.0.2; extra == "dev"
Requires-Dist: build>=0.10.0; extra == "dev"
Requires-Dist: nbformat>=5.0.0; extra == "dev"
Dynamic: license-file

<div align="center">

# 🧠 KAN-MLP Hybrid Architectures

[![PyPI version](https://img.shields.io/pypi/v/kan-mlp-hybrid.svg)](https://pypi.org/project/kan-mlp-hybrid/)
[![Python Versions](https://img.shields.io/pypi/pyversions/kan-mlp-hybrid.svg)](https://pypi.org/project/kan-mlp-hybrid/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Hybrid Kolmogorov-Arnold Multi-Layer Perceptrons for PyTorch**

</div>

## 📖 Overview

`kan-mlp-hybrid` is a PyTorch library that bridges the gap between the compositional efficiency of Multi-Layer Perceptrons (MLPs) and the univariate approximation power of Kolmogorov-Arnold Networks (KANs). 

While standard MLPs use fixed activation functions (like ReLU or SiLU) and learnable linear weights, and pure KANs use learnable spline-based activations on edges with no linear weights, **Hybrid KAN-MLPs** combine both. This allows the network to learn arbitrary univariate activation functions per neuron while maintaining the efficient high-dimensional feature routing of dense linear layers.

### Key Features
* 🚀 **Drop-in PyTorch Modules**: Easily replace `nn.Linear` or `nn.Sequential` with our hybrid layers.
* 📈 **Superior Parameter Efficiency**: Decouples the spline grid size from the quadratic width term, avoiding the $O(N^2LG)$ parameter explosion of pure KANs.
* 🧠 **Multiple Architectures**: Includes LA-MLP (Learnable Activation), PPH-Net (Parallel Pathway), and MoE-KAN-MLP (Mixture-of-Experts).
* 📊 **Benchmarked**: Proven to outperform standard MLPs on mixed-characteristic functions, tabular data, and time-series forecasting.

---

## 🛠️ Installation

You can install the package directly from PyPI:

```bash
pip install kan-mlp-hybrid
```

Or install from source for the latest updates:

```bash
git clone https://github.com/useCallback/kan-mlp-hybrid.git
cd kan-mlp-hybrid
pip install -e .
```

---

## 🚀 Quick Start

Using the **Learnable Activation MLP (LA-MLP)** is as simple as using a standard PyTorch MLP.

```python
import torch
from kan_mlp_hybrid import LAMLP

# Initialize an LA-MLP model
# in_features: 5, hidden layers: [64, 64], out_features: 1
model = LAMLP(
    in_features=5,
    hidden_features=[64, 64],
    out_features=1,
    num_layers=3,
    grid_size=5,       # Number of B-spline grid intervals
    spline_order=3     # Order of the piecewise polynomial
)

# Forward pass
x = torch.randn(32, 5)
y = model(x)

print(y.shape) # Output: torch.Size([32, 1])
```

### Available Architectures

1. **`LAMLP`**: Replaces fixed activation functions in standard MLPs with learnable, spline-based univariate functions. Best overall performance and efficiency.
2. **`PPHNet`**: Processes inputs through parallel KAN and MLP pathways, fusing them via a learned gating mechanism.
3. **`MoEKANMLP`**: A sparsely gated architecture containing a heterogeneous pool of KAN and MLP experts.
4. **`PureKAN`**: A baseline implementation of a pure Kolmogorov-Arnold Network.

---

## 📓 Interactive Colab Notebooks

We provide ready-to-run Google Colab notebooks demonstrating how to use `kan-mlp-hybrid` across various domains:

* [**Tabular Data (Classification & Regression)**](colab_notebooks/01_Tabular_Tasks.ipynb): Breast Cancer & California Housing datasets using `LAMLP` and `PPHNet`.
* [**Computer Vision**](colab_notebooks/02_Computer_Vision.ipynb): MNIST digit classification using `MoEKANMLP`.
* [**Natural Language Processing**](colab_notebooks/03_NLP_Text_Classification.ipynb): AG News text classification using TF-IDF and `PureKAN`.

---

## 🔬 Scientific Background

This library is based on the theoretical synthesis of the Universal Approximation Theorem (underpinning MLPs) and the Kolmogorov-Arnold representation theorem (underpinning KANs). 

By utilizing the `LALinear` layer, the linear transformations $W$ efficiently handle high-dimensional basis projection, while the learnable activations $\phi$ achieve exponential convergence on the projected 1D manifolds.

For a deep dive into the mathematical formulations, parameter complexity analysis, and architectural brainstorming that led to this package, please refer to the `RESEARCH_NOTES.md` and `PAPER_DRAFT.md` files in the GitHub repository.

---

## 👨‍💻 Author

**Khalfoun Mohamed El Mehdi**
* GitHub: [@useCallback](https://github.com/useCallback)
* PyPI: [FastFourierTransform](https://pypi.org/user/FastFourierTransform/)

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
