Metadata-Version: 2.4
Name: whc
Version: 1.0.0
Summary: Wormhole Hyperconnections — Drop-in replacement for residual connections
Home-page: https://github.com/fardinsabid/whc
Author: Fardin Sabid
Author-email: Fardin Sabid <fardinsabid@proton.me>
Maintainer-email: Fardin Sabid <fardinsabid@proton.me>
License: MIT
Project-URL: Homepage, https://github.com/fardinsabid/whc
Project-URL: Repository, https://github.com/fardinsabid/whc
Project-URL: Bug Reports, https://github.com/fardinsabid/whc/issues
Project-URL: Documentation, https://github.com/fardinsabid/whc#readme
Keywords: deep-learning,pytorch,residual,hyperconnection,wormhole,ai,machine-learning
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Wormhole Hyperconnections (WHC)

[![PyPI version](https://badge.fury.io/py/whc.svg)](https://badge.fury.io/py/whc)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![PyTorch](https://img.shields.io/badge/PyTorch-2.0+-red.svg)](https://pytorch.org/)

**A drop-in replacement for the standard residual connection `x = x + layer(x)`.**

WHC replaces the single residual path with **n parallel lanes** and **learnable mixing matrices**, delivering:

- **6–15% faster training** than standard residual
- **Lower final loss** on benchmark tasks
- **Proven stability** — spectral norm ≤ 1
- **82 parameters** for the mixing matrix (vs mHC's 32,768)
- **2× faster** than mHC in wall-clock time

---

## Quick Start

```python
from whc import WormholeHyperconnection

whc = WormholeHyperconnection(
    dim=512,
    expansion_rate=4,
    num_layers=12,
)

x = whc.expand(x)                      # (B, T, dim) -> (B, T, n, dim)
for i, layer in enumerate(transformer_blocks):
    x = whc(x, layer, layer_idx=i)      # replaces `x = x + layer(x)`
x = whc.reduce(x)                      # (B, T, n, dim) -> (B, T, dim)
```

---

## Installation

### From PyPI (Recommended)

```bash
pip install whc
```

### From Source

```bash
git clone https://github.com/fardinsabid/wHC.git
cd wHC
pip install -e .
```

### Requirements

- Python >= 3.8
- PyTorch >= 2.0.0

---

## Key Features

| Feature | Description |
|---------|-------------|
| **Drop-in replacement** | Replace `x = x + layer(x)` with `x = whc(x, layer=layer)` |
| **n parallel lanes** | Instead of 1 fixed path |
| **Learnable mixing** | Network learns how lanes interact |
| **Proven stability** | Spectral norm ≤ 1 guarantees no explosion |
| **No iterations** | Closed-form kernel, no Sinkhorn-Knopp |
| **Minimal overhead** | 82 parameters for the mixing matrix |
| **GPU ready** | Runs on CUDA, CPU, MPS |

---

## Benchmarks

### Signal Gain vs Depth

![Stability Comparison](assets/whc_stability.png)

| Method | Mean Final Gain | Max Final Gain |
|--------|-----------------|----------------|
| Unconstrained HC | 9.54×10⁵ | 4.77×10⁶ |
| mHC (DeepSeek) | 0.42 | 0.68 |
| **WHC** | **0.48** | **0.63** |

**WHC and mHC stay bounded; unconstrained HC explodes.**

### Training Loss

![Training Loss](assets/whc_training.png)

| Method | Final Loss | Time (s) |
|--------|------------|----------|
| Standard Residual | 0.3765 | 0.91 |
| mHC-static | 0.6966 | 11.01 |
| **WHC** | **0.3590** | **5.54** |

**WHC achieves lower loss and is 2× faster than mHC.**

### Parameter Count

| Component | Parameters |
|-----------|------------|
| WormholeKernel (n=8) | **82** |
| mHC H_res generator (dim=512, n=8) | 32,768 |

**WHC uses ~400× fewer parameters for the mixing matrix.**

---

## Usage

### 1. Basic Usage (Manual Integration)

```python
from whc import WormholeHyperconnection

# Create WHC wrapper
whc = WormholeHyperconnection(dim=512, expansion_rate=4, num_layers=12)

# Forward pass
x = whc.expand(x)
for i, layer in enumerate(transformer_blocks):
    x = whc(x, layer, layer_idx=i)
x = whc.reduce(x)
```

### 2. Single Layer

```python
whc = WormholeHyperconnection(dim=512, expansion_rate=4, num_layers=1)

x = whc.expand(x)
x = whc(x, layer, layer_idx=0)
x = whc.reduce(x)
```

### 3. With Pre-computed Layer Output

```python
layer_out = layer(x)
x = whc(x, layer_out=layer_out)
```

### 4. Custom Configuration

```python
whc = WormholeHyperconnection(
    dim=768,
    expansion_rate=8,           # Number of parallel lanes
    num_layers=24,              # Number of layers in your stack
    manifold_dim=3,             # Dimension of the wormhole manifold
    shared_kernel=False,        # Share kernel across layers?
)
```

---

## Examples

| Example | Description | Run |
|---------|-------------|-----|
| `simple_mlp.py` | MLP with WHC replacing residuals | `python examples/simple_mlp.py` |
| `transformer_whc.py` | Transformer with WHC replacing residuals | `python examples/transformer_whc.py` |
| `resnet_whc.py` | ResNet with WHC replacing residuals | `python examples/resnet_whc.py` |

---

## Tests

```bash
pytest tests/ -v
```

| Test File | What It Tests |
|-----------|---------------|
| `test_kernel.py` | Kernel stability, spectral norm ≤ 1 |
| `test_whc.py` | Shape, forward pass, parameters |
| `test_gradients.py` | Gradient flow, no NaNs, no explosion |

All 21 tests passed ✅

---

## Research Paper

The full mathematical derivation, stability proof, and experimental results are in:

📄 **[papers/whc.pdf](https://github.com/fardinsabid/wHC/blob/main/papers/wHC.pdf)**

**Key contributions:**
- **Spectral normalization** (closed-form, no iteration)
- **Wormhole kernel** (physics-inspired, interpretable)
- **82 parameters** for H_res (vs mHC's 32,768)
- **2× faster** than mHC
- **Better loss** than standard residual

---

## Project Structure

```
whc/
├── whc.py                          # Core implementation
├── README.md                       # This file
├── LICENSE                         # MIT License
├── setup.py                        # Package installer
├── pyproject.toml                  # Build config
├── requirements.txt                # Dependencies
├── examples/
│   ├── simple_mlp.py               # MLP with WHC
│   ├── transformer_whc.py          # Transformer with WHC
│   └── resnet_whc.py               # ResNet with WHC
├── papers/
│   └── whc.pdf                     # Research paper
├── assets/
│   ├── whc_stability.png           # Stability comparison
│   └── whc_training.png            # Training loss curves
└── tests/
    ├── test_kernel.py              # Kernel unit tests
    ├── test_whc.py                 # WHC unit tests
    └── test_gradients.py           # Gradient stability tests
```

---

## Citation

If you use WHC in your research, please cite:

```bibtex
@misc{sabid2026whc,
  author = {Fardin Sabid},
  title = {Wormhole Hyperconnections: A Physics-Inspired Framework for Stable Deep Residual Learning},
  year = {2026},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/fardinsabid/wHC}}
}
```

---

## License

MIT License — see [LICENSE](LICENSE) for details.

---

## Author

**Fardin Sabid**

- GitHub: [@fardinsabid](https://github.com/fardinsabid)
- Research: Deep Learning Optimization, Physics-Inspired Architectures

---

## Acknowledgments

WHC builds on the foundation of:

- **He et al. (2016)** — Residual connections
- **Zhu et al. (2025)** — Hyper-Connections (HC)
- **Xie et al. (2025/2026)** — Manifold-Constrained Hyper-Connections (mHC)
- **Kipf & Welling (2017)** — Graph Convolutional Networks

---

## Star Us

If you find WHC useful, please ⭐ star the repository!

---

**The standard residual was 2016. WHC is 2026.**
