Metadata-Version: 2.4
Name: hornets
Version: 0.1.2
Summary: HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks
Author-email: bkolosk1 <bosec99@gmail.com>
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: matplotlib>=3.10.8
Requires-Dist: numpy>=1.24.0
Requires-Dist: scikit-learn<2.0.0,>=1.6.1
Requires-Dist: scipy>=1.17.1
Requires-Dist: torch>=2.8.0
Requires-Dist: torchaudio>=2.8.0
Requires-Dist: torchvision>=0.22.0
Requires-Dist: tqdm<5.0.0,>=4.67.1
Requires-Dist: umap-learn>=0.5.11
Description-Content-Type: text/markdown

# HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks

[![arXiv](https://img.shields.io/badge/arXiv-2501.14346-b31b1b.svg)](https://arxiv.org/abs/2501.14346) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1HxBQRLPa-j54WYU6nxj9VgwLv1ywfzJ-?usp=sharing) ![build](https://github.com/bkolosk1/hornets/actions/workflows/python-install.yml/badge.svg)  ![lint](https://github.com/bkolosk1/hornets/actions/workflows/lint.yml/badge.svg) ![test](https://github.com/bkolosk1/hornets/actions/workflows/pytest.yml/badge.svg) 




**HorNets** is a Python package implementing the HorNets architecture of the ``HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks`` paper.


![alt text](image.png)


## Installation

Follow the instructions below to install **HorNets** using your preferred method.

### Using PyPI ``stable version``

You can install **HorNets** directly from PyPI using `pip`. This is the simplest method if you just want to use the package.

```bash
pip install hornets
```

### Using PyPI ``latest version``

```bash
pip install git+https://github.com/bkolosk1/hornets.git
```

### Local Development Installation

To install locally with `uv` follow the following steps:

1. Install uv: `pip install uv`  
2. Clone the repo: `git clone git@github.com:bkolosk1/hornets.git && cd hornets`  
3. Run: `uv sync`
4. Test the installation with: `uv run python examples/examples.py`


## Usage


You can run `examples/examples.py` 

Or you can run the minimal code snippet:


```python
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from hornets import HorNetClassifier, generate_synthetic_data

# Generate synthetic data
X, y = generate_synthetic_data(
    num_features=64,
    num_instances=128,
    operation="xor"
)

# Split into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Initialize and train HorNetClassifier
classifier = HorNetClassifier(
    num_rules=256,
    exp_param=4,
    activation="polyclip",
    order=5,
    learning_rate=0.1,
    batch_size=10,
    stopping_crit=100,
    num_epochs=500,
    verbose=True
)
classifier.fit(X_train, y_train)

# Predict on test data
y_pred = classifier.predict(X_test)

# Evaluate
print("Classification Report:")
print(classification_report(y_test, y_pred))
```

### Extracting Embeddings

After training, you can extract learned representations using `transform()`:

```python
# Extract embeddings (pre-classification hidden representations)
embeddings = classifier.transform(X_test)
print(embeddings.shape)  # (n_samples, num_rules) for binary input
```

For binary inputs, embeddings are the attention-weighted combination scores over feature rules. For continuous inputs, they are the attention-reweighted features. See `examples/tox171_evaluation.py` for a full example with t-SNE, UMAP, and PCA visualization on a real dataset.


## Hyperparameters


| **Parameter**     | **Type**                | **Default** | **Description**                                                                                                                                                                                                                                                                                  | **Suggested HPO Range**              |
|-------------------|-------------------------|------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------|
| `batch_size`      | `int`                  | 4           | Number of samples per training batch.                                                                                                                                             | 4, 8, 16, 32                         |
| `num_epochs`      | `int`                  | 1000        | Maximum number of epochs (complete passes through the training data). Large values allow more training time but risk overfitting.                                                                                                 | 100, 300, 500, 1000                  |
| `learning_rate`   | `float`                | 1e-4        | Learning rate for the optimizer. Higher values speed up training but risk overshooting minima; lower values may stabilize training but at the cost of slower convergence.                                                                                                                        | 1e-5 to 1e-2                         |
| `stopping_crit`   | `int`                  | 10          | Number of consecutive epochs without improvement on validation metric to wait before stopping training early.                                                                                              | 5, 10, 15                            |
| `feature_names`   | `Optional[List[str]]`  | None        | Names of input features.                                                                                                                                                                                    | Not applicable                       |
| `num_rules`       | `int`                  | 256         | Number of rules (feature combinations) in the HorNet model. Increasing this can capture more complex patterns but may lead to higher risk of overfitting or longer training times.                                                                                                               | 64, 128, 256, 512                    |
| `activation`      | `str`                  | "polyclip"  | Activation function used in the model. Options include `"polyclip"` or `"relu"` functions.                                                 | "polyclip", "relu", "sigmoid", etc.  |
| `comb_samples_fp` | `int`                  | 48          | Number of combination samples for feature processing. Larger values may capture more interaction patterns but at higher computational cost.                                                                                                              | 16, 32, 48, 64                       |
| `exp_param`       | `int`                  | 1           | Expansion parameter for polynomial clipping within the activation. Primarily relevant for `"polyclip"` activations, controlling how inputs are processed.                                                                                                                                        | 1, 2, 3                              |
| `order`           | `int`                  | 5           | Order of feature combinations. Higher orders enable capturing more complex relationships but can drastically increase computational demands.                                                                                                                                                     | 2, 3, 5                              |
| `device`          | `str`                  | "cpu"       | Device for model training and inference. Using `"cuda"` (if available) can speed up training significantly, but `"cpu"` is a safer default.                                                                                                                                                      | "cpu", "cuda"                        |
| `random_state`    | `Optional[int]`        | None        | Seed for random number generation, ensuring reproducible results. Not usually tuned as a hyperparameter, but important for experiments where repeatability is required.                                                                                                                          | Not applicable                       |
| `verbose`         | `bool`                 | False       | If True, prints progress and debug messages. Can be helpful during development or troubleshooting but is not generally part of hyperparameter optimization.                                                                                                                                       | True or False                        |



## Citation



Cite this work as:

```
﻿@Article{Koloski2025,
author={Koloski, Boshko
and Lavra{\v{c}}, Nada
and {\v{S}}krlj, Bla{\v{z}}},
title={HorNets: learning from discrete and continuous signals with routing neural networks},
journal={Machine Learning},
year={2025},
month={Feb},
day={21},
volume={114},
number={4},
pages={101},
issn={1573-0565},
doi={10.1007/s10994-024-06673-1},
url={https://doi.org/10.1007/s10994-024-06673-1}
}
```
