Metadata-Version: 2.4
Name: kanboost
Version: 0.0.3
Summary: Gradient boosting with Kolmogorov-Arnold Network (KAN) learners -- an interpretable alternative to tree-based boosting (XGBoost/LightGBM/CatBoost).
Author: Tuama M Hamzah
License: MIT License
        
        Copyright (c) 2026 Tuama M Hamzah
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/tuamah/kanboost
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pykan>=0.2.5
Requires-Dist: torch>=2.0
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=1.5
Requires-Dist: scikit-learn>=1.3
Requires-Dist: pyyaml>=5.1
Requires-Dist: matplotlib>=3.5
Requires-Dist: tqdm>=4.60
Dynamic: license-file

# KANBoost

**Gradient boosting with Kolmogorov-Arnold Network (KAN) learners** — an
interpretable, from-scratch alternative to tree-based boosting frameworks
(XGBoost, LightGBM, CatBoost).

Instead of decision trees as weak learners, KANBoost fits a sequence of
small, shallow [KAN](https://arxiv.org/abs/2404.19756) networks to the
pseudo-residuals of the previous stage, following the classic Friedman
(2001) gradient boosting recipe. Because each KAN edge is a learnable
univariate spline rather than an opaque weight, the resulting ensemble
exposes per-feature shape functions that are directly inspectable —
closer to a Generalized Additive Model than a black box.

> **Status: early-stage research project.** This is *not* a drop-in
> replacement for CatBoost/XGBoost in production. See
> [Benchmarks](#benchmarks) and [Honest limitations](#honest-limitations)
> below before using this for anything important.

## Why this exists

As of mid-2026, there is no widely-used, pip-installable library that
combines KAN with gradient boosting. A closely related idea was
published as **GB-KAN** (ICAART 2026), but no public code accompanies
that paper. KANBoost is an independent, from-scratch open-source
implementation of the same general idea, plus:

- automatic handling of categorical features (smoothed target-mean
  encoding, done fold-safe), instead of requiring manual one-hot encoding
- built-in early stopping on a validation set
- approximate feature importances derived from learned spline coefficients

## Install

```bash
git clone https://github.com/tuamah/kanboost.git
cd kanboost
pip install -r requirements.txt
pip install -e .
```

## Quickstart

```python
import pandas as pd
from sklearn.model_selection import train_test_split
from kanboost import KANBoostClassifier

df = pd.read_csv("your_data.csv")
X = df.drop(columns=["target"])
y = df["target"].values

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2)

model = KANBoostClassifier(
    n_estimators=100,
    learning_rate=0.2,
    kan_hidden=4,
    kan_grid=3,
    categorical_cols=["region", "plan_type"],  # optional
    early_stopping_rounds=10,
)
model.fit(X_train, y_train, eval_set=(X_val, y_val))

probs = model.predict_proba(X_val)[:, 1]
importances = model.feature_importances()
```

## Benchmarks

Preliminary results on a real-world telecom churn dataset (100K rows,
10 numeric features used, 8K-row sample for the KANBoost run due to
current training-speed limits):

| Model | Test AUC | Notes |
|---|---|---|
| CatBoost (tuned, full data, ~100 columns) | **0.6992** | production baseline |
| KANBoostClassifier (this repo, 10 features, 8K sample) | 0.64 | early prototype, untuned |
| Plain KAN (no boosting) | 0.65 | single model, same features |
| Plain MLP | 0.59–0.62 | same features |

**Read this table honestly**: KANBoost does not yet beat CatBoost on
this dataset. The goal of this repo, at this stage, is to establish a
working, extensible implementation and an honest baseline — not to claim
state-of-the-art results.

## Honest limitations

- **Speed**: each weak learner is a full KAN forward/backward pass in
  pure PyTorch. This is currently far slower per-iteration than a
  histogram-based tree split in XGBoost/CatBoost/LightGBM.
- **Tuning**: hyperparameters (`kan_grid`, `kan_hidden`, `kan_steps`,
  `learning_rate`) interact in ways that are not yet well understood;
  expect to need real tuning for your dataset.
- **Categorical encoding** is a simple smoothed target-mean encoder, not
  CatBoost's ordered boosting scheme — it can leak on small folds if not
  used carefully.
- **Missing values** are not yet handled natively; impute before fitting.

## Roadmap

See [`ROADMAP.md`](./ROADMAP.md) for the full project plan, including
planned speed optimizations (FastKAN-style RBF basis, `torch.compile`),
symbolic-formula extraction for the full ensemble, and benchmark
expansion to standard UCI datasets.

## Contributing

Issues and PRs welcome, especially:
- speed optimizations for the per-iteration KAN fit
- better categorical encoding
- benchmark results on additional public datasets

## License

MIT — see [`LICENSE`](./LICENSE).

## Citation / related work

If you use this, please also cite the KAN paper and, where relevant,
the GB-KAN paper this project is conceptually closest to:

```
Liu, Z., Wang, Y., Vaidya, S., et al. (2024). KAN: Kolmogorov-Arnold
Networks. arXiv:2404.19756.

[GB-KAN authors] (2026). Gradient Boosting with Interpretable
Kolmogorov-Arnold Networks. ICAART 2026.
```
