Metadata-Version: 2.4
Name: mpsboost
Version: 0.3.0
Summary: Lightweight gradient boosting for Apple Silicon with an MPS/Metal backend
Keywords: apple-silicon,catboost,c-plus-plus,extra-trees,gbdt,gradient-boosting,machine-learning,metal,mps,random-forest,tree-models
Author: MPSBoost contributors
License-Expression: Apache-2.0
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Project-URL: Homepage, https://github.com/billzi2016/MPSBoost
Project-URL: Repository, https://github.com/billzi2016/MPSBoost
Project-URL: Issues, https://github.com/billzi2016/MPSBoost/issues
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Provides-Extra: test
Requires-Dist: numpy>=1.24; extra == "test"
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: scikit-learn>=1.4; extra == "test"
Description-Content-Type: text/markdown

# MPSBoost

<p align="center">
  <img src="assets/icons/mpsboost-icon.png" alt="MPSBoost icon" width="160">
</p>

[![PyPI](https://img.shields.io/pypi/v/mpsboost)](https://pypi.org/project/mpsboost/)
[![Python](https://img.shields.io/pypi/pyversions/mpsboost)](https://pypi.org/project/mpsboost/)
[![README](https://img.shields.io/badge/README-English%20%7C%20%E4%B8%AD%E6%96%87-blue)](#mpsboost)

MPSBoost is an early-stage gradient boosting project for Apple Silicon. Its current accelerated backend uses custom Metal compute kernels for squared-error gradients and two-stage histogram construction while keeping one deterministic tree-building implementation shared with the CPU oracle.

中文：MPSBoost 是面向 Apple Silicon 的树模型加速项目，目标是在低依赖、低权限、易安装的前提下，为 GBDT、分类树、随机森林等模型提供统一的 CPU/MPS 后端。

> **Development status:** `0.3.0` is the v2 arboretum milestone. It extends the original MPS histogram engine with sklearn-style classifiers, native CPU multiclass softmax, random forests, ExtraTrees, decision trees, CatBoost-like numeric estimators, advanced regression objectives, feature explanations, CPU-suitable anomaly/ranking estimators, explicit environment guidance, and documented backend-selection boundaries.

## Project origin

MPSBoost was started by a Purdue CS PhD student working across algorithms, systems, AI, compilers, and formal verification. The immediate motivation is practical: Apple Silicon has a strong GPU stack, but common tree-based machine learning workloads still lack a simple, fast, low-permission MPS-accelerated path.

The project is built with an SDD workflow. Work is split into clear stages: first validate real MPS/Metal kernels and runtime behavior, then lock the specs and product requirements, then settle the technical stack, and finally execute the task list against those specs. Specs are treated as the project contract, not as after-the-fact notes.

For AI agents and automation, [mps_boost_skill.md](mps_boost_skill.md) is the canonical usage
entry point. It describes the complete target usage pattern, import style, estimator names,
backend policy, sklearn model selection, model persistence, diagnostics, and implementation
constraints.

## Installation

```bash
python -m pip install mpsboost
```

Accelerated releases provide prebuilt Apple Silicon wheels; normal users will not need a heavyweight framework, package manager, CMake, or a local Metal shader compiler.

## Estimator-style API

```python
import mpsboost as mb

model = mb.GradientBoostingRegressor(
    n_estimators=200,
    learning_rate=0.05,
    max_depth=6,
    device="mps",
)

model.fit(X_train, y_train)
prediction = model.predict(X_test)
importance = model.feature_importances_
permutation = model.permutation_importance(X_test, y_test)
model.save_model("model.mb")

restored = mb.GradientBoostingRegressor(device="mps")
restored.load_model("model.mb")
```

`MPSBoostRegressor` remains available as a backwards-compatible project-branded alias for
the same implementation.

`fit(..., sample_weight=...)` and `score(..., sample_weight=...)` are supported by the shared
CPU/MPS training path. Weights are applied to native gradients, Hessians, split gains, leaf values,
and default estimator scores rather than handled as a Python-only post-processing step.

## sklearn model selection

MPSBoost estimators are designed to follow the sklearn estimator protocol, so users should be
able to use the standard sklearn model-selection stack instead of learning a project-specific
search API.

```python
from sklearn.model_selection import GridSearchCV
import mpsboost as mb

search = GridSearchCV(
    mb.GradientBoostingRegressor(device="auto"),
    param_grid={
        "max_depth": [3, 6, 9],
        "learning_rate": [0.03, 0.1],
        "n_estimators": [100, 300],
    },
    cv=3,
    n_jobs=2,
)

search.fit(X_train, y_train)
best_model = search.best_estimator_
```

The same direction applies to `RandomizedSearchCV`, `cross_val_score`, and classifier estimators.
Current regressors and classifiers expose `get_params()`, `set_params()`, `fit()`, `predict()`,
`predict_proba()` where applicable, and default `score()` methods needed by standard sklearn
search utilities. Real-world Iris acceptance includes a standard `GridSearchCV` multiclass run.

Multiprocessing is supported through sklearn/joblib at the outer search level. CPU jobs can run
in multiple processes. MPS jobs should be scheduled more conservatively: several Python workers
competing for one Apple GPU can be slower than one well-sized GPU worker because command queues,
unified memory bandwidth, and synchronization overhead become the bottleneck. The intended policy
is to let `device="auto"` choose CPU for small search jobs and reserve MPS for workloads where the
measured tree hot path is large enough.

## Tree estimator names

The primary public API uses concise sklearn-style estimator names, so users can usually switch
libraries by changing the import and keeping familiar model names.

| Model family | Primary names | Status |
| --- | --- | --- |
| Histogram gradient boosting | `GradientBoostingRegressor` | Available |
| Histogram gradient boosting classification | `GradientBoostingClassifier` | Available for binary labels and multiclass one-vs-rest |
| Random forest | `RandomForestRegressor`, `RandomForestClassifier` | Available |
| Extra trees | `ExtraTreesRegressor`, `ExtraTreesClassifier` | Available |
| Single decision tree | `DecisionTreeRegressor`, `DecisionTreeClassifier` | Available |
| CatBoost-like ordered boosting | `CatBoostRegressor`, `CatBoostClassifier` | Available for numeric features |
| Isolation forest | `IsolationForest`, `MPSIsolationForest` | Available; CPU is selected for this branch-heavy workload because it is expected to be faster than Apple GPU |
| Ranking trees | `LearningToRankRegressor` | Available; CPU is selected for this latency-sensitive workflow because it is expected to be faster than Apple GPU |

Objective variants such as quantile, Poisson, Tweedie, logistic, and ranking losses should be
selected through estimator parameters when they share the same tree engine. Separate class names
are added only when the model family has different fit/predict semantics.

The same status information is available from Python:

```python
import mpsboost as mb

print(mb.available_estimators())
print(mb.planned_estimators())
mb.require_estimator_supported("CatBoostRegressor")
```

The `0.3.0` milestone delivers the v2 arboretum foundation: one shared tree-family registry,
unified semantics for boosting/bagging/random-split models, honest backend policy, and no
placeholder estimator classes.

Random forest row sampling, feature subsampling, ExtraTrees random thresholds, and CatBoost-like
ordered permutations share one deterministic randomization contract. Categorical features can be
marked with `categorical_features=[...]`; CatBoost-like estimators also accept `cat_features=[...]`
as an alias. Categories are ordered by training-target statistics and then passed through the same
native histogram split engine. Unknown prediction categories are encoded as missing values and use
the native missing-value default direction. Categorical model persistence intentionally fails until
the public model format stores category mappings.

Validation metric history and early stopping also share one estimator-independent monitoring
contract, so future classifiers and tree ensembles do not duplicate callback semantics.

LightGBM-like controlled leaf-wise growth is available through
`growth_strategy="leaf_wise"`. The implementation uses the same native tree engine as
level-wise boosting and adds explicit controls for `max_leaves`, `max_active_leaves`, and
`min_gain_to_split`. Benchmark scripts compare level-wise and leaf-wise end to end; users should
prefer the strategy that wins on their workload rather than assuming one growth policy is always
faster.

## CPU and MPS backends

MPSBoost treats CPU as a first-class backend, not as a temporary fallback. The CPU oracle/backend
is implemented inside this project and shares the same quantization, objective, sampling,
monitoring, split-gain, and model-format contracts as the MPS backend. MPSBoost does not call
XGBoost, LightGBM, CatBoost, or scikit-learn as hidden training engines.

The intended long-term policy is:

- `device="cpu"` forces the in-project CPU backend.
- `device="mps"` forces the in-project MPS/Metal backend.
- `device="auto"` chooses CPU for small or synchronization-heavy workloads and MPS for
  workloads where the measured tree hot path can dominate transfer and launch overhead.

MPS is an acceleration backend, not a requirement. If CPU is faster or more stable for a given
workload, the project should say so and use CPU under `auto`.

## Prediction path

All delivered tree families share one prediction contract. Gradient boosting, single trees,
random forests, ExtraTrees, and CatBoost-like numeric estimators all reuse the native flat-tree
model format plus a shared Python aggregation helper for feature-count validation, feature-subset
slicing, and forest averaging.

Current MPS acceleration targets training hot paths. Prediction uses the shared native tree
traversal path and may run faster on CPU for small batches. A future MPS batch traversal kernel
can replace the traversal backend without changing estimator APIs or model files.

## Backend diagnostics

The native backend exposes non-sensitive device and cache diagnostics:

```python
import mpsboost as mb

print(mb.__version__)
print(mb.is_available())
print(mb.system_info())
print(mb.mps_setup_instructions())
print(mb.cache_info())
```

If `device="mps"` is requested on a machine or session where Apple GPU acceleration is unavailable,
MPSBoost reports copy-paste setup commands:

```bash
xcode-select --install
xcodebuild -downloadComponent MetalToolchain
python -m pip install --upgrade --force-reinstall mpsboost
python -c "import mpsboost as mb; print(mb.system_info())"
```

CPU-only scripts, managed CI, and multiprocessing model-selection workers can skip import-time GPU
environment checks without disabling CPU training:

```bash
MPSBOOST_SKIP_ENV_CHECK=1 python your_script.py
```

`cache_info()` only reports paths and existence; it does not create directories. `create_cache()`
explicitly creates the L2 cache directories, and `clear_cache()` safely removes the MPSBoost cache
root after rejecting dangerous targets such as the filesystem root, the user home directory, or
symlinks. Cache deletion never changes model results.

## Project principles

- Familiar XGBoost/scikit-learn-style Python entry points.
- `device="mps"` as the stable user-facing Apple GPU backend name.
- Custom Metal kernels for tree-specific irregular computation.
- Prebuilt wheels and no heavyweight Python runtime dependency.
- Observable backend policy: invalid inputs fail clearly, while CPU-suitable MPS requests warn and
  record the backend decision in the training summary.
- End-to-end benchmarks, including preprocessing and synchronization.

## Status

The public API currently includes `GradientBoostingRegressor`, `GradientBoostingClassifier`,
their backwards-compatible project-branded aliases, the estimator capability registry,
deterministic randomization and monitoring helpers, cache diagnostics and management helpers,
`is_available`, `system_info`, and `__version__`. Training supports dense finite
`float32`/`float64`-compatible data, ordered categorical feature encoding, feature-level
monotonic constraints, path-level interaction constraints, L1/L2/gamma regularization, leaf-value
clipping, squared error regression, binary-logistic classification, multiclass one-vs-rest
classification, quantile, Poisson, Tweedie, isolation-forest anomaly scoring, pointwise ranking,
deterministic quantization, depth-limited histogram trees, sklearn-compatible `score()`, model
save/load for numeric non-categorical binary/native models, gain/split/permutation feature
importance, approximate SHAP-like explanations, random forest `n_jobs`, explicit `device="mps"`,
explicit `device="cpu"`, and initial `device="auto"` selection.

The checked-in S6 benchmark records both regressions and wins. On the M2 Ultra validation machine, small end-to-end training remains slower on MPS, while the `gbdt-large-wide` scenario reached a 1.629x median speedup with maximum prediction difference around `5.4e-6` versus the CPU oracle.

Sparse matrices, categorical model persistence, public GPU prediction, and full third-party API
compatibility are outside this milestone. Small datasets are expected to be slower on the GPU
because fixed device setup and synchronization costs dominate; the checked-in benchmark report
preserves this regression region alongside larger wins.

## Release audits

The `0.3.0` release gate includes:

- CPU, packaging, integration, and real Metal GPU tests on Python 3.10 and 3.13.
- Wheel content checks excluding specs, tests, caches, and build artifacts.
- Dynamic-link checks for the native extension.
- Apache-2.0 project licensing and runtime dependency review.
- Fresh PyPI installation and real MPS smoke validation.

## Independence notice

MPSBoost is an independent open-source project. It is not affiliated with, endorsed by, or
sponsored by Apple Inc., the XGBoost project, the LightGBM project, the CatBoost project, or the
scikit-learn project. The MPS/Metal backend is an independent implementation based on public
papers, public API documentation, and original engineering work; it is not derived from those
libraries. Apple, Metal, and Metal Performance Shaders may be trademarks of Apple Inc.

## License

Apache-2.0
