Metadata-Version: 2.4
Name: polars-optbinning
Version: 0.2.1
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Dist: polars>=1.37.1
License-File: LICENSE
Summary: PyO3-Polars OptBinning extension with Rust-side optimization kernels
Keywords: polars,binning,optimal-binning,feature-engineering,credit-scoring,rust
Home-Page: https://gitee.com/Ky1eYang/polars-optbinning
Author-email: "kyle.yang" <ky1eyangs225@gmail.com>
Maintainer-email: "kyle.yang" <ky1eyangs225@gmail.com>
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://gitee.com/Ky1eYang/polars-optbinning/blob/master/CHANGELOG.md
Project-URL: Documentation, https://gitee.com/Ky1eYang/polars-optbinning#readme
Project-URL: Homepage, https://gitee.com/Ky1eYang/polars-optbinning
Project-URL: Issues, https://gitee.com/Ky1eYang/polars-optbinning/issues
Project-URL: Repository, https://gitee.com/Ky1eYang/polars-optbinning

# polars-optbinning

Rust-powered rule binning for Polars DataFrames and expressions.

The main API is `BinningTransformer`: fit optimized 1D rules for one or more
features, keep the fitted rule groups, and reuse them in Polars pipelines.

## Installation

Install the published package from PyPI:

```bash
pip install polars-optbinning
```

For local development:

```bash
uv sync --group dev
uv run maturin develop --release
```

## Usage

```python
import polars as pl
from polars_optbinning import BinningTransformer, Objective, Stat

df = pl.DataFrame(
    {
        "x1": [1.0, 1.2, 2.0, 2.2, 3.1, 3.4, 4.5, 5.0, 5.2, 6.0],
        "x2": [10.0, 9.8, 9.5, 8.9, 8.1, 7.3, 6.5, 5.9, 5.1, 4.2],
        "y": [0, 0, 0, 1, 0, 1, 1, 1, 1, 1],
    }
)

transformer = BinningTransformer(
    features=[pl.col("x1"), pl.col("x2")],
    target=pl.col("y"),
)

transformer.fit(
    df,
    objective=Objective.iv(),
    mapping=[Stat.woe(), Stat.count()],
    max_bins=4,
)

eager_out = transformer.transform(df)
pipeline_out = df.with_columns(transformer.pl_transform)
print(eager_out)
print(pipeline_out)
print(transformer.summary())
```

`summary()` can re-execute the current rules and calculate statistics that are
independent of the transform mapping:

```python
stats = transformer.summary(
    extra_stat=[
        Stat.count(),
        Stat.feat_avg(),
        Stat.target_avg(),
        Stat.avg("x2"),
        Stat.woe(),
    ]
)
```

`feat_avg()`, `feat_min()`, and `feat_max()` always return a list covering the
features used by each rule group. `avg(expr)`, `min(expr)`, and `max(expr)`
return scalar statistics for one Polars expression. Expression-backed output
names are derived from the expression, for example `Stat.avg("x2")` produces
`x2_avg`. `woe()`, `iv()`, and `event_rate()` use the initialized target by
default and also accept an explicit expression.

Fitted transformers reuse their selected training frame. Pass
`summary(extra_stat=[...], df=new_df)` to calculate the statistics on another
data set. Parsed and manually constructed transformers require `df` and may
use explicit expression statistics such as `Stat.woe("label")`; target-default
statistics require `BinningTransformer(target=...)`. The
`mapping` supplied to `fit()` or `parse()` remains the explicit schema produced
by `transform()` and `pl_transform`; Rule stats not selected by mapping are not
automatically expanded into transform columns.

Pairwise 2D binning creates one feature for each feature pair:

```python
pairwise = BinningTransformer(
    features=[pl.col("x1"), pl.col("x2")],
    target=pl.col("y"),
    prefix="pair_",
)

pairwise.fit(
    df,
    objective=Objective.iv(),
    mapping=[Stat.index()],
    max_bins=4,
    mode="pairwise_2d",
    progress=True,
)

out = df.with_columns(pairwise.pl_transform)
```

For a continuous target regression-style example, see `examples/example_linear.py`.

Native XGBoost and LightGBM tree models can be parsed into reusable rule groups:

```python
model = BinningTransformer.parse(
    "examples/models/multiclass_xgb.json",
    mapping=[Stat.bin_value()],
    prefix="xgb_",
)

tree_outputs = df.with_columns(model.pl_transform)
raw_margin = df.with_columns(prediction=model.predict())
probability = df.with_columns(prediction=model.predict(output="probability"))
classes = df.with_columns(prediction=model.predict(output="class"))
```

`parse()` accepts XGBoost `save_model()` JSON/UBJ and LightGBM
`Booster.save_model()` text files, paths, and readable file objects. Parsed
transformers expose one editable `RuleGroup` per tree and cannot be fitted
again. See `examples/example_multiclass_models.py` for native model training,
serialization, parsing, and Polars prediction comparisons.

Manual rule engines are also supported:

```python
from polars_optbinning import Condition, Rule, RuleGroup

group = RuleGroup(
    "x",
    ["x"],
    [
        Rule(Condition.interval("x", None, 2.0), [Stat.const(0.1)]),
        Rule(Condition.interval("x", 2.0, None), [Stat.const(0.9)]),
    ],
    [Stat.const(0.0)],
)

engine = BinningTransformer(groups=[group])
out = pl.DataFrame({"x": [1.0, 2.5]}).with_columns(engine.pl_transform)
```

Conditions can be composed as sets across one or more features:

```python
high_risk = (Condition.interval("age", 65.0, None) & Condition.is_in("region", [1.0, 2.0]))
eligible = high_risk | Condition.interval("score", 700.0, None)
excluded = eligible - Condition.is_in("region", [9.0])
assert excluded.features == ["age", "region", "score"]
excluded = excluded.simplify()

group = RuleGroup(
    "decision",
    ["age", "region", "score"],
    [
        Rule(excluded, [Stat.const(1.0)]),
        Rule(Condition.otherwise(), [Stat.const(0.0)]),
    ],
    [Stat.const(0.0)],
)
```

`otherwise` is optional, but when present it must be the final rule. Empty groups and
unmatched rows without an `otherwise` rule produce null outputs. Fitted binning groups
are ordered by descending fitted count and always include a final `otherwise` rule.

Rules may be compiled into an explicit binary execution chain:

```python
group.compile()
assert group.execution_kind == "rule_chain"
```

## Perpetual GBDT

Multidimensional gradient boosting is available through the same transformer:

```python
model = BinningTransformer(features=["x1", "x2"], target="y").fit(
    df,
    mode="gbdt",
    objective=Objective.squared_loss(),
    extra_params={
        "budget": 0.5,
        "iteration_limit": 100,
        "categorical_features": ["x2"],
    },
)

raw = df.select(model.predict())
tree_outputs = df.select(model.pl_transform)
importance = model.feature_importance()
```

Classification uses `Objective.log_loss()`. Integer class labels are preserved, including
non-contiguous labels. `extra_params` is validated strictly and feature-related settings
use feature names rather than column indices.

Tree rules expose `Stat.gain()`, defined as the cumulative split gain along the root-to-leaf
path. `feature_importance()` instead counts every internal split exactly once and returns
`feature`, `gain`, `gain_normalized`, and `split_count`. Structural rule edits invalidate
the original tree importance. See `examples/example_perpetual.py` for a Titanic survival
classification example with categorical and missing-value handling.

The Rust integration currently pins `perpetual` 3.0.0-rc.2, the first compatible release
line that builds on stable Rust; perpetual 2.1.0 requires a nightly-only compiler feature.

## Current Scope

- Optimized multi-feature 1D binning via `BinningTransformer.fit`.
- Typed `Objective` and `Stat` selectors.
- Eager `transform(df)` and expression-friendly `pl_transform`.
- `summary()` returning fitted rule statistics as a Polars DataFrame.
- Pairwise 2D rectangular binning via `mode="pairwise_2d"`.
- XGBoost/LightGBM numeric tree parsing, leaf-rule transforms, and model prediction.

ND optimized fitting is intentionally paused while the reusable rule engine
interface settles.


## Publishing

The release script verifies that `pyproject.toml` and `Cargo.toml` use the same
version, builds optimized wheel and source distributions, publishes them to
PyPI, creates the matching `v<version>` Gitee tag, and mirrors the files to a
Gitee Release.

Token authentication is recommended:

```shell
$env:GITEE_TOKEN = "<personal-access-token>"
$env:MATURIN_PYPI_TOKEN = "<pypi-api-token>"
uv run python scripts/gitee_release.py --no-install-project
```

`PYPI_API_TOKEN` is accepted as an alias. Account/password authentication is
also supported through either `MATURIN_USERNAME` + `MATURIN_PASSWORD` or
`PYPI_USERNAME` + `PYPI_PASSWORD`.

PyPI distributions are immutable. On a repeated run, `--skip-existing` leaves
already published files untouched while allowing wheels for additional
platforms to be uploaded. The Gitee Release is updated and same-named
attachments are replaced. Existing wheels for other platforms are preserved.

Set `SKIP_BUILD=1` to upload files already present in `WHEEL_DIR` (`dist` by
default), `SKIP_PYPI=1` for a Gitee-only mirror, or `SKIP_GITEE=1` for a
PyPI-only release. All credentials must be stored in the CI service's secret
variable store.

Release checklist:

1. Update the version in both `pyproject.toml` and `Cargo.toml`.
2. Update `CHANGELOG.md`.
3. Run Clippy and the Python test suite.
4. Run the release script from the commit that should receive the version tag.

See [CONTRIBUTING.md](CONTRIBUTING.md), [SECURITY.md](SECURITY.md), and
[LICENSE](LICENSE) for the project policies.

