# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).


## [0.21.0] 09.08.2026

### Changes
- Categorical features (`cat_cater_impl`) treat null rows as a regular category in the binning via an internal null sentinel (`MISSING_SENTINEL`) instead of excluding them from the group-by; the missing bin can now be merged with other categories by the quantile binning
- Mapping keys of `CatCats` are always strings: the feature column is cast to `Utf8` in `CatCats.cut` before applying the mapping, so numeric categorical features (e.g. MNIST pixels) stay consistent
- Deterministic tie-breaking when sorting categories by weight / target weight
- `ContCats.get_single_filter`: a feature whose group set is entirely null (no cuts learned) now renders its value bin as `is not null` instead of raising an `IndexError`; combined with the existing `is null` missing bin this yields a clean "has value vs missing" two-way split

### Fixes
- The binning (and therefore `calc_diff`) is now invariant to whether missing rows are represented as `NULL` or imputed with `missing_value`: identical rows produce identical category groupings and scores
- `CatCats.cut` no longer fails on null-dtype columns; the `Utf8` cast resolves the invalid `i64 -> null` conversion for all-null count sets
- `dataframes.set_diff_df`: use `how="full"` instead of the deprecated `how="outer"` join argument (polars >= 0.20.29)


## [0.20.2] 08.08.2026

### Fixes
- `darkwing.get_df_from_files`: read multiple CSV/parquet files with `pl.concat` instead of `pl.read_csv(",".join(files))` (polars does not accept a comma-joined string)
- `train.cont_cater_impl`: normalize cumulative weights by the non-null weight sum so quantile cuts work for features where null rows carry significant weight (previously `IndexError: index 0 is out of bounds`)
- `dataframes.get_left_right_filter`: treat bins with `diff == 0` as neutral instead of non-target when `neutral_faktor` is 0 (matches the documented three-way split semantics)
- `train.counter`: return `(None, None, None)` when no feature survives categorization instead of raising `IndexError`, so the node becomes a leaf
- `pilz.get_where_sql`: render spores without cuts as `WHEN TRUE` so a root leaf with an empty cut list produces valid SQL


## [0.20.1] 08.08.2026

### Changes
- Add docstrings to the public modules (cli, datacard, settings, dataframes, filter, pilz, darkwing, datacardhelper, eval, thresholds, train, mermaid_fence)
- Split the monolithic `tests/test_models.py` into per-module test files with shared fixtures, and add tests for the CLI, training, eval, darkwing, thresholds and datacardhelper


## [0.20.0] 08.08.2026

### Added
- Missing values in features without `missing_value` are now treated as a separate "missing" category during categorization (detected per subsample), producing `IS NULL`/`IS NOT NULL` conditions in the trained spores
- `IS NULL`/`IS NOT NULL` SQL support in the filter layer

### Changes
- Features with `missing_value` set are still imputed at load time; features without it keep nulls and learn an explicit missing category instead


## [0.19.0] 06.08.2026

### Changes
- Move `train_files`/`test_files` from the DataCard into the settings: `train_files` (and optional `eval_files`) in `TrainSettings`, `test_files` in `EvalSettings`
- Thresholds during training are computed on `eval_files` (falls back to `train_files` if empty)


## [0.18.0] 04.08.2026

### Added
- Compute threshold files (youden and f1) at the end of the training step on the train data

### Changes
- Eval only reads the threshold files from the training folder (selected via different_target_pilz_comb_method), thresholds are not computed in eval anymore
- Add max back as different_target_pilz_comb_method (highest score, no threshold file)

## [0.17.4] 04.08.2026

### Added
- Combine different target pilze with max and youden as optional argument

## [0.17.3] 02.08.2026

### Added
- Combine same target pilze with max as optional argument

### Fixes
- Bug for eval with colum names conatining - 

## [0.17.2] 30.07.2026

### Changes
- Change weakening factor to log scale

## [0.17.1] 30.07.2026

### Added
- Add weakening factor for score for low popopulated spores

## [0.17.0] 29.07.2026

### Added
- Add n_rep which repeats the cut calulation to avoid statisccal fluctations


## [0.16.2] 07.06.2026

### Updates
- rm use_neutral_state cli arg
- disable simplfy usage
- Calc Youden index for multiclass result

## [0.16.1] 12.05.2026

### Updates
- Add missing files

## [0.16.0] 12.05.2026

### Updates
- Docs now in a good shape


## [0.15.3] 10.05.2026

### Updates
- update docs

## [0.15.2] 27.04.2026

### Added
- update docs

## [0.15.1] 26.04.2026

### Added
- KI generated documentation
- Fix links

## [0.15.0] 22.04.2026

### Added
- add churn example
- neutral state calulations


## [0.14.0] 12.03.2026

### Added
- implement infer (same as evel except no label is given in data)
- implemnet output file for eval/infer 
- added keep extra columns for evel/infer
- no hist plots (they need more work)


## [0.13.1] 09.03.2026

### Fixed
- Fix Homepage Path

## [0.13.0] 04.03.2026

- Rename to Pilz


## [0.12.0] 25.01.2026

- cli for creating data card
- add support for parquete files
- handle missing values 


## [0.11.0] 05.01.2026

### Added

- add neutral state for low populated states (optionla feature)
- add espresso minmizer for smaller queries
- introduce batched eval, since larger where clauses are not good for duckdb


## [0.10.0] 10.12.2025

### Changed

- use Sympy to handle binary logic (very slow but works, could also simplify numerical expressions but then even more slow)
- Bank Term Deposit example (many cat features)

### Added
- Works with Categorical features


## [0.9.1] 07.10.2025

### Added
- First version of CICD pipeline

### Changed

- make eval folde to list

## [0.9.0] 08.09.2025

### Added

- Initial version of project
- Works with continus features only
- Example version runs with mnist 
- No docmention so far