Metadata-Version: 2.4
Name: pysubgroup
Version: 2.0.0
Summary: Fast subgroup discovery with exhaustive and heuristic search.
Keywords: subgroup discovery,exceptional model mining,data mining,pattern mining
Author: Florian Lemmerich
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Dist: numpy>=2.0
Requires-Dist: pandas>=2.2
Requires-Dist: scikit-learn>=1.5,<1.8
Requires-Dist: scipy>=1.11,<2
Requires-Dist: cupy-cuda13x[ctk]>=14 ; extra == 'gpu'
Requires-Dist: numba>=0.60 ; extra == 'numba'
Requires-Dist: matplotlib>=3.7 ; extra == 'plot'
Requires-Python: >=3.11, <3.15
Project-URL: Homepage, https://github.com/flemmerich/pysubgroup
Project-URL: Documentation, https://flemmerich.github.io/pysubgroup/
Project-URL: Issues, https://github.com/flemmerich/pysubgroup/issues
Project-URL: Changelog, https://github.com/flemmerich/pysubgroup/blob/main/CHANGELOG.md
Provides-Extra: gpu
Provides-Extra: numba
Provides-Extra: plot
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/flemmerich/pysubgroup/main/logo.png" alt="pysubgroup" width="280">
</p>

> ### Welcome to pysubgroup v2!

This repository and the `pysubgroup` package on PyPI now hold pysubgroup v2,
a complete rewrite that replaces the previous version. The old version
remains available in the git branch `legacy`. What is new:

- **Faster.** In our benchmarks more than ten times faster than the old pysubgroup, thanks to batched evaluation of candidates on packed bitsets and optional numba kernels. It also finishes tasks where the old version times out.
- **More reliable.** More than 1,900 tests, parity checks against the old pysubgroup (identical top-10 qualities in every benchmark task both versions finished), deterministic tie-breaking, and clearly defined handling of missing values.
- **Still easy to use.** An intuitive API in which a search takes a few lines, as before: a target, a search space, a quality function, and an algorithm.
- **Modern API.** Targets fitted in the scikit-learn style, immutable `Condition` and `Rule` objects, results as plain `(quality, rule)` pairs, NumPy 2 and Python 3.11 to 3.14.
- **Comprehensive.** Search algorithms (DFS, best-first, OPUS, DpSubgroup, Apriori, SD-Map, SD-Map*, GP-growth, NumBSD, VLSD, beam search, DSSD, and generalization-aware search) for binary, numeric, multiclass, nominal, and vector-valued targets, plus exceptional model mining, significance filtering, and visualizations. Each algorithm is implemented from its publication.

> ⚠️ pysubgroup v2 was essentially rewritten from scratch, with most of the code actually generated by AI tools (at the time of writing, primarily Claude Opus and GPT-5.6 Sol), while design decisions and high-level code reviews were handled by humans.
>
> While this might not sit well with purists, it has enabled us to implement more features, produce more efficient, robust, and thoroughly tested code, and focus our attention on the most scientifically interesting decisions—all while aiming for production-ready (we hope!) reliability.
> In fact, I personally feel much more confident in the current version of pysubgroup than in the previous version, which I wrote manually over the years. This is also reflected in the version jump from 0.9 to 2.0.
>
> Note that the old, human-written version of pysubgroup is still available in the `legacy` branch.

## A first example

Which groups of people are most likely to earn more than 50K a year?

```python
from sklearn.datasets import fetch_openml
import pysubgroup as sd

data = fetch_openml("adult", version=2, as_frame=True).frame.drop(columns="fnlwgt")

target = sd.BinaryTarget().fit(data, "class", ">50K")
conditions = sd.generate_conditions(data.drop(columns="class"))
task = sd.SubgroupDiscoveryTask(
    data, target, conditions, sd.WRAccQF(), result_set_size=5, depth=2
)
result = sd.DFS().execute(task)
print(sd.to_dataframe(result, data, target))
```

```text
   quality                                                          subgroup   size  relative_size  target_share   lift
0    0.095                                 marital-status=Married-civ-spouse  22379          0.458         0.446  1.864
1    0.089  marital-status=Married-civ-spouse ∧ native-country=United-States  20003          0.410         0.457  1.910
2    0.088                    marital-status=Married-civ-spouse ∧ race=White  20054          0.411         0.454  1.897
3    0.085                                   relationship=Husband ∧ sex=Male  19715          0.404         0.449  1.875
4    0.085                                              relationship=Husband  19716          0.404         0.449  1.875
```

## Installation

pysubgroup v2 needs Python 3.11 to 3.14 with NumPy (>= 2), pandas, scikit-learn,
and SciPy.

```console
pip install pysubgroup
```

Optional extras:

```console
pip install "pysubgroup[plot]"    # matplotlib visualizations
pip install "pysubgroup[numba]"   # numba kernels for numeric targets and NumBSD
```

The old version (v0.x) can still be installed from the legacy branch:

```console
pip install "git+https://github.com/flemmerich/pysubgroup.git@legacy"
```

## Subgroup Discovery

Subgroup Discovery is a well-established data mining technique that allows you to identify patterns in your data.
More precisely, the goal of subgroup discovery is to identify descriptions of data subsets that show an interesting distribution with respect to a pre-specified target concept.
For example, given a dataset of patients in a hospital, we could be interested in subgroups of patients for which a certain treatment X was successful.
One example result could then be stated as:

_"While in general the operation is successful in only 60% of the cases, for the subgroup
of female patients under 50 that also have been treated with drug D, the success rate was 82%."_

Here, a variable _operation success_ is the target concept, and the identified subgroup has the interpretable description _female=True AND age<50 AND drug_D=True_. We call these single selection expressions (such as _female=True_) _conditions_.
The interesting behavior for this subgroup is that the distribution of the target concept differs significantly from the distribution in the overall dataset.
A discovered subgroup could also be seen as a rule:
```
female=True AND age<50 AND drug_D=True ==> Operation_outcome=SUCCESS
```
Computationally, subgroup discovery is challenging since a large number of such conjunctive subgroup descriptions have to be considered. Of course, finding computable criteria for which subgroups are likely to interest a user is also an eternal struggle.
Therefore, a lot of literature has been devoted to the topic of subgroup discovery (including some of my own work). Overviews of the topic are, for example:

* Herrera, Francisco, et al. ["An overview on subgroup discovery: foundations and applications."](https://scholar.google.de/scholar?q=Herrera%2C+Francisco%2C+et+al.+%E2%80%9CAn+overview+on+subgroup+discovery%3A+foundations+and+applications.%E2%80%9D+Knowledge+and+Information+Systems+29.3+(2011)%3A+495-525.) Knowledge and Information Systems 29.3 (2011): 495-525.
* Atzmueller, Martin. ["Subgroup discovery."](https://scholar.google.de/scholar?q=Atzmueller%2C+Martin.+%E2%80%9CSubgroup+discovery.%E2%80%9D+Wiley+Interdisciplinary+Reviews%3A+Data+Mining+and+Knowledge+Discovery+5.1+(2015)%3A+35-49.) Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5.1 (2015): 35-49.
* And of course, my point of view on the topic is [summarized in my dissertation](https://opus.bibliothek.uni-wuerzburg.de/files/9781/Dissertation-Lemmerich.pdf).

## How to use

A search has four ingredients: a **target** (the property of interest), a
**search space** (a list of conditions), a **quality function**, and an
**algorithm**. They are combined in a `SubgroupDiscoveryTask`:

```python
import pysubgroup as sd

target = sd.BinaryTarget().fit(data, "class", ">50K")
conditions = sd.generate_conditions(data.drop(columns="class"))
task = sd.SubgroupDiscoveryTask(
    data,
    target,
    conditions,
    sd.WRAccQF(),
    result_set_size=10,   # number of subgroups returned
    depth=2,              # maximum number of conditions per subgroup
)
result = sd.DFS().execute(task)
```

The result is a list of `(quality, rule)` pairs, best first:

```python
for quality, rule in result:
    print(f"{quality:.4f}  {rule}")
# 0.0948  [marital-status=Married-civ-spouse]
# 0.0892  [marital-status=Married-civ-spouse ∧ native-country=United-States]
# ...
```

`sd.to_dataframe(result, data, target)` turns it into a table with sizes and
target statistics, and `sd.subgroup_data(result[0], data)` returns the rows
that a subgroup covers.

### Numeric targets

For a numeric target, use `NumericTarget` with a numeric quality function.
Filters such as a minimum subgroup size restrict the results:

```python
target = sd.NumericTarget().fit(data, "hours-per-week")
conditions = sd.generate_conditions(data.drop(columns=["class", "hours-per-week"]))
task = sd.SubgroupDiscoveryTask(
    data, target, conditions, sd.StandardQFNumeric(a=0.5),
    result_set_size=5, depth=2, filters=[sd.MinSizeFilter(100)],
)
result = sd.DFS().execute(task)
print(sd.to_dataframe(result, data, target))
```

```text
   quality                                                  subgroup   size  relative_size    mean  mean_difference
0  525.687                           relationship=Husband ∧ sex=Male  19715          0.404  44.166            3.744
1  525.670                                      relationship=Husband  19716          0.404  44.166            3.744
2  525.060              marital-status=Married-civ-spouse ∧ sex=Male  19899          0.407  44.145            3.722
...
```

### Other algorithms

All exhaustive algorithms return the same subgroups; heuristic ones trade
completeness for speed or diversity. Swap the algorithm without changing the
task:

```python
sd.BestFirstSearch().execute(task)
sd.BeamSearch(beam_width=10).execute(task)   # heuristic
sd.NumBSD().execute(task)                    # exhaustive, for numeric targets
sd.DSSD().execute(task)                      # diverse subgroup sets
```

### Your own search space

`generate_conditions` returns an ordinary list of conditions; extend or replace
it with your own:

```python
from pysubgroup import Condition

conditions = sd.generate_conditions(data.drop(columns="class"), n_bins=5)
conditions.append(Condition("age", "gt", 35))
conditions.append(Condition("age", "interval", lower=20, upper=30))  # (20, 30]
```

## Key classes

- `Condition`: an atomic condition such as `age > 35` or `sex = Female`.
- `Rule`: a conjunction of conditions, the description of a subgroup.
- Targets (`BinaryTarget`, `NumericTarget`, `MulticlassTarget`, ...): fitted on
  the data; they hold the target statistics.
- Quality functions (`WRAccQF`, `StandardQF`, `StandardQFNumeric`, ...): score
  a subgroup and, for exhaustive searches, bound the quality of its
  refinements (optimistic estimates).
- `SubgroupDiscoveryTask`: what to search for; an algorithm's `execute` runs it.

## Learn more

- [The pysubgroup book](https://flemmerich.github.io/pysubgroup/): the full documentation, from a first
  discovery run to custom quality functions, with a reference part (API,
  choosing methods, glossary, troubleshooting).
- [examples/](https://github.com/flemmerich/pysubgroup/tree/main/examples): notebooks on binary and numeric targets,
  visualizations, auditing a classifier (SubROC), exceptional model mining, and
  writing your own quality functions.

## Cite

If you are using pysubgroup for your research, please consider citing our demo paper:

    Lemmerich, F., & Becker, M. (2018, September). pysubgroup: Easy-to-use subgroup discovery in python. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECMLPKDD). pp. 658-662.

bibtex:

```bibtex
@inproceedings{lemmerich2018pysubgroup,
  title={pysubgroup: Easy-to-use subgroup discovery in python},
  author={Lemmerich, Florian and Becker, Martin},
  booktitle={Joint European Conference on Machine Learning and Knowledge Discovery in Databases},
  pages={658--662},
  year={2018}
}
```

## License

We are happy about anyone using this software. Thus, this work is put under an Apache license. However, if this constitutes any hindrance to your application, please feel free to contact us; we are sure that we can work something out.

Apache License 2.0; see [LICENSE](https://github.com/flemmerich/pysubgroup/blob/main/LICENSE).
