Welcome to pysubgroup v2!
This repository and the pysubgroup package on PyPI now hold pysubgroup v2,
a complete rewrite that replaces the previous version. The old version
remains available in the git branch legacy. What is new:
- Faster. In our benchmarks more than ten times faster than the old pysubgroup, thanks to batched evaluation of candidates on packed bitsets and optional numba kernels. It also finishes tasks where the old version times out.
- More reliable. More than 1,900 tests, parity checks against the old pysubgroup (identical top-10 qualities in every benchmark task both versions finished), deterministic tie-breaking, and clearly defined handling of missing values.
- Still easy to use. An intuitive API in which a search takes a few lines, as before: a target, a search space, a quality function, and an algorithm.
- Modern API. Targets fitted in the scikit-learn style, immutable
ConditionandRuleobjects, results as plain(quality, rule)pairs, NumPy 2 and Python 3.11 to 3.14. - Comprehensive. Search algorithms (DFS, best-first, OPUS, DpSubgroup, Apriori, SD-Map, SD-Map*, GP-growth, NumBSD, VLSD, beam search, DSSD, and generalization-aware search) for binary, numeric, multiclass, nominal, and vector-valued targets, plus exceptional model mining, significance filtering, and visualizations. Each algorithm is implemented from its publication.
⚠️ pysubgroup v2 was essentially rewritten from scratch, with most of the code actually generated by AI tools (at the time of writing, primarily Claude Opus and GPT-5.6 Sol), while design decisions and high-level code reviews were handled by humans.
While this might not sit well with purists, it has enabled us to implement more features, produce more efficient, robust, and thoroughly tested code, and focus our attention on the most scientifically interesting decisions—all while aiming for production-ready (we hope!) reliability. In fact, I personally feel much more confident in the current version of pysubgroup than in the previous version, which I wrote manually over the years. This is also reflected in the version jump from 0.9 to 2.0.
Note that the old, human-written version of pysubgroup is still available in the
legacybranch.
A first example
Which groups of people are most likely to earn more than 50K a year?
from sklearn.datasets import fetch_openml
import pysubgroup as sd
data = fetch_openml("adult", version=2, as_frame=True).frame.drop(columns="fnlwgt")
target = sd.BinaryTarget().fit(data, "class", ">50K")
conditions = sd.generate_conditions(data.drop(columns="class"))
task = sd.SubgroupDiscoveryTask(
data, target, conditions, sd.WRAccQF(), result_set_size=5, depth=2
)
result = sd.DFS().execute(task)
print(sd.to_dataframe(result, data, target))
quality subgroup size relative_size target_share lift
0 0.095 marital-status=Married-civ-spouse 22379 0.458 0.446 1.864
1 0.089 marital-status=Married-civ-spouse ∧ native-country=United-States 20003 0.410 0.457 1.910
2 0.088 marital-status=Married-civ-spouse ∧ race=White 20054 0.411 0.454 1.897
3 0.085 relationship=Husband ∧ sex=Male 19715 0.404 0.449 1.875
4 0.085 relationship=Husband 19716 0.404 0.449 1.875
Installation
pysubgroup v2 needs Python 3.11 to 3.14 with NumPy (>= 2), pandas, scikit-learn, and SciPy.
pip install pysubgroup
Optional extras:
pip install "pysubgroup[plot]" # matplotlib visualizations
pip install "pysubgroup[numba]" # numba kernels for numeric targets and NumBSD
The old version (v0.x) can still be installed from the legacy branch:
pip install "git+https://github.com/flemmerich/pysubgroup.git@legacy"
Subgroup Discovery
Subgroup Discovery is a well-established data mining technique that allows you to identify patterns in your data. More precisely, the goal of subgroup discovery is to identify descriptions of data subsets that show an interesting distribution with respect to a pre-specified target concept. For example, given a dataset of patients in a hospital, we could be interested in subgroups of patients for which a certain treatment X was successful. One example result could then be stated as:
"While in general the operation is successful in only 60% of the cases, for the subgroup of female patients under 50 that also have been treated with drug D, the success rate was 82%."
Here, a variable operation success is the target concept, and the identified subgroup has the interpretable description female=True AND age<50 AND drug_D=True. We call these single selection expressions (such as female=True) conditions. The interesting behavior for this subgroup is that the distribution of the target concept differs significantly from the distribution in the overall dataset. A discovered subgroup could also be seen as a rule:
female=True AND age<50 AND drug_D=True ==> Operation_outcome=SUCCESS
Computationally, subgroup discovery is challenging since a large number of such conjunctive subgroup descriptions have to be considered. Of course, finding computable criteria for which subgroups are likely to interest a user is also an eternal struggle. Therefore, a lot of literature has been devoted to the topic of subgroup discovery (including some of my own work). Overviews of the topic are, for example:
- Herrera, Francisco, et al. "An overview on subgroup discovery: foundations and applications." Knowledge and Information Systems 29.3 (2011): 495-525.
- Atzmueller, Martin. "Subgroup discovery." Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5.1 (2015): 35-49.
- And of course, my point of view on the topic is summarized in my dissertation.
How to use
A search has four ingredients: a target (the property of interest), a
search space (a list of conditions), a quality function, and an
algorithm. They are combined in a SubgroupDiscoveryTask:
import pysubgroup as sd
target = sd.BinaryTarget().fit(data, "class", ">50K")
conditions = sd.generate_conditions(data.drop(columns="class"))
task = sd.SubgroupDiscoveryTask(
data,
target,
conditions,
sd.WRAccQF(),
result_set_size=10, # number of subgroups returned
depth=2, # maximum number of conditions per subgroup
)
result = sd.DFS().execute(task)
The result is a list of (quality, rule) pairs, best first:
for quality, rule in result:
print(f"{quality:.4f} {rule}")
# 0.0948 [marital-status=Married-civ-spouse]
# 0.0892 [marital-status=Married-civ-spouse ∧ native-country=United-States]
# ...
sd.to_dataframe(result, data, target) turns it into a table with sizes and
target statistics, and sd.subgroup_data(result[0], data) returns the rows
that a subgroup covers.
Numeric targets
For a numeric target, use NumericTarget with a numeric quality function.
Filters such as a minimum subgroup size restrict the results:
target = sd.NumericTarget().fit(data, "hours-per-week")
conditions = sd.generate_conditions(data.drop(columns=["class", "hours-per-week"]))
task = sd.SubgroupDiscoveryTask(
data, target, conditions, sd.StandardQFNumeric(a=0.5),
result_set_size=5, depth=2, filters=[sd.MinSizeFilter(100)],
)
result = sd.DFS().execute(task)
print(sd.to_dataframe(result, data, target))
quality subgroup size relative_size mean mean_difference
0 525.687 relationship=Husband ∧ sex=Male 19715 0.404 44.166 3.744
1 525.670 relationship=Husband 19716 0.404 44.166 3.744
2 525.060 marital-status=Married-civ-spouse ∧ sex=Male 19899 0.407 44.145 3.722
...
Other algorithms
All exhaustive algorithms return the same subgroups; heuristic ones trade completeness for speed or diversity. Swap the algorithm without changing the task:
sd.BestFirstSearch().execute(task)
sd.BeamSearch(beam_width=10).execute(task) # heuristic
sd.NumBSD().execute(task) # exhaustive, for numeric targets
sd.DSSD().execute(task) # diverse subgroup sets
Your own search space
generate_conditions returns an ordinary list of conditions; extend or replace
it with your own:
from pysubgroup import Condition
conditions = sd.generate_conditions(data.drop(columns="class"), n_bins=5)
conditions.append(Condition("age", "gt", 35))
conditions.append(Condition("age", "interval", lower=20, upper=30)) # (20, 30]
Key classes
Condition: an atomic condition such asage > 35orsex = Female.Rule: a conjunction of conditions, the description of a subgroup.- Targets (
BinaryTarget,NumericTarget,MulticlassTarget, ...): fitted on the data; they hold the target statistics. - Quality functions (
WRAccQF,StandardQF,StandardQFNumeric, ...): score a subgroup and, for exhaustive searches, bound the quality of its refinements (optimistic estimates). SubgroupDiscoveryTask: what to search for; an algorithm'sexecuteruns it.
Learn more
- The pysubgroup book: the full documentation, from a first discovery run to custom quality functions, with a reference part (API, choosing methods, glossary, troubleshooting).
- examples/: notebooks on binary and numeric targets, visualizations, auditing a classifier (SubROC), exceptional model mining, and writing your own quality functions.
Cite
If you are using pysubgroup for your research, please consider citing our demo paper:
Lemmerich, F., & Becker, M. (2018, September). pysubgroup: Easy-to-use subgroup discovery in python. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECMLPKDD). pp. 658-662.
bibtex:
@inproceedings{lemmerich2018pysubgroup,
title={pysubgroup: Easy-to-use subgroup discovery in python},
author={Lemmerich, Florian and Becker, Martin},
booktitle={Joint European Conference on Machine Learning and Knowledge Discovery in Databases},
pages={658--662},
year={2018}
}
License
We are happy about anyone using this software. Thus, this work is put under an Apache license. However, if this constitutes any hindrance to your application, please feel free to contact us; we are sure that we can work something out.
Apache License 2.0; see LICENSE.
Metadata
Release files for pysubgroup 2.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pysubgroup-2.0.0.tar.gz | 138.5 kB | Without provenance | Details for pysubgroup-2.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pysubgroup-2.0.0-py3-none-any.whl | Python 3 | none | any | Details for pysubgroup-2.0.0-py3-none-any.whl |
Total release size: 295.2 kB
Release files / pysubgroup-2.0.0.tar.gz
| Download URL | pysubgroup-2.0.0.tar.gz |
|---|---|
| Size | 138.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4af6bb97cf2bd795a02b81d7b691e8afc8f09f7b1c7067c6c61f14130b6681bb
|
|
BLAKE2b-256 checksum How to use checksums |
e4b387739a0586456522958edbcf54416fb90cea94c11f15516e479c55a5a332
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Release files / pysubgroup-2.0.0-py3-none-any.whl
| Download URL | pysubgroup-2.0.0-py3-none-any.whl |
|---|---|
| Size | 156.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5afa58dc1561037f6a61a51f246d44a7d6572678712b2bdd93effd2919078976
|
|
BLAKE2b-256 checksum How to use checksums |
6246a0b352d1d6b119a81a0b09ab37d75e5b08b14b735e911cb6fdd1941282cd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Security for pysubgroup 2.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.