slicemap
Find the data slices where a new model regressed against an old one. Headless, file in, table out.
A new model can lift the overall metric while quietly getting worse on a
segment that matters: one country, one age band, one product category. An
aggregate number hides it. slicemap takes a predictions file with both
models' outputs and the features, scores every slice, and lists the ones where
the new model lost ground, ranked by how many rows are affected.
$ slicemap compare preds.parquet --true label --old pred_v1 --new pred_v2
slicemap: accuracy overall 0.910 -> 0.918
feature slice size old new regression
country BR 842 0.904 0.731 -0.173
device tablet 311 0.880 0.795 -0.085
age [55, 70) 540 0.901 0.860 -0.041
Install
$ pip install slicemap-cli # from PyPI, once released
$ pip install git+https://github.com/jmweb-org/slicemap # latest, available now
Reads one CSV, Parquet or JSON Lines file containing the truth column, both prediction columns, and the feature columns.
Usage
$ slicemap compare preds.parquet --true y --old pred_a --new pred_b
$ slicemap compare preds.csv --true y --old a --new b --features country,age
$ slicemap compare preds.csv --true y --old a --new b --metric error
$ slicemap compare preds.csv --true y --old a --new b --min-slice 50
$ slicemap compare preds.csv --true y --old a --new b --json
$ slicemap compare preds.csv --true y --old a --new b --check
If --features is omitted, every column except the truth and prediction columns
is treated as a feature.
JSON output schema
--json writes a single object to stdout:
{
"metric": "accuracy",
"old_overall": 0.910,
"new_overall": 0.918,
"regressions": [
{
"feature": "country",
"slice": "BR",
"size": 842,
"old_score": 0.904,
"new_score": 0.731,
"regression": 0.173,
"impact": 145.766
}
]
}
| Field | Type | Description |
|---|---|---|
metric |
string | Metric name used for scoring (e.g. "accuracy") |
old_overall |
number | Overall metric score for the old model |
new_overall |
number | Overall metric score for the new model |
regressions |
array | Slices where the new model is worse, sorted by impact descending |
regressions[].feature |
string | Column name the slice is drawn from |
regressions[].slice |
string | Slice label (a category value or a quantile bin like "[55, 70)") |
regressions[].size |
integer | Number of rows in the slice |
regressions[].old_score |
number | Old model's metric score on this slice |
regressions[].new_score |
number | New model's metric score on this slice |
regressions[].regression |
number | Absolute degradation (always positive) |
regressions[].impact |
number | regression × size — used for ranking |
All numeric values are rounded to six decimal places.
In CI
Fail a model update when any slice regresses:
- run: slicemap compare preds.parquet --true y --old champion --new challenger --check
How slicing works
Categorical features slice by value; numeric features slice by quantile bins.
For each slice the metric is computed for both models, and the slice is flagged
when the new model is worse. Slices smaller than --min-slice are skipped to
avoid noise, and findings are ranked by impact (regression size times the
number of rows), so the segments worth fixing first come first.
Metrics
| Metric | Direction |
|---|---|
accuracy |
higher is better |
error |
lower is better |
mae |
lower is better |
Exit codes
| Code | Meaning |
|---|---|
| 0 | Compared; no slice regressed (or --check not set) |
| 1 | --check found at least one regressed slice |
| 2 | A column was missing, the metric is unknown, or the file is unsupported |
License
MIT. See LICENSE.
Metadata
Release files for slicemap-cli 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| slicemap_cli-0.2.0.tar.gz | 11.0 kB | Without provenance | Details for slicemap_cli-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| slicemap_cli-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for slicemap_cli-0.2.0-py3-none-any.whl |
Total release size: 21.8 kB
Release files / slicemap_cli-0.2.0.tar.gz
| Download URL | slicemap_cli-0.2.0.tar.gz |
|---|---|
| Size | 11.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3cc23d04ce37d1a571846f0d6656a9bc2349d06fa17323e6b2be8e3bfbd4404a
|
|
BLAKE2b-256 checksum How to use checksums |
773cc9017a103e7a8987e1795ff7c4e85cc36e4931e021bc252affa77bba0ffe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.20 {"installer":{"name":"uv","version":"0.11.20","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / slicemap_cli-0.2.0-py3-none-any.whl
| Download URL | slicemap_cli-0.2.0-py3-none-any.whl |
|---|---|
| Size | 10.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
20ed0899ebad69f707d5ba4420ef70f61dbcef25e25ec02f6ad98dd42504f90c
|
|
BLAKE2b-256 checksum How to use checksums |
3897178edd32945a2b9412d7eae1cf2742ce39a5f5189158e47e5ab1b320686c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.20 {"installer":{"name":"uv","version":"0.11.20","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for slicemap-cli 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.