meddeid-eval
Reproducible evaluation for clinical de-identification. meddeid-eval computes
exact-span metrics, character-level recall, core-PII recall, non-PII redaction
rate, and stability results from canonical MedDeID JSONL files.
See the suite evaluation workflow for the end-to-end handoff. This repository remains authoritative for metric definitions, commands, stability configuration, and plotting support.
Installation
python -m pip install meddeid-eval
Optional extras add model inference and plotting support:
python -m pip install 'meddeid-eval[infer,plots]'
Usage
meddeid-eval score --gold meddeid-dutch-synthetic-benchmark.jsonl --predictions predictions.jsonl
meddeid-eval stability expand --config stability.yaml
Gold and prediction files are matched by document_id and use half-open
[begin, end) Unicode-code-point offsets. The score command reports exact
precision, recall, and F1 together with character coverage and redaction
metrics.
Core-PII recall is the label-agnostic fraction of protocol-defined core PII
characters covered by any predicted redaction. Each primary gold span owns a
nested subannotations list. formatting, additional_info, medical_info,
title, and time segments are excluded from the denominator.
Stability perturbations use the complete nl-BE resources from
meddeid-language-nl; incomplete resources are reported as installation
errors.
External comparators
Comparison systems run in their own environments. Export their predictions in
the canonical MedDeID JSONL schema and evaluate them with the same score
command. Belgian DEDUCE is not installed by meddeid-eval.
Development
pip install -e '.[dev]'
pytest
Licence
AGPL-3.0-only. External comparison systems retain their own licence terms.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file meddeid_eval-0.2.1.tar.gz.
File metadata
- Download URL: meddeid_eval-0.2.1.tar.gz
- Upload date:
- Size: 45.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c80c0de8cdc7f153cbf700f046a95a19b2c487b9d1e8da6ca976d20cad7a2d79
|
|
| MD5 |
b5122df725581a2005b64bb74ef02425
|
|
| BLAKE2b-256 |
388c2f2062684571726a04303e63e18662bad12f57aa6d9f5831220202e0e339
|
Provenance
The following attestation bundles were made for meddeid_eval-0.2.1.tar.gz:
Publisher:
publish-python.yml on stighellemans/meddeid-eval
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
meddeid_eval-0.2.1.tar.gz -
Subject digest:
c80c0de8cdc7f153cbf700f046a95a19b2c487b9d1e8da6ca976d20cad7a2d79 - Sigstore transparency entry: 2498916411
- Sigstore integration time:
-
Permalink:
stighellemans/meddeid-eval@ef9f7176a20397b55802260fe9600cc37cce7b9b -
Branch / Tag:
refs/tags/v0.2.1 - Owner: https://github.com/stighellemans
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-python.yml@ef9f7176a20397b55802260fe9600cc37cce7b9b -
Trigger Event:
push
-
Statement type:
File details
Details for the file meddeid_eval-0.2.1-py3-none-any.whl.
File metadata
- Download URL: meddeid_eval-0.2.1-py3-none-any.whl
- Upload date:
- Size: 51.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3a4faba3fd2cb05b39f8113be85380ffe12ab28d0cd602d3ab6fa4a4704f3e96
|
|
| MD5 |
e7b58be2428271bdcf618032d28277aa
|
|
| BLAKE2b-256 |
a2d43eac6111c4b592cbe08b78054e1b0c93da93bf96901baa0cd96db87d4e99
|
Provenance
The following attestation bundles were made for meddeid_eval-0.2.1-py3-none-any.whl:
Publisher:
publish-python.yml on stighellemans/meddeid-eval
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
meddeid_eval-0.2.1-py3-none-any.whl -
Subject digest:
3a4faba3fd2cb05b39f8113be85380ffe12ab28d0cd602d3ab6fa4a4704f3e96 - Sigstore transparency entry: 2498916423
- Sigstore integration time:
-
Permalink:
stighellemans/meddeid-eval@ef9f7176a20397b55802260fe9600cc37cce7b9b -
Branch / Tag:
refs/tags/v0.2.1 - Owner: https://github.com/stighellemans
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-python.yml@ef9f7176a20397b55802260fe9600cc37cce7b9b -
Trigger Event:
push
-
Statement type:
