pyprocessors_reconciliation
Reconciliation annotations coming from different annotators.
Installation
pip install pyprocessors-reconciliation
Overview
ReconciliationProcessor is a pymultirole processor plugin that reconciles overlapping annotations produced by multiple annotators (NER models, knowledge-base linkers, white/kill lists) into a single coherent set.
The processor is registered under the pyprocessors.plugins entry point as reconciliation.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
type |
ReconciliationType |
linker |
Reconciliation strategy (currently only linker) |
kill_label |
str | None |
None |
Label whose annotations suppress matching model annotations |
white_label |
str | None |
None |
Label treated as authoritative (terms stripped so it acts like a model annotation) |
whitelisted_lexicons |
list[str] | None |
None |
Lexicons whose annotations are duplicated as term-free model candidates |
person_label |
str | None |
None |
Label used to identify person annotations for last-name resolution |
remove_suspicious |
bool |
True |
Drop model annotations that contain no capitalised word (numbers, percentages, etc.) |
resolve_lastnames |
bool |
False |
Resolve isolated last names / first names using full names seen earlier in the document |
How it works
- Sentence filtering —
sentence-labelled annotations are removed before processing. - Whitelist marking (
mark_whitelisted) — annotations matchingwhite_labelhave their terms cleared so they behave like model candidates; annotations fromwhitelisted_lexiconsget a term-free duplicate added alongside the original. - Grouping (
group_annotations) — annotations are grouped by their first term's lexicon (empty string = model / no-lexicon). Same-span annotations in the same group have their term lists merged and deduplicated. - Linker consolidation (
consolidate_linker):- Suspicious model annotations (no capitalised word) are optionally dropped.
- Kill-list annotations suppress matching model annotations.
- KB annotations at the same span enrich the matching model annotation with their terms.
- Overlapping or mismatched-label KB matches are logged as warnings and skipped.
- Last-name resolution — when
resolve_lastnames=True, isolated person names (single token) are resolved to the full-name annotation seen earliest in the document.
Development
The build is driven by Task and uv,
with the shared stages coming from the python-archetype submodule.
Getting started
The stages live in a Git submodule, so clone with --recurse-submodules:
git clone --recurse-submodules git@bitbucket.org:kairntech/pyprocessors_reconciliation.git
cd pyprocessors_reconciliation
sh -c "$(curl -sSL https://taskfile.dev/install.sh)" -- -d -b ~/.local/bin
task
Already cloned without it? The submodule directory is empty, and task fails on:
task: No Taskfile found at ".../submodules/python-archetype/resources/Taskfile.yml"
which means exactly that, and nothing worse:
git submodule update --init
Task is the only manual prerequisite. An archetype cannot bootstrap itself: uv and the
Python interpreter install themselves on demand (every task that runs uv depends on an
internal install-python task), but the thing that runs them does not. Make sure
~/.local/bin is on your PATH — that is where task and uv both land.
Running the pipeline
task stages # print the pipeline stages, in order
task # run the pipeline up to (but excluding) py:publish
task -- --skip-tests # same, without the test stage
task up-to -- py:lint # run the pipeline up to and including one stage
task jenkins # run every stage, exactly what Jenkins runs
task with no argument is safe by construction: it runs every stage but the last, and that
bound is computed from the STAGES list rather than written down. The last stage is the
only one with an effect outside your machine.
STAGES, declared once in Taskfile.yml, is the single definition of the pipeline order —
so what you run locally is what Jenkins runs.
Individual stages
| Task | Description |
|---|---|
task py:sync |
Install the project and its dependencies (uv sync) |
task py:lint |
ruff check and ruff format --check |
task py:format |
Reformat the code with ruff |
task py:test |
Run the test suite |
task py:test-marker -- <m> |
Run the tests carrying one pytest marker |
task py:sbom |
Generate a CycloneDX SBOM of the resolved environment |
task py:check-vulnerabilities |
Check for known CVEs |
task py:check-updates |
Check for dependency updates |
task py:build |
Build the wheel and sdist (uv build) |
task py:publish |
Publish the distributions (uv publish) |
task py:version-file |
Print the path of the file carrying __version__ |
task py:set-version VERSION=x |
Write that version into it |
uv.lock is not versioned here, so py:sync always resolves from scratch (--upgrade):
a stale lock lying around on a machine would otherwise make you test and audit versions the
CI never sees.
SBOM & vulnerability check
task py:sbom and task py:check-vulnerabilities wrap the underlying tools. To run them
by hand:
uv sync --extra test --extra sbom
uv run cyclonedx-py environment -o sbom.cdx.json --output-format json
uv run pip-audit --skip-editable --format json --output audit-report.json
Ask for all the extras, not just sbom: uv sync synchronises rather than adds, so
whatever the requested extras do not pull in gets removed. --extra sbom on its own
uninstalls ruff and pytest, leaving an environment that can no longer lint or test.
--skip-editable skips the project itself, which no advisory database can know about. The
tasks do the same, and pip-audit still exits non-zero when it finds a real vulnerability —
which is the contract the CI relies on. Avoid --strict: it turns "dependency not found on
PyPI" into a fatal error, so it breaks as soon as the local version is not a published one.
1 maintainer
Metadata
1 maintainer
Release files for pyprocessors-reconciliation 1.6.65
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pyprocessors_reconciliation-1.6.65.tar.gz | 10.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pyprocessors_reconciliation-1.6.65-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 20.3 kB
Release files / pyprocessors_reconciliation-1.6.65.tar.gz
| Download URL | pyprocessors_reconciliation-1.6.65.tar.gz |
|---|---|
| Size | 10.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
021298b1efab8797c142dbb8c56051c65e6bf4dfa6b911afba742cbf2a0b221d
|
|
BLAKE2b-256 checksum How to use checksums |
3f9b752b1b0fe6786ccb911a488d0e961489174edbff195d8c8f7cfa59f7fbca
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pyprocessors_reconciliation-1.6.65-py3-none-any.whl
| Download URL | pyprocessors_reconciliation-1.6.65-py3-none-any.whl |
|---|---|
| Size | 9.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b107b2dedf5e484f3a584f717c12313e9148b0c1e12ea47f1f683d7cf64d19bb
|
|
BLAKE2b-256 checksum How to use checksums |
2eaeb7c67d090eb081ba26ec606925e35c2036afe6da6648f38d6bd5180bdf36
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
