spaCy Data Debug
spaCy Data Debug has utilities to help you debug your custom NER data. It checks for inconsistencies in labels for the same text,
Install
pip install spacy-data-debug
How to use
from pathlib import Path
import srsly
from spacy_data_debug.core import *
from spacy_data_debug.pipeline import *
0. Load your Data in the Prodigy Annotation Format
train = list(srsly.read_jsonl(base_dir / "train.jsonl"))
dev = list(srsly.read_jsonl(base_dir / "dev.jsonl"))
test = list(srsly.read_jsonl(base_dir / "test.jsonl"))
Clean, format and filter overlapping entities
While working on a large annotation projects the format of your data can get weird from different annotation sessions by different people.
This ensures you have data in a format useful for the other functions in this spacy-data-debug
train = fix_annotations_format(train)
dev = fix_annotations_format(dev)
test = fix_annotations_format(test)
Or construct a Pipeline
A Pipeline holds your datasets together and runs spacy_data_debug functions across all datasets.
This can make sure you have consistent annotations across your datasets split
pipeline = Pipeline(train, dev, test)
pipeline.apply(fix_annotations_format)
1 maintainer
Metadata
1 maintainer
Release files for spacy-data-debug 0.0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| spacy_data_debug-0.0.3.tar.gz | 5.4 kB | Details |
Release files / spacy_data_debug-0.0.3.tar.gz
| Download URL | spacy_data_debug-0.0.3.tar.gz |
|---|---|
| Size | 5.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7c4b75c2363108c04db5901e927189b305f9461b07a3f5e90a369c5fbc31527a
|
|
BLAKE2b-256 checksum How to use checksums |
06fa733405d977c9bf6c758cb207308369df52d28dadad02b55cc58dc68214ad
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.8
|
