spaCy Data Debug has utilities to help you debug your custom NER data. It checks for inconsistencies in labels for the same text.
spaCy Data Debug
spaCy Data Debug has utilities to help you debug your custom NER data. It checks for inconsistencies in labels for the same text,
pip install spacy-data-debug
How to use
from pathlib import Path import srsly from spacy_data_debug.core import * from spacy_data_debug.pipeline import *
0. Load your Data in the Prodigy Annotation Format
train = list(srsly.read_jsonl(base_dir / "train.jsonl")) dev = list(srsly.read_jsonl(base_dir / "dev.jsonl")) test = list(srsly.read_jsonl(base_dir / "test.jsonl"))
Clean, format and filter overlapping entities
While working on a large annotation projects the format of your data can get weird from different annotation sessions by different people.
This ensures you have data in a format useful for the other functions in this
train = fix_annotations_format(train) dev = fix_annotations_format(dev) test = fix_annotations_format(test)
Or construct a Pipeline
Pipeline holds your datasets together and runs
spacy_data_debug functions across all datasets.
This can make sure you have consistent annotations across your datasets split
pipeline = Pipeline(train, dev, test) pipeline.apply(fix_annotations_format)
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size spacy_data_debug-0.0.3.tar.gz (5.4 kB)||File type Source||Python version None||Upload date||Hashes View hashes|