Skip to main content

spacy-span-analyzer

A simple tool to analyze the Spans in your dataset. It's tightly integrated with spaCy, so you can easily incorporate it to existing NLP pipelines. This is also a reproduction of Papay, et al's work on Dissecting Span Identification Tasks with Performance Prediction (EMNLP 2020).

⏳ Install

Using pip:

pip install spacy-span-analyzer

Directly from source (I highly recommend running this within a virtual environment):

git clone git@github.com:ljvmiranda921/spacy-span-analyzer.git
cd spacy-span-analyzer
pip install .

⏯ Usage

You can use the Span Analyzer as a command-line tool:

spacy-span-analyzer ./path/to/dataset.spacy

Or as an imported library:

import spacy
from spacy.tokens import DocBin
from spacy_span_analyzer import SpanAnalyzer

nlp = spacy.blank("en")  # or any Language model

# Ensure that your dataset is a DocBin
doc_bin = DocBin().from_disk("./path/to/data.spacy")
docs = list(doc_bin.get_docs(nlp.vocab))

# Run SpanAnalyzer and get span characteristics
analyze = SpanAnalyzer(docs)
analyze.frequency  
analyze.length
analyze.span_distinctiveness
analyze.boundary_distinctiveness

Inputs are expected to be a list of spaCy Docs or a DocBin (if you're using the command-line tool).

Working with Spans

In spaCy, you'd want to store your Spans in the doc.spans property, under a particular spans_key (sc by default). Unlike the doc.ents property, doc.spans allows overlapping entities. This is useful especially for downstream tasks like Span Categorization.

A common way to do this is to use char_span to define a slice from your Doc:

doc = nlp(text)
spans = []
from annotation in annotations:
    span = doc.char_span(
        annotation["start"],
        annotation["end"],
        annotation["label"],
    )
    spans.append(span)

# Put all spans under a spans_key
doc.spans["sc"] = spans

You can also achieve the same thing by using set_ents or by creating a SpanGroup.

Metadata

Release files for spacy-span-analyzer 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for spacy-span-analyzer 0.3.0
File Size Uploaded Provenance Actions
spacy-span-analyzer-0.3.0.tar.gz 11.5 kB Without provenance Details for spacy-span-analyzer-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for spacy-span-analyzer 0.3.0
File Interpreter ABI Platform Actions
spacy_span_analyzer-0.3.0-py3-none-any.whl Python 3 none any Details for spacy_span_analyzer-0.3.0-py3-none-any.whl

Total release size: 24.7 kB

Release files / spacy-span-analyzer-0.3.0.tar.gz

Download URL spacy-span-analyzer-0.3.0.tar.gz
Size 11.5 kB
Tags Source
SHA-256 checksum
How to use checksums
089cd3ef0db03d4981d546b64d250adf80c6efbe28b320786c5869ca8597c8e1
BLAKE2b-256 checksum
How to use checksums
bff380317c5b1fdd5df5f84c0f3a58a6d6b55d23b9f442364cf3af77cfbaf036
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.0 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

Release files / spacy_span_analyzer-0.3.0-py3-none-any.whl

Download URL spacy_span_analyzer-0.3.0-py3-none-any.whl
Size 13.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
db14c8c0d7bbc2b4db8ee0e6aef0c320988a0d2c9c470c829f357e99a86128ad
BLAKE2b-256 checksum
How to use checksums
f57dbd6d44e7e059814eb091050d268a64d46fe709163dd04497063e7f5884db
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.0 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

Security for spacy-span-analyzer 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page