Entity linking evaluation
Python command-line evaluation scripts for TAC entity linking and related wikification, named entity disambiguation, and within- and cross-document coreference tasks.
It aims for fast and flexible coreference resolution and sophisticated named entity recognition evaluation, such as partial scores for partial overlap between gold and system mentions. CEAF, in particular, is much faster to calculate here than in the CoNLL-11/12 scorer. It boasts features such as configurable metrics; accounting for or ignoring cross-document coreference (see the evaluate --by-doc flag); plotting to compare evaluation by system, measure and corpus subset; and bootstrap-based confidence interval calculation for document-wise evaluation metrics.
Requires that python (2.7, with Py3k support experimental/partial) be installed on your system with numpy (and preferably scipy for fast CEAF calculation) and joblib. matplotlib is required for the plot-systems command.
See a list of commands with:
./nel --help
Or install onto your Python path (e.g. with pip install git+https://github.com/wikilinks/neleval) then
python -m neleval --help
TAC-KBP 2014 EDL quickstart
./scripts/run_tac14_evaluation.sh \
/path/to/gold.xml \ # TAC14 gold standard queries/mentions
/path/to/gold.tab \ # TAC14 gold standard link and nil annotations
/system/output/directory \ # directory containing (only) TAC14 system output files
/script/output/directory \ # directory to which results are written
number_of_jobs # number of jobs for parallel mode
Each file in in the system output directory is scored against gold.tab.
Similar facility is available for TAC-KBP’15 EDL.
More details
See the documentation for more details.
Metadata
Release files for neleval 3.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| neleval-3.1.1.tar.gz | 51.6 kB | Without provenance | Details for neleval-3.1.1.tar.gz |
Release files / neleval-3.1.1.tar.gz
| Download URL | neleval-3.1.1.tar.gz |
|---|---|
| Size | 51.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fc396945f308f033a7d2bd5201c896c6d7aa11f7ef18aaa9e771b59264390b20
|
|
BLAKE2b-256 checksum How to use checksums |
aaf9c55bd10a392bbc2e9d4199497b2797b0a7dae08bb84a9e6407d39018937e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3
|
Security for neleval 3.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.