Evaluation utilities for named entity linking and (cross-document) coreference resolution
./scripts/run_tac14_evaluation.sh \ /path/to/gold.xml \ # TAC14 gold standard queries/mentions /path/to/gold.tab \ # TAC14 gold standard link and nil annotations /system/output/directory \ # directory containing (only) TAC14 system output files /script/output/directory \ # directory to which results are written number_of_jobs # number of jobs for parallel mode
Each file in in the system output directory is scored against gold.tab.
Similar facility is available for TAC-KBP‘15 EDL.
See the project wiki for more details.
This project extends the work described in:
It was used as the official scorer for Entity (Discovery and) Linking in 2014 and 2015: