Skip to main content

Dashboard for Quality-driven NER.

Project description

ner-eval-dashboard

Dashboard for Quality-driven NER.

concept

The idea of this project is to provide a more elaborated evaluation for NER models. That way, it should be easier to fix labeling mistakes, better understand the positive and negative aspects of the trained NER model and see how it applies on unlabeled data.

progress

version 0.1.0 provides standard F1 scores for Exact Match, Type Match, and Position Match. So far only Flair models are implemented. See Issues to view planned features

installation

The ner eval dashboard can be installed via:

pip install ner-eval-dashboard==0.1

usage

The ner eval dashboard can be used on various ways: cli, api or via docker. It is important to always specify a model, a dataset and a tokenizer.

Note: To run the examples, you need to once manually download the CONLL03 dataset and put it into the {FLAIR_CACHE_ROOT}/.flair/datasets/conll_03 folder.

cli

The ner eval dashboard can be use via the command line interface:

ner_eval_dashboard [--dataset_path DATASET_PATH] [--extra_unlabeled_data EXTRA_UNLABELED_DATA] [--use USE [USE ...]] [--exclude EXCLUDE [EXCLUDE ...]] {FLAIR} predictor_name_or_path {SPACE} {RAW,COLUMN_DATASET,JSONL_DATASET,CONLL03,CONLL03_GERMAN,CONLL03_DUTCH,CONLL03_SPANISH,WNUT17,ARABIC_ANER,ARABIC_AQMAR,BASQUE,WEIBO,DANE,MOVIE_SIMPLE,MOVIE_COMPLEX,SEC_FILLINGS,RESTAURANT,STACKOVERFLOW,TWITTER
,PERSON,WEBPAGES,WNUT2020,WIKIGOLD,FINER,BIOFID,EUROPARL,LEGAL_NER,GERMEVAL,POLITICS,BUSINESS,ICELANDIC_NER,HIRONSAN,MASAKHANE,MULTI_CONER,WIKIANN,XTREME,WIKINER,SWEDISH_NER,TURKU}

For example the following can be used to evaluate the Bi-LSTM-CRF model based on Flair embeddings on CONLL03:

ner_eval_dashboard FLAIR flair/ner-english SPACE CONLL03

api

from ner_eval_dashboard.dataset.flair import FlairConll03
from ner_eval_dashboard.predictor import FlairPredictor
from ner_eval_dashboard.tokenizer import SpaceTokenizer
from ner_eval_dashboard.app import create_app

tokenizer = SpaceTokenizer()
dataset = FlairConll03(tokenizer)
predictor = FlairPredictor("flair/ner-english")

app = create_app("my-app", predictor, dataset)

app.run_server()

docker

docker images are publicly available at docker hub

docker run -it --rm -p 8050:8050 helpmefindaname/ner-eval-dashboard FLAIR flair/ner-english SPACE CONLL03

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ner-eval-dashboard-0.1.0.tar.gz (18.3 kB view details)

Uploaded Source

Built Distribution

ner_eval_dashboard-0.1.0-py3-none-any.whl (23.3 kB view details)

Uploaded Python 3

File details

Details for the file ner-eval-dashboard-0.1.0.tar.gz.

File metadata

  • Download URL: ner-eval-dashboard-0.1.0.tar.gz
  • Upload date:
  • Size: 18.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for ner-eval-dashboard-0.1.0.tar.gz
Algorithm Hash digest
SHA256 31abf8dda996397853e7a746b4dcdb4ddbb0db665582e93efd036661cd8f1e8c
MD5 d848b133dff2196baf28daeb87934dd0
BLAKE2b-256 8164fb44d37af5b28c797627e0b77235d46a4c86c6179f38c0ee78381d24f017

See more details on using hashes here.

File details

Details for the file ner_eval_dashboard-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for ner_eval_dashboard-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 0bbfe66edbbedcbbe9a517282ac7f5c8bd7c8168ea9aae426b51b02197068139
MD5 cbed8a24a99bcef2129fb9d1e33d74ac
BLAKE2b-256 73309eb0ea4715cb73662eeb6980d4c1ebe349671ed514457be0a5fa88c35eca

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page