A python library for named entity recognition evaluation
Project description
miNER
A python library for NER (Named Entity Recognition) evaluation
We can evaluate the performance of NER by distinguishing between known entities and unknown entities using this library.
Support
- Tagging Scheme
- IOB2
- BIOES
- BIOUL
- metrics
- precision
- recall
- f1
Requirements
- python3
- cython
Installation
pip install cython # must execute before `pip install mi-ner`
pip install mi-ner
Usage
Sample
>>> from miner import Miner
>>> answers = [
'B-PSN O O B-LOC O O O O'.split(' '),
'B-PSN I-PSN O O B-LOC I-LOC O O O O'.split(' '),
'S-PSN O O S-PSN O O B-LOC I-LOC E-LOC O O O O'.split(' ')
]
>>> predicts = [
'B-PSN O O B-LOC O O O O'.split(' '),
'B-PSN B-PSN O O B-LOC I-LOC O O O O'.split(' '),
'S-PSN O O O O O B-LOC I-LOC E-LOC O O O O'.split(' ')
]
>>> sentences = [
'花子 さん は 東京 に 行き まし た'.split(' '),
'山田 太郎 君 は 東京 駅 に 向かい まし た'.split(' '),
'花子 さん と ボブ くん は 東京 スカイ ツリー に 行き まし た'.split(' '),
]
>>> knowns = {'PSN': ['花子'], 'LOC': ['東京']} # known words (words included in training data)
>>> m = Miner(answers, predicts, sentences, knowns)
>>> m.default_report(True)
precision recall f1_score num
LOC 1.000 1.000 1.000 3
PSN 0.500 0.500 0.500 4
overall 0.714 0.714 0.714 7
{'LOC': {'precision': 1.0, 'recall': 1.0, 'f1_score': 1.0, 'num': 3},
'PSN': {'precision': 0.5, 'recall': 0.5, 'f1_score': 0.5, 'num': 4},
'overall': {'precision': 0.7142857142857143, 'recall': 0.7142857142857143, 'f1_score': 0.7142857142857143, 'num': 7}}
>>> m.unknown_only_report(True)
precision recall f1_score num
LOC 1.000 1.000 1.000 2
PSN 0.000 0.000 0.000 2
overall 0.500 0.500 0.500 4
{'LOC': {'precision': 1.0, 'recall': 1.0, 'f1_score': 1.0, 'num': 2},
'PSN': {'precision': 0.0, 'recall': 0.0, 'f1_score': 0, 'num': 2},
'overall': {'precision': 0.5, 'recall': 0.5, 'f1_score': 0.5, 'num': 4}}
>>> m.return_predict_named_entities()
{'known': {'LOC': ['東京'], 'PSN': ['花子'], 'overall': []},
'unknown': {'LOC': ['東京スカイツリー', '東京駅'], 'PSN': ['山田', '太郎'], 'overall': []}}
Methods
method | description |
---|---|
default_report(print_) | return result of named entity recognition. if print_=True, showing result |
known_only_report(print_) | return result of known named entity recognition. |
unknown_only_report(print_) | return result of unknown named entity recognition. |
return_predict_named_entities() | return named entities along predicted label(predicts). |
return_answer_named_entities() | return named entities along answer label(answer). |
return_miss_labelings() | return miss labeling sentences. |
segmentation_score(mode) | show parcentages of matching answer and predict labels. if known or unknown for mode , return labeling accuracy for known or unknown NE. |
License
MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mi-ner-0.6.1.tar.gz
(6.7 kB
view hashes)
Built Distribution
Close
Hashes for mi_ner-0.6.1-cp37-cp37m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8ef68b99fe418d9a372bd919c556e336d844638c1a807577e4249fd97a2a7abf |
|
MD5 | d369add62cfab85786bbca5839f19f86 |
|
BLAKE2b-256 | 37eb0c0c901e004e7db8409f0ad90851d83c5214a37af9103e5c745828ac60ac |