seqeval
seqeval is a Python framework for sequence labeling evaluation. seqeval can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, semantic role labeling and so on.
This is well-tested by using the Perl script conlleval, which can be used for measuring the performance of a system that has processed the CoNLL-2000 shared task data.
Support features
seqeval supports following formats:
- IOB1
- IOB2
- IOE1
- IOE2
- IOBES
and supports following metrics:
| metrics | description |
|---|---|
| accuracy_score(y_true, y_pred) | Compute the accuracy. |
| precision_score(y_true, y_pred) | Compute the precision. |
| recall_score(y_true, y_pred) | Compute the recall. |
| f1_score(y_true, y_pred) | Compute the F1 score, also known as balanced F-score or F-measure. |
| classification_report(y_true, y_pred, digits=2) | Build a text report showing the main classification metrics. digits is number of digits for formatting output floating point values. Default value is 2. |
Usage
Behold, the power of seqeval:
>>> from seqeval.metrics import accuracy_score
>>> from seqeval.metrics import classification_report
>>> from seqeval.metrics import f1_score
>>>
>>> y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>> y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>>
>>> f1_score(y_true, y_pred)
0.50
>>> accuracy_score(y_true, y_pred)
0.80
>>> classification_report(y_true, y_pred)
precision recall f1-score support
MISC 0.00 0.00 0.00 1
PER 1.00 1.00 1.00 1
micro avg 0.50 0.50 0.50 2
macro avg 0.50 0.50 0.50 2
Keras Callback
Seqeval provides a callback for Keras:
from seqeval.callbacks import F1Metrics
id2label = {0: '<PAD>', 1: 'B-LOC', 2: 'I-LOC'}
callbacks = [F1Metrics(id2label)]
model.fit(x, y, validation_data=(x_val, y_val), callbacks=callbacks)
Installation
To install seqeval, simply run:
$ pip install seqeval[cpu]
If you want to install seqeval on GPU environment, please run:
$ pip install seqeval[gpu]
Requirement
- numpy >= 1.14.0
- tensorflow(optional)
1 maintainer
Metadata
1 maintainer
Release files for seqeval 0.0.19
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| seqeval-0.0.19.tar.gz | 30.3 kB | Details |
Release files / seqeval-0.0.19.tar.gz
| Download URL | seqeval-0.0.19.tar.gz |
|---|---|
| Size | 30.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
63209166fd74b11e88f1fa0132614ce852c6e83f678de76f7a53672c2d16c8d2
|
|
BLAKE2b-256 checksum How to use checksums |
93e5b7705156a77f742cfe4fc6f22d0c71591edb2d243328dff2f8fc0f933ab6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/50.3.0 requests-toolbelt/0.9.1 tqdm/4.50.0 CPython/3.6.7
|
