Testing framework for sequence labeling
Project description
# seqeval
seqeval is a Python framework for sequence labeling evaluation.
seqeval can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, semantic role labeling and so on.
This is well-tested by using the Perl script [conlleval](https://www.clips.uantwerpen.be/conll2002/ner/bin/conlleval.txt),
which can be used for measuring the performance of a system that has processed the CoNLL-2000 shared task data.
## Support features
seqeval supports following formats:
* IOB1
* IOB2
* IOE1
* IOE2
* IOBES
and supports following metrics:
| metrics | description |
|---|---|
| accuracy_score(y\_true, y\_pred) | Compute the accuracy. |
| precision_score(y\_true, y\_pred) | Compute the precision. |
| recall_score(y\_true, y\_pred) | Compute the recall. |
| f1_score(y\_true, y\_pred) | Compute the F1 score, also known as balanced F-score or F-measure. |
| classification_report(y\_true, y\_pred, digits=2) | Build a text report showing the main classification metrics. `digits` is number of digits for formatting output floating point values. Default value is `2`. |
## Usage
Behold, the power of seqeval:
```python
>>> from seqeval.metrics import accuracy_score
>>> from seqeval.metrics import classification_report
>>> from seqeval.metrics import f1_score
>>>
>>> y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>> y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>>
>>> f1_score(y_true, y_pred)
0.50
>>> accuracy_score(y_true, y_pred)
0.80
>>> classification_report(y_true, y_pred)
precision recall f1-score support
MISC 0.00 0.00 0.00 1
PER 1.00 1.00 1.00 1
micro avg 0.50 0.50 0.50 2
macro avg 0.50 0.50 0.50 2
```
### Keras Callback
Seqeval provides a callback for Keras:
```python
from seqeval.callbacks import F1Metrics
id2label = {0: '<PAD>', 1: 'B-LOC', 2: 'I-LOC'}
callbacks = [F1Metrics(id2label)]
model.fit(x, y, validation_data=(x_val, y_val), callbacks=callbacks)
```
## Installation
To install seqeval, simply run:
```
$ pip install seqeval[cpu]
```
If you want to install seqeval on GPU environment, please run:
```bash
$ pip install seqeval[gpu]
```
## Requirement
* numpy >= 1.14.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
seqeval-0.0.10.tar.gz
(6.4 kB
view details)
Built Distribution
File details
Details for the file seqeval-0.0.10.tar.gz
.
File metadata
- Download URL: seqeval-0.0.10.tar.gz
- Upload date:
- Size: 6.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.6.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69a72e26371fed1ccba8d423537e69f24157d6094eb7376c74c0ebcf8dfdff43 |
|
MD5 | bce3bbd60e931a5cfca04ce702af6b85 |
|
BLAKE2b-256 | d5381aa0d5e59ff627891759df458b08f3a157c6afd2fe3238057bd365e79c46 |
File details
Details for the file seqeval-0.0.10-py3-none-any.whl
.
File metadata
- Download URL: seqeval-0.0.10-py3-none-any.whl
- Upload date:
- Size: 7.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.1.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 501a59f21a3deb3cda05f1fee9113dd3532db393fe33c748881d55a608b76390 |
|
MD5 | 8acd08d7c15d67b8e332bcb735263bbf |
|
BLAKE2b-256 | 55dd3bf1c646c310daabae47fceb84ea9ab66df7f518a31a89955290d82b8100 |