Skip to main content

Testing framework for sequence labeling

Project description


# seqeval
seqeval is a Python framework for sequence labeling evaluation.
seqeval can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, semantic role labeling and so on.

This is well-tested by using the Perl script [conlleval](https://www.clips.uantwerpen.be/conll2002/ner/bin/conlleval.txt),
which can be used for measuring the performance of a system that has processed the CoNLL-2000 shared task data.

## Support features
seqeval supports following formats:
* IOB1
* IOB2
* IOE1
* IOE2
* IOBES

and supports following metrics:

| metrics | description |
|---|---|
| accuracy_score(y\_true, y\_pred) | Compute the accuracy. |
| precision_score(y\_true, y\_pred) | Compute the precision. |
| recall_score(y\_true, y\_pred) | Compute the recall. |
| f1_score(y\_true, y\_pred) | Compute the F1 score, also known as balanced F-score or F-measure. |
| classification_report(y\_true, y\_pred, digits=2) | Build a text report showing the main classification metrics. `digits` is number of digits for formatting output floating point values. Default value is `2`. |

## Usage
Behold, the power of seqeval:

```python
>>> from seqeval.metrics import accuracy_score
>>> from seqeval.metrics import classification_report
>>> from seqeval.metrics import f1_score
>>>
>>> y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>> y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>>
>>> f1_score(y_true, y_pred)
0.50
>>> accuracy_score(y_true, y_pred)
0.80
>>> classification_report(y_true, y_pred)
precision recall f1-score support

MISC 0.00 0.00 0.00 1
PER 1.00 1.00 1.00 1

micro avg 0.50 0.50 0.50 2
macro avg 0.50 0.50 0.50 2
```

### Keras Callback

Seqeval provides a callback for Keras:

```python
from seqeval.callbacks import F1Metrics

id2label = {0: '<PAD>', 1: 'B-LOC', 2: 'I-LOC'}
callbacks = [F1Metrics(id2label)]
model.fit(x, y, validation_data=(x_val, y_val), callbacks=callbacks)
```

## Installation
To install seqeval, simply run:

```
$ pip install seqeval[cpu]
```

If you want to install seqeval on GPU environment, please run:

```bash
$ pip install seqeval[gpu]
```

## Requirement
* numpy >= 1.14.0


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

seqeval-0.0.10.tar.gz (6.4 kB view details)

Uploaded Source

Built Distribution

seqeval-0.0.10-py3-none-any.whl (7.5 kB view details)

Uploaded Python 3

File details

Details for the file seqeval-0.0.10.tar.gz.

File metadata

  • Download URL: seqeval-0.0.10.tar.gz
  • Upload date:
  • Size: 6.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.6.3

File hashes

Hashes for seqeval-0.0.10.tar.gz
Algorithm Hash digest
SHA256 69a72e26371fed1ccba8d423537e69f24157d6094eb7376c74c0ebcf8dfdff43
MD5 bce3bbd60e931a5cfca04ce702af6b85
BLAKE2b-256 d5381aa0d5e59ff627891759df458b08f3a157c6afd2fe3238057bd365e79c46

See more details on using hashes here.

File details

Details for the file seqeval-0.0.10-py3-none-any.whl.

File metadata

  • Download URL: seqeval-0.0.10-py3-none-any.whl
  • Upload date:
  • Size: 7.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.1.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.4

File hashes

Hashes for seqeval-0.0.10-py3-none-any.whl
Algorithm Hash digest
SHA256 501a59f21a3deb3cda05f1fee9113dd3532db393fe33c748881d55a608b76390
MD5 8acd08d7c15d67b8e332bcb735263bbf
BLAKE2b-256 55dd3bf1c646c310daabae47fceb84ea9ab66df7f518a31a89955290d82b8100

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page