Skip to main content

Testing framework for sequence labeling

Project description


# seqeval
seqeval is a Python framework for sequence labeling evaluation.
seqeval can evaluate the performance of chunking tasks such as named-entity recognition, part-of-speech tagging, semantic role labeling and so on.

This is well-tested by using the Perl script [conlleval](https://www.clips.uantwerpen.be/conll2002/ner/bin/conlleval.txt),
which can be used for measuring the performance of a system that has processed the CoNLL-2000 shared task data.

## Support features
seqeval supports following formats:
* IOB1
* IOB2
* IOE1
* IOE2
* IOBES

and supports following metrics:

| metrics | description |
|---|---|
| accuracy_score(y\_true, y\_pred) | Compute the accuracy. |
| precision_score(y\_true, y\_pred) | Compute the precision. |
| recall_score(y\_true, y\_pred) | Compute the recall. |
| f1_score(y\_true, y\_pred) | Compute the F1 score, also known as balanced F-score or F-measure. |
| classification_report(y\_true, y\_pred, digits=2) | Build a text report showing the main classification metrics. `digits` is number of digits for formatting output floating point values. Default value is `2`. |

## Usage
Behold, the power of seqeval:

```python
>>> from seqeval.metrics import accuracy_score
>>> from seqeval.metrics import classification_report
>>> from seqeval.metrics import f1_score
>>>
>>> y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>> y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
>>>
>>> f1_score(y_true, y_pred)
0.50
>>> accuracy_score(y_true, y_pred)
0.80
>>> classification_report(y_true, y_pred)
precision recall f1-score support

MISC 0.00 0.00 0.00 1
PER 1.00 1.00 1.00 1

avg / total 0.50 0.50 0.50 2
```

### Keras Callback

Seqeval provides a callback for Keras:

```python
from seqeval.callbacks import F1Metrics

id2label = {0: '<PAD>', 1: 'B-LOC', 2: 'I-LOC'}
callbacks = [F1Metrics(id2label)]
model.fit(x, y, validation_data=(x_val, y_val), callbacks=callbacks)
```

## Installation
To install seqeval, simply run:

```
$ pip install seqeval
```

## Requirement
* numpy >= 1.14.0


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

seqeval-0.0.7-py3-none-any.whl (7.3 kB view details)

Uploaded Python 3

File details

Details for the file seqeval-0.0.7-py3-none-any.whl.

File metadata

  • Download URL: seqeval-0.0.7-py3-none-any.whl
  • Upload date:
  • Size: 7.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.1.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.4

File hashes

Hashes for seqeval-0.0.7-py3-none-any.whl
Algorithm Hash digest
SHA256 5a8264e6678f74dcead1aeeabeb1c5a9d31b48b8ce43e2408fad090ceadb6244
MD5 06d49deb3de21ca16d4ee8a07fd9ad30
BLAKE2b-256 e4b0605b34d76782dab9d6e010b396c67b7d583b38e8d53d2edb8b18a2978e65

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page