Skip to main content

Automatic text metrics---BLEU, ROUGE, and METEOR, pllus extras like vocab and ngrams.

Project description

textmetrics

Automatic text metrics---BLEU, ROUGE, and METEOR, plus extras like vocab and ngrams.

Usage

# Compares each candidate (c) separately against all references (r).
python -m textmetrics.main c1.txt c2.txt --references r1.txt r2.txt r3.txt

Installation

Requires:

  • Perl (for BLEU)
  • Java 1.8 (for METEOR)
  • Python 3.6+
pip install textmetrics

Features

  • BLEU
  • ROUGE
  • METEOR

Notes

BLEU and METEOR use the refernce implementations (in Perl and Java, respectively). We originally used the reference Perl implementation for ROUGE as well, but it ran so slowly that we opted for a Python reimplementation instead. (ROUGE's original Perl implementation is also more difficult to setup, even with wrapper libraries.)

Worklist

  • pypi

  • API support

  • ROUGE crashes things if it decides there aren't sentences (e.g., run with README.md as input and reference)

  • Add back in orig ROUGE for completeness (place behind switch)

  • ngrams has divide by zero error. With two simple files (two lines each, same first line, differing second line) running with 2.txt --references 1.txt 1.txt triggered this divide by zero

  • Demo for better README

  • Tests

  • Early check in each module for whether program runnable + nice error message (e.g., no java or bad version, no perl or bad version, etc.)

Note to self: I followed this guide for packaging to pypi, and future uploads will probably look like:

# (1) ensure tests pass

# (2) bump version in setup.py

# (3) commit + push to github

# (4) generate distribution
python setup.py sdist bdist_wheel

# (5) Upload
twine upload dist/*

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

textmetrics-0.0.2.tar.gz (9.1 kB view details)

Uploaded Source

Built Distribution

textmetrics-0.0.2-py3-none-any.whl (12.6 kB view details)

Uploaded Python 3

File details

Details for the file textmetrics-0.0.2.tar.gz.

File metadata

  • Download URL: textmetrics-0.0.2.tar.gz
  • Upload date:
  • Size: 9.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.6.5

File hashes

Hashes for textmetrics-0.0.2.tar.gz
Algorithm Hash digest
SHA256 f54fcccf3eb569c199969af5e006aaa2f8207ef08182f62acd954e068af92420
MD5 94e3452ddb99c28ad5580281d8f899d1
BLAKE2b-256 86047e68be0a76283471cf71a7ebb76a34ecbbcb7a729aab3a773540732b4e4e

See more details on using hashes here.

File details

Details for the file textmetrics-0.0.2-py3-none-any.whl.

File metadata

  • Download URL: textmetrics-0.0.2-py3-none-any.whl
  • Upload date:
  • Size: 12.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.6.5

File hashes

Hashes for textmetrics-0.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 168ba006c86e92e172672ed85fc08564a50f2f619fd01e47d2b0b720ba7a7bf6
MD5 37059f43d95e10ce20500777646dc7b4
BLAKE2b-256 4c64759abdb8a35bd8fd1314cb234b29841c42077ac37d1048a7639745d69a15

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page