Skip to main content
Join the official Python Developers Survey 2018 and win valuable prizes: Start the survey!

Abydos NLP/IR library

Project description

Abydos

CI Status Travis-CI Build Status Circle-CI Build Status AppVeyor Build Status Semaphore Build Status
Code Quality Code Climate Scrutinizer Codacy CodeFactor Ebert
Dependencies Requirements Status Known Vulnerabilities Updates
Test Coverage Coverage Status
Local Analysis Pylint Score pycodestyle Errors flake8 Errors
Usage Documentation Status Binder License: GPL v3 Libraries.io SourceRank Zenodo
Contribution CII Best Practices 'Waffle.io - Columns and their card count' OpenHUB
PyPI PyPI PyPI versions
conda-forge conda-forge conda-forge downloads conda-forge platforms

abydos

Abydos NLP/IR library
Copyright 2014-2018 by Christopher C. Little

Abydos is a library of phonetic algorithms, string distance measures & metrics, stemmers, and string fingerprinters including:

  • Phonetic algorithms
    • Robert C. Russell’s Index
    • American Soundex
    • Refined Soundex
    • Daitch-Mokotoff Soundex
    • Kölner Phonetik
    • NYSIIS
    • Match Rating Algorithm
    • Metaphone
    • Double Metaphone
    • Caverphone
    • Alpha Search Inquiry System
    • Fuzzy Soundex
    • Phonex
    • Phonem
    • Phonix
    • SfinxBis
    • phonet
    • Standardized Phonetic Frequency Code
    • Statistics Canada
    • Lein
    • Roger Root
    • Oxford Name Compression Algorithm (ONCA)
    • Eudex phonetic hash
    • Haase Phonetik
    • Reth-Schek Phonetik
    • FONEM
    • Parmar-Kumbharana
    • Davidson’s Consonant Code
    • SoundD
    • PSHP Soundex/Viewex Coding
    • an early version of Henry Code
    • Norphone
    • Dolby Code
    • Phonetic Spanish
    • Spanish Metaphone
    • MetaSoundex
    • SoundexBR
    • NRL English-to-phoneme
    • Beider-Morse Phonetic Matching
  • String distance metrics
    • Levenshtein distance
    • Optimal String Alignment distance
    • Levenshtein-Damerau distance
    • Hamming distance
    • Tversky index
    • Sørensen–Dice coefficient & distance
    • Jaccard similarity coefficient & distance
    • overlap similarity & distance
    • Tanimoto coefficient & distance
    • Minkowski distance & similarity
    • Manhattan distance & similarity
    • Euclidean distance & similarity
    • Chebyshev distance
    • cosine similarity & distance
    • Jaro distance
    • Jaro-Winkler distance (incl. the strcmp95 algorithm variant)
    • Longest common substring
    • Ratcliff-Obershelp similarity & distance
    • Match Rating Algorithm similarity
    • Normalized Compression Distance (NCD) & similarity
    • Monge-Elkan similarity & distance
    • Matrix similarity
    • Needleman-Wunsch score
    • Smither-Waterman score
    • Gotoh score
    • Length similarity
    • Prefix, Suffix, and Identity similarity & distance
    • Modified Language-Independent Product Name Search (MLIPNS) similarity & distance
    • Bag distance
    • Editex distance
    • Eudex distances
    • Sift4 distance
    • Baystat distance & similarity
    • Typo distance
    • Indel distance
    • Synoname
  • Stemmers
    • the Lovins stemmer
    • the Porter and Porter2 (Snowball English) stemmers
    • Snowball stemmers for German, Dutch, Norwegian, Swedish, and Danish
    • CLEF German, German plus, and Swedish stemmers
    • Caumann’s German stemmer
    • UEA-Lite Stemmer
    • Paice-Husk Stemmer
    • Schinke Latin stemmer
    • S stemmer
  • String Fingerprints
    • string fingerprint
    • q-gram fingerprint
    • phonetic fingerprint
    • Pollock & Zomora’s skeleton key
    • Pollock & Zomora’s omission key
    • Cisłak & Grabowski’s occurrence fingerprint
    • Cisłak & Grabowski’s occurrence halved fingerprint
    • Cisłak & Grabowski’s count fingerprint
    • Cisłak & Grabowski’s position fingerprint
    • Synoname Toolcode

Installation

Required libraries:

  • Numpy
  • Six

Recommended libraries:

  • PylibLZMA (Python 2 only–for LZMA compression string distance metric)

To install Abydos (master) from Github source:

git clone https://github.com/chrislit/abydos.git --recursive
cd abydos
python setup install

If your default python command calls Python 2.7 but you want to install for Python 3, you may instead need to call:

python3 setup install

To install Abydos (latest release) from PyPI using pip:

pip install abydos

To install from conda-forge:

conda install abydos

It should run on Python 2.7 and Python 3.3-3.7.

Testing & Contributing

To run the whole test-suite just call tox:

tox

The tox setup has the following environments: py27, py36, doctest, py27-regression, py36-regression, pylint, pycodestyle, flake8, doc8, badges, docs, py27-fuzz, & py36-fuzz. So if only want to generate documentation (in HTML, EPUB, & PDF formats), just call:

tox -e docs

In order to only run & generate Flake8 reports, call:

tox -e flake8

Contributions such as bug reports, PRs, suggestions, desired new features, etc. are welcome through the Github Issues & Pull requests.

Release History

0.3.0 (2018-10-15)

  • Fixed implementation of Bag distance
  • Updated BMPM to version 3.10
  • Fixed Sphinx documentation on readthedocs.org
  • Split string fingerprints out of clustering into their own module
  • Added support for q-grams to skip-n characters
  • New phonetic algorithms:
    • Statistics Canada
    • Lein
    • Roger Root
    • Oxford Name Compression Algorithm (ONCA)
    • Eudex phonetic hash
    • Haase Phonetik
    • Reth-Schek Phonetik
    • FONEM
    • Parmar-Kumbharana
    • Davidson’s Consonant Code
    • SoundD
    • PSHP Soundex/Viewex Coding
    • an early version of Henry Code
    • Norphone
    • Dolby Code
    • Phonetic Spanish
    • Spanish Metaphone
    • MetaSoundex
    • SoundexBR
    • NRL English-to-phoneme
  • New string fingerprints:
    • Cisłak & Grabowski’s occurrence fingerprint
    • Cisłak & Grabowski’s occurrence halved fingerprint
    • Cisłak & Grabowski’s count fingerprint
    • Cisłak & Grabowski’s position fingerprint
    • Synoname Toolcode
  • New distance measures:
    • Minkowski distance & similarity
    • Manhattan distance & similarity
    • Euclidean distance & similarity
    • Chebyshev distance & similarity
    • Eudex distances
    • Sift4 distance
    • Baystat distance & similarity
    • Typo distance
    • Indel distance
    • Synoname
  • New stemmers:
    • UEA-Lite Stemmer
    • Paice-Husk Stemmer
    • Schinke Latin stemmer
  • Eliminated ._compat submodule in favor of six
  • Transitioned from PEP8 to flake8, etc.
  • Phonetic algorithms now consistently use max_length=-1 to indicate that there should be no length limit
  • Added example notebooks in binder directory

0.2.0 (2015-05-27)

  • Added Caumanns’ German stemmer
  • Added Lovins’ English stemmer
  • Updated Beider-Morse Phonetic Matching to 3.04
  • Added Sphinx documentation

0.1.1 (2015-05-12)

  • First Beta release to PyPI
Authors

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
abydos-0.3.0-py2.7.egg (373.4 kB) Copy SHA256 hash SHA256 Egg 2.7 Oct 15, 2018
abydos-0.3.0-py2.py3-none-any.whl (188.7 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Oct 15, 2018
abydos-0.3.0-py3.3.egg (381.7 kB) Copy SHA256 hash SHA256 Egg 3.3 Oct 15, 2018
abydos-0.3.0-py3.4.egg (380.9 kB) Copy SHA256 hash SHA256 Egg 3.4 Oct 15, 2018
abydos-0.3.0-py3.5.egg (378.5 kB) Copy SHA256 hash SHA256 Egg 3.5 Oct 15, 2018
abydos-0.3.0-py3.6.egg (384.2 kB) Copy SHA256 hash SHA256 Egg 3.6 Oct 15, 2018
abydos-0.3.0-py3.7.egg (359.6 kB) Copy SHA256 hash SHA256 Egg 3.7 Oct 15, 2018
abydos-0.3.0.tar.gz (196.4 kB) Copy SHA256 hash SHA256 Source None Oct 15, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page