Fast Python phonetic algorithms
Project description
Fuzzy is a python library implementing common phonetic algorithms quickly. Typically this is in string similarity exercises, but they’re pretty versatile.
It uses C Extensions (via Pyrex) for speed.
The algorithms are:
Double Metaphone Based on Maurice Aubrey’s C code from his perl implementation.
Copyright
Fuzzy is Copyright 2011 YouGov, Plc. and released under the MIT license except where noted in specific source files.
Installation
Installation should be easy if you have a C compiler such as gcc. All you should need to do is easy_install/pip install it. If you have Pyrex it will regenerate the C code, otherwise it will use the pre-generated code. Here’s a basic installation on a clean virtualenv:
(fuzzy_cean)Kotai:~ chmullig$ pip install https://bitbucket.org/yougov/fuzzy/get/1.0.tar.gz Downloading/unpacking https://bitbucket.org/yougov/fuzzy/get/1.0.tar.gz Downloading 1.0.tar.gz Running setup.py egg_info for package from https://bitbucket.org/yougov/fuzzy/get/1.0.tar.gz Installing collected packages: Fuzzy Running setup.py install for Fuzzy building 'fuzzy' extension gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/fuzzy.c -o build/temp.macosx-10.6-universal-2.6/src/fuzzy.o gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/double_metaphone.c -o build/temp.macosx-10.6-universal-2.6/src/double_metaphone.o gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc -arch x86_64 build/temp.macosx-10.6-universal-2.6/src/fuzzy.o build/temp.macosx-10.6-universal-2.6/src/double_metaphone.o -o build/lib.macosx-10.6-universal-2.6/fuzzy.so Successfully installed Fuzzy Cleaning up... (fuzzy_cean)Kotai:~ chmullig$
Usage
The functions are quite easy to use!
>>> import fuzzy >>> soundex = fuzzy.Soundex(4) >>> soundex('fuzzy') 'F200' >>> dmeta = fuzzy.DMetaphone() >>> dmeta('fuzzy') ['FS', None] >>> fuzzy.nysiis('fuzzy') 'FASY'
Performance
Fuzzy’s Double Metaphone was ~10 times faster than the pure python implementation by Andrew Collins in some recent testing. Soundex and NYSIIS should be similarly faster. Using iPython’s timeit:
In [3]: timeit soundex('fuzzy') 1000000 loops, best of 3: 326 ns per loop In [4]: timeit dmeta('fuzzy') 100000 loops, best of 3: 2.18 us per loop In [5]: timeit fuzzy.nysiis('fuzzy') 100000 loops, best of 3: 13.7 us per loop
Distance Metrics
We recommend the Python-Levenshtein module for fast, C based string distance/similarity metrics. Among others functions it includes:
Levenshtein edit distance
Jaro distance
Jaro-Winkler distance
In testing it’s been several times faster than comparable pure python implementations of those algorithms.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file Fuzzy-1.1.tar.gz
.
File metadata
- Download URL: Fuzzy-1.1.tar.gz
- Upload date:
- Size: 20.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57863b9e9aacf82a0b8530ff3b1435cd9b83fe7e526f358eacc366a35eb0cdff |
|
MD5 | 9520b910dbbddd5449c68720ce0dedfc |
|
BLAKE2b-256 | 0ecb24003d3553ba239bee615de9ad198746a12f4dfd57dac3289e07ecfcb44a |