Fast Levenshtein and Damerau optimal string alignment algorithms.
Project description
editdistpy
editdistpy is a fast implementation of the Levenshtein edit distance and the Damerau-Levenshtein optimal string alignment (OSA) edit distance algorithms. The original C# project can be found at SoftWx.Match.
Installation
The easiest way to install editdistpy is using pip
:
pip install -U editdistpy
Usage
You can specify the max_distance
you care about, if the edit distance exceeds
this max_distance
, -1
will be returned. Specifying a sensible max distance
can result in significant speed improvement.
You can also specify max_distance=sys.maxsize
if you wish for the actual edit
distance to always be computed.
Levenshtein
import sys
from editdistpy import levenshtein
string_1 = "flintstone"
string_2 = "hanson"
max_distance = 2
print(levenshtein.distance(string_1, string_2, max_distance))
# expected output: -1
max_distance = sys.maxsize
print(levenshtein.distance(string_1, string_2, max_distance))
# expected output: 6
Damerau-Levenshtein OSA
import sys
from editdistpy import damerau_osa
string_1 = "flintstone"
string_2 = "hanson"
max_distance = 2
print(damerau_osa.distance(string_1, string_2, max_distance))
# expected output: -1
max_distance = sys.maxsize
print(damerau_osa.distance(string_1, string_2, max_distance))
# expected output: 6
Benchmark
A simple benchmark was done on Python 3.8.12 against editdistance which implements the Levenshtein edit distance algorithm.
The script used by the benchmark can be found here.
For clarity, the following string pairs were used.
Single word (completely different)
"xabxcdxxefxgx"
"1ab2cd34ef5g6"
Single word (similar)
"example"
"samples"
Single word (identical ending)
"kdeisfnexabxcdxlskdixefxgx"
"xabxcdxlskdixefxgx"
Short string
"short sentence with words"
"shrtsen tence wit mispeledwords"
Long string
"Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod rem"
"Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium"
single_dif string
test_damerau_osa 0.5202 usec/pass 1040.36 msec total 2000000 iterations
test_levenshtein 0.3547 usec/pass 709.40 msec total 2000000 iterations
test_editdistance 0.6399 usec/pass 1279.81 msec total 2000000 iterations
test_damerau_osa early_cutoff 0.5134 usec/pass 1026.72 msec total 2000000 iterations
test_levenshtein early_cutoff 0.3862 usec/pass 772.31 msec total 2000000 iterations
single_sim string
test_damerau_osa 0.2983 usec/pass 596.57 msec total 2000000 iterations
test_levenshtein 0.2433 usec/pass 486.68 msec total 2000000 iterations
test_editdistance 0.3942 usec/pass 788.36 msec total 2000000 iterations
test_damerau_osa early_cutoff 0.2865 usec/pass 572.90 msec total 2000000 iterations
test_levenshtein early_cutoff 0.2363 usec/pass 472.61 msec total 2000000 iterations
single_end string
test_damerau_osa 0.3332 usec/pass 666.32 msec total 2000000 iterations
test_levenshtein 0.3300 usec/pass 659.93 msec total 2000000 iterations
test_editdistance 0.7902 usec/pass 1580.42 msec total 2000000 iterations
test_damerau_osa early_cutoff 0.3199 usec/pass 639.74 msec total 2000000 iterations
test_levenshtein early_cutoff 0.3205 usec/pass 641.01 msec total 2000000 iterations
short string
test_damerau_osa 0.9925 usec/pass 1984.97 msec total 2000000 iterations
test_levenshtein 0.6379 usec/pass 1275.76 msec total 2000000 iterations
test_editdistance 0.9587 usec/pass 1917.37 msec total 2000000 iterations
test_damerau_osa early_cutoff 0.7535 usec/pass 1506.91 msec total 2000000 iterations
test_levenshtein early_cutoff 0.5794 usec/pass 1158.79 msec total 2000000 iterations
long string
test_damerau_osa 8.6244 usec/pass 17248.73 msec total 2000000 iterations
test_levenshtein 4.2367 usec/pass 8473.36 msec total 2000000 iterations
test_editdistance 2.0407 usec/pass 4081.31 msec total 2000000 iterations
test_damerau_osa early_cutoff 1.0795 usec/pass 2158.99 msec total 2000000 iterations
test_levenshtein early_cutoff 0.9031 usec/pass 1806.28 msec total 2000000 iterations
While max_distance=10
significantly improves the computation time, it may not
be a sensible value in some cases.
editdistpy is also seen to perform better with shorter length strings and can be the more suitable library if your use case mainly deals with comparing short strings.
Changelog
See the changelog for a history of notable changes to edistdistpy.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file editdistpy-0.1.4.tar.gz
.
File metadata
- Download URL: editdistpy-0.1.4.tar.gz
- Upload date:
- Size: 98.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f73f7213b5f74976cbd3fb6c14da5743a160520a38924f0948014ee566a4d7de |
|
MD5 | 0f90c02b87ff32ca08be16ae7a565a60 |
|
BLAKE2b-256 | caa4740425b98a684144dd182ff1df2e5cf56fe7c599a0512e834f238597eebd |