fastzy 0.3.1

Python library for fast fuzzy search over a big file written in Rust

Python library for fast fuzzy search over a big file written in Rust

Table of Contents

• Table of Contents
• About The Project
  • Built With
  • Performance
  • Installation
• Usage
• License
• Contact

About The Project

Fastzy is a library written in Rust used for searching over a file for a text based on its distance (levenshtein). The library uses mbleven algorithm for a k-bounded levenshtein distance measurement. When the max distance requested is above 3, where mbleven is slower, the distance algorithm is replaced with Wagner–Fischer. The library loads the whole file into memory, and create a lightweight index, based on the lengths of the lines. It helps to narrow down the amount of lookups to only potential lines.

Built With

• mbleven
• Pyo3

Performance

Library	Text Size	Function	Time
python-Levenshtein	500mb	Levenshtein.distance('text')	13.93s
fastzy	500mb	fastzy.search('text)	0.023s

Installation

pip3 install fastzy

Usage

import fastzy

# open a file and index it in memory
searcher = fastzy.Searcher(
    file_path='input_text_file.txt',
    separator='',
)

# search for the input text 'text' with the distance of 1
searcher.search(
    pattern='text',
    max_distance=1,
)
['test', 'texts', 'next']

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Gal Ben David - [e-mail]

Project Link: https://github.com/Intsights/fastzy