A fast implementation of Bloom filter for Python 3 built on mmap
The goal of
pybloomfiltermmap3 is simple: to provide a fast, simple, scalable, correct library for Bloom filters in Python.
There are a couple reasons to use this module:
- It natively uses mmaped files.
- It is fast (see benchmarks).
- It natively does the set things you want a Bloom filter to do.
After you install, the interface to use is a cross between a file interface and an ste interface. As an example:
>>> import pybloomfilter >>> fruit = pybloomfilter.BloomFilter(100000, 0.1, '/tmp/words.bloom') >>> fruit.update(('apple', 'pear', 'orange', 'apple')) >>> len(fruit) 3 >>> 'mike' in fruit False >>> 'apple' in fruit True
To create an in-memory filter, simply omit the file location:
>>> cakes = pybloomfilter.BloomFilter(10000, 0.1)
Caveat: it is currently not possible to persist this filter later.
Current docs are available at pybloomfiltermmap3.rtfd.io.
$ pip install pybloomfiltermmap3
and you should be set.
Note to Python 2 to < 3.5 users
This library is specifically meant for Python 3.5 and above. As of 2020, we strongly advise you to switch to an actively maintained distribution of Python 3. If for any reason your current environment is restricted to Python 2, please see pybloomfiltermmap. Please note that the latter is not actively maintained and will lack bug fixes and new features.
History and Future
pybloomfiltermmap is an excellent Bloom filter implementation for Python 2 by @axiak and contributors. I, @prashnts, made initial changes to add support for Python 3 sometime in 2016 as the current pybloomfiltermmap3 on
PyPI. Since then, with the help of contributors, there have been incremental improvements and bug fixes while maintaining the API from versions
0.4.x and below.
Some new features and changes were first introduced in version
0.5.0. From this point on, the goal is to reach stability, as well as add a few more APIs to expand upon the use cases. While we can't guarantee that we won't change the current interface, the transition from versions
0.4.x and below should be quick one liners. Please open an issue if we broke your build!
Suggestions, bug reports, and / or patches are welcome!
Contributions and development
When contributing, you should set up an appropriate Python 3 environment and install the dependencies listed in
Package installation depends on a generated
pybloomfilter.c file, which requires Cython module to be in your current environment.
See the LICENSE file. It's under the MIT License.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.