bloomfilter-py

Overview

Yet another Bloomfilter implementation in Python, compatible with Java's Guava library.

I was looking for a Python library which is capable of reading what Bloomfilter of Java's Guava library serializes and is also able to output byte array which is recognizable by Java. But unfortunately failed. Hence I developed this library by borrowing how Guava implements Bloomfilter serialization/deserialization a lot to deal with Bloomfilters on both Python and Java sides.

As for Bloomfilter usage in Java world, please refer to this post.

Here's a brief introduction to Bloomfilter.

Requirements

• Python 3.8+

Install

pip install bloomfilter-py

Usage Examples

Basic Usage

>>> from bloomfilter import BloomFilter
>>> bloom_filter = BloomFilter(expected_insertions=500, err_rate=0.01)
>>> for i in range(100):
...     bloom_filter.put(i)
...
>>> 1 in bloom_filter
True
>>> 100 in bloom_filter
False
>>>

Serialize Bloomfilter

You can easily serialize BloomFilter instance to a byte array

>>> dumps = bloom_filter.dumps()
>>> with open("dumps.out", "wb") as f:
...     f.write(dumps)
...
>>>

or to a hex string

>>> hex_str = bloom_filter.dumps_to_hex()

or to a base64 encoded bytes

base64_bytes = bloom_filter.dumps_to_base64()

Deserialize Bloomfilter

And you can easily initialize a BloomFilter instance from a byte array

>>> with open("dumps.out", "rb") as f:
...     bf = BloomFilter.loads(f.read())
...
>>> 1 in bf
True
>>> 100 in bf
False
>>>

or from a hex string

>>> bf = BloomFilter.loads_from_hex(hex_str)
>>> 1 in bf
True
>>> 100 in bf
False

or from a base64 encoded bytes

>>> bf = BloomFilter.loads_from_base64(base64_bytes)
>>> 100 in bf
False
>>> 200 in bf
False
>>> 1 in bf
True
>>> 99 in bf
True

Development and Testing

From the repository root, create an isolated virtual environment and install the project in editable mode with pytest:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e . pytest

Run the complete test suite:

python -m pytest -v

To run the same formatting, type, and security checks used during development, install the additional tools and execute:

python -m pip install black mypy bandit
python -m black --check bloomfilter tests
python -m mypy bloomfilter --strict
python -m bandit -r bloomfilter

On Windows PowerShell, activate the virtual environment with .venv\Scripts\Activate.ps1 instead of source .venv/bin/activate.