Bloom filters and other filter algorithms based on memory and redis
Project description
由python实现的一系列高效的过滤器算法实现
- 基于redis和memory
- 低时间复杂度
Useage
from pyfilters import MemoryBloomFilter
bf = MemoryBloomFilter(10000, 0.00001)
for i in range(1000):
bf.add(i)
for i in range(1000):
assert i in bf
assert 1001 not in bf
Advanced usage
- 计数形布隆过滤器,可以删除数据
from pyfilters import CountMemoryBloomFilter
cbf = CountMemoryBloomFilter(10000, 0.00001)
for i in range(1000):
cbf.add(i)
for i in range(1000):
assert i in cbf
cbf.remove(1)
assert 1 not in cbf
- redis分块布隆过滤器,避免单key过大
from redis import Redis
from pyfilters import ChunkedRedisBloomFilter
bf = ChunkedRedisBloomFilter(Redis(), "test_bloomfilter", 10000, 0.00001)
for i in range(1000):
bf.add(i)
for i in range(1000):
assert i in bf
assert 1001 not in bf
- 分块计数形redis布隆过滤器,可以删除数据
from redis import Redis
from pyfilters import CountRedisBloomFilter
rcbf = CountRedisBloomFilter(Redis(), "test_countbloomfilter", 10000, 0.00001)
for i in range(1000):
rcbf.add(i)
for i in range(1000):
assert i in rcbf
rcbf.remove(1)
assert 1 not in rcbf
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyfilters-0.1.2.tar.gz
(7.5 kB
view hashes)
Built Distribution
Close
Hashes for pyfilters-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0c5a0089c156c3bd8c10c3896a20ff663fbab21b94b7674917fb29b77412b6ea |
|
MD5 | 1b6d67f46f053bd6d0a9809a14c4dec0 |
|
BLAKE2b-256 | ee7620d42dd1c8eb6e0c0cf3ec697e1c03635214b63eef3516a184b20fc13c8f |