Python bindings for C implementation of xorfilter
Project description
pyxorfilter
Python bindings for C implementation of Xor Filters: Faster and Smaller Than Bloom and Cuckoo Filters and of Binary Fuse Filters: Fast and Smaller Than Xor Filters.
Installation
pip install pyxorfilter
From Source
git clone --recurse-submodules https://github.com/glitzflitz/pyxorfilter
cd pyxorfilter
python setup.py build_ext
python setup.py install
Usage
>>> from pyxorfilter import Xor8, Xor16, Fuse8, Fuse16
>>> filter = Xor8(5) #or Xor16(size)
>>> #Supports unicode strings and heterogeneous types
>>> test_str = ["あ","अ", 51, 0.0, 12.3]
>>> filter.populate(test_str)
True
>>> filter.contains("अ")
True
>>> filter[51] #You can use __getitem__ instead of contains
True
>>> filter["か"]
False
>>> filter.contains(150)
False
>>> filter.size_in_bytes()
60
Caveats
Accuracy
For more accuracy(less false positives) use larger but more accurate Xor16 for Fuse16.
For large sets (contain millions of keys), Fuse8/Fuse16 filters are faster and smaller than Xor8/Xor16.
>>> filter = Xor8(1000000)
>>> filter.size_in_bytes()
1230054
>>> filter = Fuse8(1000000)
>>> filter.size_in_bytes()
1130536
Overflow
Both Xor8/Fuse8 and Xor16/Fuse16 take uint8_t and uint_16t respectively. Make sure that the input is unsigned.
TODO
- Add unit tests
- Add CI support for distributing pyxorfilter with PyPI.
Links
- C Implementation
- Go Implementation
- Erlang bindings
- Rust Implementation: 1 and 2
- C++ Implementation
- Java Implementation
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyxorfilter-1.0.0.tar.gz
(9.1 kB
view hashes)
Built Distributions
Close
Hashes for pyxorfilter-1.0.0-pp39-pypy39_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 29e7187dfa1a64e3ec37b5dd5279e2790a3a271fbaabe707a870e6dd8d989cdd |
|
MD5 | 610c0540ceaec734f7aa17e1099788ad |
|
BLAKE2b-256 | 0f1439e52d89347a575c09694b45c3d05d01d86205b9e1d7a1a9f7a6e6d42e75 |
Close
Hashes for pyxorfilter-1.0.0-pp39-pypy39_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | af963437a7e2b5e88128fecffb2642bfc1ae954267081ac6fc31d8c984f8ce0a |
|
MD5 | 912839c5f5fdfd196d405a49465429a6 |
|
BLAKE2b-256 | be4c4fc7b9b8d462be13c9ce6898578fca0df252e682243eba00902b3eec0309 |
Close
Hashes for pyxorfilter-1.0.0-pp38-pypy38_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69adaba90c201415d07deaa77f4fea56121ec7f2665a8de78e46d71412739981 |
|
MD5 | a89b6814b5e0e2c15115d83417569682 |
|
BLAKE2b-256 | 9f328057813c71180f2bd147e06b0c56e0fcf249c2b70408935f96cd60740c1b |
Close
Hashes for pyxorfilter-1.0.0-pp38-pypy38_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 94ecf045bc621c53106a302c7a2a70ab091dca8c2c045d88abf16d69da2b67e5 |
|
MD5 | 8073547bbfed3edf27c3df9f4591fb3c |
|
BLAKE2b-256 | ad6b8d6817286b5fe5c17dc9cedc419829be68dc1c0bdebc8053654e1e874f22 |
Close
Hashes for pyxorfilter-1.0.0-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 914fbe3693a06fdce4d5d3ecc33120ff931f806c0bc735d74a1da4bcc740d75a |
|
MD5 | bd0eebec2d206bf6a57ea95ad18ef4ee |
|
BLAKE2b-256 | f3e2791eda17460dc9f97d89b45e50fc356654cd26728b73b291b8cb1c5742cb |
Close
Hashes for pyxorfilter-1.0.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4e48918bcb27f5cc42f3f77b229c0b5cbb5a2746d0bc579c95f5001af4450f8 |
|
MD5 | a5d6bab15cf9f47e78ac538c39fcc3b4 |
|
BLAKE2b-256 | ee2f261fa7b9ef47ccd7eb766cef324c072afbe40e510c22aa4e3ba280f09fa6 |
Close
Hashes for pyxorfilter-1.0.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce766aba16a03ef951771cfae501684300bb27aaeb3fa2ad29cf168e32f10a5b |
|
MD5 | 18942e9aca047d87c0db9cbe0bb2a9f1 |
|
BLAKE2b-256 | f050a31164caba94f4bd64decfdc057ff3f5c59cf062bfd6ae333530d278d2d4 |
Close
Hashes for pyxorfilter-1.0.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6c19a8c3c07c117792da400aa414fc4d8b3aba27c5ae80cbbe2ab4644af56b8 |
|
MD5 | 9a0b6ae7710b532f71832021a28847b6 |
|
BLAKE2b-256 | 7ce7626394fde70fe4e3c115a2094cd5c96f44bfc785591544cb85af8670b0d8 |
Close
Hashes for pyxorfilter-1.0.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9efaeb6c13a052ae45a130336653e64b8beacc94dec8ae612296e666ba9fce00 |
|
MD5 | 12a8033bf333d559a80e493aa9423830 |
|
BLAKE2b-256 | 49068441c3e5c703f796df9e3c12dd2d36432dfb567fb7360b3eb2d3902a0cd5 |
Close
Hashes for pyxorfilter-1.0.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 21caf73c2f0abad3ce3c1a3995c6b8d15c64bdbe8d49eafa302481f8e845b5f6 |
|
MD5 | f0c3611999a4cb3f6ee373e31c2f5c54 |
|
BLAKE2b-256 | 1649a65dacdab194bab670ec36895cd6c2b6025a337144f666dae41916e9126c |
Close
Hashes for pyxorfilter-1.0.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 179216d58c616883dd69f61be39d559c2adbd8b74bb1e3a84a97d73212598b84 |
|
MD5 | a36235b0859dddb686d2ffac8f03cc04 |
|
BLAKE2b-256 | 02d4ef13553e44b78a2220b2ffcd2a9ee0cce9cbb7ddc9357a67f839dde5b682 |
Close
Hashes for pyxorfilter-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 49e900c961999b2285531da265f72ffdeb301b6395a4cb3f6513aaf3f47a1e10 |
|
MD5 | a5d9a2fa909c0e0a9b3fe6378e767364 |
|
BLAKE2b-256 | b6255ee558bed7b199d00ffcbbadcc876b269d73ce0a5e06306b245eaee608aa |
Close
Hashes for pyxorfilter-1.0.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72506a487536591af606f9a36c6b0a5f28ca6cab9d2e836ecff50adfa4039b25 |
|
MD5 | 3ac3408a1e1101010942ff9962db1b98 |
|
BLAKE2b-256 | a012e724403a8ddeeff338bbfc3e01e69a39790fc8f20281f82f4ec6847af87c |
Close
Hashes for pyxorfilter-1.0.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ae7008bb631ab0641236536874ace48c02d2f92798c770537398167649b6baf8 |
|
MD5 | bdcada665dd3b3a3d257bbeec97befca |
|
BLAKE2b-256 | 6637d4d2a07e33860eadd5bfa503acd9fb4693ad2af828c1fc107679411cea5a |
Close
Hashes for pyxorfilter-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 25a4e5b3f53d266e243cf258932c983a76acdf62399370ca0f84d3a79b72f228 |
|
MD5 | 1113b97ce519328139d53ede9d49df29 |
|
BLAKE2b-256 | 09aa2ea9d44f66c0654659a071dd67485b1de3219e657a0ca4557dcf9d843127 |
Close
Hashes for pyxorfilter-1.0.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f93e40ec4545a50ea805b82bc11b67e6022ddd54deb8c820100af8d9c8cc5587 |
|
MD5 | 7379be79e91bd94e9d0fd0839239b553 |
|
BLAKE2b-256 | d7dc960aff9ff87c65427cc8ad3a76d72533c961792e2d854fa73eb338a7e300 |
Close
Hashes for pyxorfilter-1.0.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 086252ec9ba289a708b31c465f0ddd0b6b68fb3c8ae6f0f4baf9d4ca9a33bfa3 |
|
MD5 | 887157f07ec12e0cf8ffc67ec99e178f |
|
BLAKE2b-256 | f6dc2226f05e529a45a96649a5bcd6930e14e19783da0c2e35233b17ca244252 |
Close
Hashes for pyxorfilter-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59b7f7c3e8d8eca14e72eef3e2343dca08ea39379fc49cc2ec1b739f448ad56c |
|
MD5 | f681113290c2089f03810307dc564010 |
|
BLAKE2b-256 | 04d61dbe60551386d17450fbf18ac8a381c980559d1c9710aa1e7284ecbb0a7b |
Close
Hashes for pyxorfilter-1.0.0-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3cd6f949a15dcec9b167347ab324fb6c8e4ff7acc3d0cbe69420fee205ba4c8 |
|
MD5 | 8511267694d760e4d247e363115da205 |
|
BLAKE2b-256 | dc8507703c89383e701605ab2459cb402ece4006e52f2a43d4fec85ff711e757 |
Close
Hashes for pyxorfilter-1.0.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb11beea868b62d8715615bd90519ed502411f6736cc0cd6b19aba84da6d37fb |
|
MD5 | f757818af720f6dd9ccb385e8a510c5c |
|
BLAKE2b-256 | 4a5452259c7a047b7a2cd8edbed31670c27c2bfcaf655dfdf58e115e7ceff6ab |
Close
Hashes for pyxorfilter-1.0.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 654669c26a0e144aaca57cf04aa9afc203906c70b448d47b71d06b8106e01875 |
|
MD5 | 7526de13330fdac5dbdd0333b9381517 |
|
BLAKE2b-256 | ee352439029f67af9854e54f698fb7b32c3c90e1eee806f26747479572d3c1bc |