Python bindings for C implementation of xorfilter
Project description
pyxorfilter
Python bindings for C implementation of Xor Filters: Faster and Smaller Than Bloom and Cuckoo Filters and of Binary Fuse Filters: Fast and Smaller Than Xor Filters.
Installation
pip install pyxorfilter
From Source
git clone --recurse-submodules https://github.com/glitzflitz/pyxorfilter
cd pyxorfilter
python setup.py build_ext
python setup.py install
Usage
>>> from pyxorfilter import Xor8, Xor16, Fuse8, Fuse16
>>> filter = Xor8(5) #or Xor16(size)
>>> #Supports unicode strings and heterogeneous types
>>> test_str = ["あ","अ", 51, 0.0, 12.3]
>>> filter.populate(test_str)
True
>>> filter.contains("अ")
True
>>> filter[51] #You can use __getitem__ instead of contains
True
>>> filter["か"]
False
>>> filter.contains(150)
False
>>> filter.size_in_bytes()
60
You can serialize a filter with the serialize()
method which returns a buffer, and you can recover the filter with the deserialize(buffer)
method, which returns a filter:
> f = open('/tmp/output', 'wb')
> f.write(filter.serialize())
> f.close()
> recoverfilter = Xor8.deserialize(open('/tmp/output', 'rb').read())
Caveats
Accuracy
For more accuracy(less false positives) use larger but more accurate Xor16 for Fuse16.
For large sets (contain millions of keys), Fuse8/Fuse16 filters are faster and smaller than Xor8/Xor16.
>>> filter = Xor8(1000000)
>>> filter.size_in_bytes()
1230054
>>> filter = Fuse8(1000000)
>>> filter.size_in_bytes()
1130536
TODO
- Add unit tests
- Add CI support for distributing pyxorfilter with PyPI.
Links
- C Implementation
- Go Implementation
- Erlang bindings
- Rust Implementation: 1 and 2
- C++ Implementation
- Java Implementation
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyxorfilter-1.1.2.tar.gz
(9.6 kB
view hashes)
Built Distributions
Close
Hashes for pyxorfilter-1.1.2-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd6d9418bd88314b4f1af7589416418fccdaa57ba121566ba48b9dfc66fc847c |
|
MD5 | 68645a85f67b11ad70092f1a4e5bd34e |
|
BLAKE2b-256 | a6a4af0bee9ba8c69b5689046c4652ff1b662660d3b7418d4d1b540aedc4170d |
Close
Hashes for pyxorfilter-1.1.2-pp310-pypy310_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de9fe655d8ffadb8c3b3227eec58e7f9cc092c94f8b6cd30d983c9d4661546ac |
|
MD5 | e79b1bfff588ff810a351449343df1c6 |
|
BLAKE2b-256 | 2eaba499a0b02bc6a181bffaaabfd0e6a11216028f3ce4aa3555f0c4253c2a9a |
Close
Hashes for pyxorfilter-1.1.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab8a4ede9df460b8079f0104c1a4a7339688199969152d118dda63e3b7767e45 |
|
MD5 | 1039c622a9c7b919a3a09f958f9ebea0 |
|
BLAKE2b-256 | 34434e8eefcb30f066a484bf05b06fbb07460a92479d6ceedf16b4411d0e24d0 |
Close
Hashes for pyxorfilter-1.1.2-pp39-pypy39_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 251e3d9c87ef705b8ef6d8e30002aab3b62e40b249352945ba840f678eee2c94 |
|
MD5 | d109d5dac307145a15cebb77e860cfd7 |
|
BLAKE2b-256 | 32d9b75600640058b9c1d67de2c9ba1441597ba71a9961aed84a5edf97898a72 |
Close
Hashes for pyxorfilter-1.1.2-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f0f41e09a520e740526aa814ac85cc587e09300361419e5527011549a24d6a3 |
|
MD5 | de7e27dd2eec8581a82b5645bf40c4f4 |
|
BLAKE2b-256 | 68033a5f1043a67c64ca425356d742eb15f1692faa524d74ada21a523dff6d87 |
Close
Hashes for pyxorfilter-1.1.2-pp38-pypy38_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72aa08d2d3c923769c2698f62711c39cc2dcb8d581f70fabc6bae28745cf934c |
|
MD5 | 8b9e5f91747c960970bd7c324207570e |
|
BLAKE2b-256 | 92645a94d69e988c8f49f2537362bcc684db01a9dc49db719e0ca9f1abd9e343 |
Close
Hashes for pyxorfilter-1.1.2-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55a2d5d6787553fedc75b216f226fec6893d8bd5f4a08e506616ed2459be4c48 |
|
MD5 | dad3d52e5af5e24ca700536441c263b8 |
|
BLAKE2b-256 | 156fc84641fb1d7397ed59b290421e732372a41eceaea9cd5a671b1053573cdf |
Close
Hashes for pyxorfilter-1.1.2-pp37-pypy37_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69dc65c5219f8368def7130253ed749d1b5963b4ce21ae12e331ee04685d8c09 |
|
MD5 | 4eb54ab96c9417a87c27c5be87e6524f |
|
BLAKE2b-256 | 32649d9bdb9cc3f4b67398fc1ff111573b5f4a7a476804a48c2c80317e67dcbc |
Close
Hashes for pyxorfilter-1.1.2-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4324103b4226a2fe99491a0ef29e887c9bd488226b18650875df2d4fc9c290d8 |
|
MD5 | 09c6ce50b6d0fea83f7198cb32ff105e |
|
BLAKE2b-256 | 70b4c905c16ecd35eeb3a330446243785c584764cd130250dd3ae9dcecb45eb8 |
Close
Hashes for pyxorfilter-1.1.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6726661a04dfb83fc59e5375a55a29370da9793f38de26d6734e2aaea8d2879 |
|
MD5 | 4c5bdc2d25c31489cad8752801daf721 |
|
BLAKE2b-256 | 0910185a98b392d522edf33cdb9347adb679a65fe36ec5ff9603f53cdf7f5dfa |
Close
Hashes for pyxorfilter-1.1.2-cp312-cp312-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3877c16bd221cafd93860b14b28cdc531715e5ad252fc09ff14680291440ed5 |
|
MD5 | 6a9fdbca6170cc3046fd7155d554b28d |
|
BLAKE2b-256 | dc0dd68a4cb07e54f8374595f45e8f0f63928c37ebe12a8e11a94f23232d0e9d |
Close
Hashes for pyxorfilter-1.1.2-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97ab8e9071d7e02d0ebc35a8f5cc4c37a50033e6fe698c300c065ea08706c0dc |
|
MD5 | f693fe4a158a3812dac4e9fda52157e0 |
|
BLAKE2b-256 | 1d26f6fdd67082839edc76484e126ce5abe99c1be96231f247eb65ca396b294c |
Close
Hashes for pyxorfilter-1.1.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9802a3140ac971cf3a0ec76fdc4258d8f8a38903c960c9749b14f16158637389 |
|
MD5 | 0db9b005fa20188e78a16de215b32f9b |
|
BLAKE2b-256 | 62f3bffcbe6c3c79b8fa4fa90e5c88791fa4b6a75ad3f7de0127112ccec4ef37 |
Close
Hashes for pyxorfilter-1.1.2-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3b266b035b5e43c5fd83c97506c0d65ddd796ccdefc58ae249a74072fe05d77 |
|
MD5 | 2ae53b20aa2682e182ad71b65393080e |
|
BLAKE2b-256 | 07796d81705f517ddd238baae0db47dc594aa0855ad29629d3d5177f33faca28 |
Close
Hashes for pyxorfilter-1.1.2-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bac2448a435e67b96ae464f34aa51af1c861bad4f4161330b2e0abef933c8d18 |
|
MD5 | ee5ce97257f0d0ab29150f0e019d40fc |
|
BLAKE2b-256 | 3378f61e8d276451e96760c2ede14d34d7ef0fb36f26e029a82ed52b23f2e73c |
Close
Hashes for pyxorfilter-1.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3691b8691a412f13de72e637ca52be3ef4838656ef1834fc821674392fff9f1 |
|
MD5 | 94aed0e8a7ce3b1489081407f30a316c |
|
BLAKE2b-256 | 717d6f8a297c37062eee10327a81fc9a489b645406a8c8db1961d8ce2c554524 |
Close
Hashes for pyxorfilter-1.1.2-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 90b1c82984d8851d76d9a42c510a116d376fc5b7277a8656d37aa8eb7ea064ce |
|
MD5 | 7d3da448659f162940f075ba86d2a1b8 |
|
BLAKE2b-256 | a0c37660f2dfdcf852b915c067e29b6256b933d3cefaecdb3a2420faa9f2c5e4 |
Close
Hashes for pyxorfilter-1.1.2-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1c68528ec7325c58989cc3de6a7efe955c7b9b792100bd5cc8bc744c4cdb3940 |
|
MD5 | e47ec649f5f05fd1a5c64a2791fb9cac |
|
BLAKE2b-256 | 481b14a4cc02c3c12b6661b7f4653cb4bd40c154a005bb568286c425ea6c9d49 |
Close
Hashes for pyxorfilter-1.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4bfc4654570eac511f940cf46ceb98be6b1cfd5e988151b89646584bd894bd4 |
|
MD5 | 87e7d32654d0e553f736a5ba1b09a8f2 |
|
BLAKE2b-256 | ac7a7c5a9205dd2ac103a1322d7927e4470f275d5451dbb9f655d3fd896adcdf |
Close
Hashes for pyxorfilter-1.1.2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a92ae3824577050dbd384e4b10015d0edb4dc042331bef72820777aa7e076356 |
|
MD5 | 2ab74dec34d6ecd0d42f7802d8b2f98d |
|
BLAKE2b-256 | 2c20333fe93aaaa97eeb0f69fb87c969b4646db00d7cfa74e7607cb5739cf5a4 |
Close
Hashes for pyxorfilter-1.1.2-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ceff57f6b12cb0c3ce2ced2c2992212f79cc25310c46740f7cbb18a4d6d32ed |
|
MD5 | d3329fe28eed3a044d5e9d7ed939e65c |
|
BLAKE2b-256 | d153798a6014ee557d7b8968dfe6cf59fb19e3dc460d4ee32536e4ab1ee5496c |
Close
Hashes for pyxorfilter-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dc78e2aacbe070feff224f7516b0955a411edf79612e8bc095cd1824223e6928 |
|
MD5 | 2d7416b93f93f066bd81ff841631230a |
|
BLAKE2b-256 | 56d893641f38589bf1176dd5539d5d5954dd872b0c6bf5eaaadbdef36628c4d8 |
Close
Hashes for pyxorfilter-1.1.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd34d53dd75109e64671004c54b401c740f5b1bfbd2216e3778d734047c780bb |
|
MD5 | f513cac6cc1300a577c9fd131e0ad45b |
|
BLAKE2b-256 | 3519a64b12d73c76916673323148b33e3837ac2fa7c4ef553d06b23b84f6db0c |
Close
Hashes for pyxorfilter-1.1.2-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f22a103ad75fca43a6c9500d014811d7015975938529287ba616f91ccfa53160 |
|
MD5 | 3fc85aca5e3343ace1fe67ba019f1503 |
|
BLAKE2b-256 | 2a09b3ae9bf3195f1d5348cb02ebcc84b834adbc88e9c7e5af7b698ba4e93336 |
Close
Hashes for pyxorfilter-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1bd70d2c20010e1cb2f18a461f58547f90a84b8f84db7c6bdf656dd58f34a60d |
|
MD5 | 00bfe7d3326f81800e86f5c42185b2fc |
|
BLAKE2b-256 | 9266be312b3822b48bfcf0c3cd02b3027426521b6085aba37b103f35a6fa528f |
Close
Hashes for pyxorfilter-1.1.2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55500ccc249f371ad98d79ddb158745816d88239919b3f7a0cb8c6b98330466b |
|
MD5 | 13666c45ea5aa5ba1f88674bede46a36 |
|
BLAKE2b-256 | fd1772b9ea8924ad0c4d3bbed8225c648a55994812703f4eae1823d744aa41e0 |
Close
Hashes for pyxorfilter-1.1.2-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 02e6af1625dd58028f57da8f7b956a223d549785a2e2c0ff1c606dbcf92953ec |
|
MD5 | 3d861502aa3af29e3e8ff37aa7d74b41 |
|
BLAKE2b-256 | 26b15fc32f7a8c44e6935c5c4c1f68434c7c7ae07039ed116a28e70d7d16df0f |
Close
Hashes for pyxorfilter-1.1.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 937ffb5553e93c62998402f6c658d40efb7a4c4754b95e9e6ae0f9159fbf2a82 |
|
MD5 | 143b0f8f0d8a4e6e732f5f7d29ca7e49 |
|
BLAKE2b-256 | 06dcd8f01941ab36a5fa11fcadbba417e12808de8cc4a400b7b6c7b4e1bae3e8 |
Close
Hashes for pyxorfilter-1.1.2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed304740763ca0992e0f6cf97be8c8a9ab0be39f59f8dc8418984432e328f748 |
|
MD5 | 075638b5cbd53850f989528028024c23 |
|
BLAKE2b-256 | bca08265f433eaa1a064f5a5e4e0fd3ef31faa6a394b07afd356436cfa8a4bd9 |