Python binding for xxHash
Project description
xxhash is a Python binding for the xxHash library by Yann Collet.
Installation
$ pip install xxhash
Installing From Source
$ pip install --no-binary xxhash xxhash
Prerequisites
On Debian/Ubuntu:
$ apt-get install python-dev gcc
On CentOS/Fedora:
$ yum install python-devel gcc redhat-rpm-config
Linking to libxxhash.so
By default python-xxhash will use bundled xxHash, we can change this by specifying ENV var XXHASH_LINK_SO:
$ XXHASH_LINK_SO=1 pip install --no-binary xxhash xxhash
Usage
Module version and its backend xxHash library version can be retrieved using the module properties VERSION AND XXHASH_VERSION respectively.
>>> import xxhash
>>> xxhash.VERSION
'1.4.0'
>>> xxhash.XXHASH_VERSION
'0.7.1'
This module is hashlib-compliant, which means you can use it in the same way as hashlib.md5.
update() – update the current digest with an additional stringdigest() – return the current digest valuehexdigest() – return the current digest as a string of hexadecimal digitsintdigest() – return the current digest as an integercopy() – return a copy of the current xxhash objectreset() – reset state
md5 digest returns bytes, but the original xxh32 and xxh64 C APIs return integers. While this module is made hashlib-compliant, intdigest() is also provided to get the integer digest.
Constructors for hash algorithms provided by this module are xxh32() and xxh64().
For example, to obtain the digest of the byte string b'Nobody inspects the spammish repetition':
>>> import xxhash
>>> x = xxhash.xxh32()
>>> x.update(b'Nobody inspects')
>>> x.update(b' the spammish repetition')
>>> x.digest()
b'\xe2);/'
>>> x.digest_size
4
>>> x.block_size
16
More condensed:
>>> xxhash.xxh32(b'Nobody inspects the spammish repetition').hexdigest()
'e2293b2f'
>>> xxhash.xxh32(b'Nobody inspects the spammish repetition').digest() == x.digest()
True
An optional seed (default is 0) can be used to alter the result predictably:
>>> import xxhash
>>> xxhash.xxh64('xxhash').hexdigest()
'32dd38952c4bc720'
>>> xxhash.xxh64('xxhash', seed=20141025).hexdigest()
'b559b98d844e0635'
>>> x = xxhash.xxh64(seed=20141025)
>>> x.update('xxhash')
>>> x.hexdigest()
'b559b98d844e0635'
>>> x.intdigest()
13067679811253438005
Be careful that xxh32 takes an unsigned 32-bit integer as seed, while xxh64 takes an unsigned 64-bit integer. Although unsigned integer overflow is defined behavior, it’s better not to make it happen:
>>> xxhash.xxh32('I want an unsigned 32-bit seed!', seed=0).hexdigest()
'f7a35af8'
>>> xxhash.xxh32('I want an unsigned 32-bit seed!', seed=2**32).hexdigest()
'f7a35af8'
>>> xxhash.xxh32('I want an unsigned 32-bit seed!', seed=1).hexdigest()
'd8d4b4ba'
>>> xxhash.xxh32('I want an unsigned 32-bit seed!', seed=2**32+1).hexdigest()
'd8d4b4ba'
>>>
>>> xxhash.xxh64('I want an unsigned 64-bit seed!', seed=0).hexdigest()
'd4cb0a70a2b8c7c1'
>>> xxhash.xxh64('I want an unsigned 64-bit seed!', seed=2**64).hexdigest()
'd4cb0a70a2b8c7c1'
>>> xxhash.xxh64('I want an unsigned 64-bit seed!', seed=1).hexdigest()
'ce5087f12470d961'
>>> xxhash.xxh64('I want an unsigned 64-bit seed!', seed=2**64+1).hexdigest()
'ce5087f12470d961'
digest() returns bytes of the big-endian representation of the integer digest:
>>> import xxhash
>>> h = xxhash.xxh64()
>>> h.digest()
b'\xefF\xdb7Q\xd8\xe9\x99'
>>> h.intdigest().to_bytes(8, 'big')
b'\xefF\xdb7Q\xd8\xe9\x99'
>>> h.hexdigest()
'ef46db3751d8e999'
>>> format(h.intdigest(), '016x')
'ef46db3751d8e999'
>>> h.intdigest()
17241709254077376921
>>> int(h.hexdigest(), 16)
17241709254077376921
Besides xxh32/xxh64 mentioned above, oneshot functions are also provided, so we can avoid allocating XXH32/64 state on heap:
xxh32_digest(bytes, seed=0)xxh32_intdigest(bytes, seed=0)xxh32_hexdigest(bytes, seed=0)xxh64_digest(bytes, seed=0)xxh64_intdigest(bytes, seed=0)xxh64_hexdigest(bytes, seed=0)
>>> import xxhash
>>> xxhash.xxh64('a').digest() == xxhash.xxh64_digest('a')
True
>>> xxhash.xxh64('a').intdigest() == xxhash.xxh64_intdigest('a')
True
>>> xxhash.xxh64('a').hexdigest() == xxhash.xxh64_hexdigest('a')
True
>>> xxhash.xxh64_hexdigest('xxhash', seed=20141025)
'b559b98d844e0635'
>>> xxhash.xxh64_intdigest('xxhash', seed=20141025)
13067679811253438005L
>>> xxhash.xxh64_digest('xxhash', seed=20141025)
'\xb5Y\xb9\x8d\x84N\x065'
In [1]: import xxhash
In [2]: %timeit xxhash.xxh64_hexdigest('xxhash')
268 ns ± 24.1 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [3]: %timeit xxhash.xxh64('xxhash').hexdigest()
416 ns ± 17.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
Caveats
SEED OVERFLOW
xxh32 takes an unsigned 32-bit integer as seed, and xxh64 takes an unsigned 64-bit integer as seed. Make sure that the seed is greater than or equal to 0.
ENDIANNESS
As of python-xxhash 0.3.0, digest() returns bytes of the big-endian representation of the integer digest. It used to be little-endian.
DONT USE XXHASH IN HMAC
Though you can use xxhash as an HMAC hash function, but it’s highly recommended not to.
xxhash is NOT a cryptographic hash function, it is a non-cryptographic hash algorithm aimed at speed and quality. Do not put xxhash in any position where cryptographic hash functions are required.
Copyright and License
Copyright (c) 2014-2019 Yue Du - https://github.com/ifduyue
Licensed under BSD 2-Clause License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for xxhash-1.4.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef73978ffb59236470f5f3e3fe9c97b38657a0aafc3d8948d4454ab5d6598aa2 |
|
MD5 | 502ed1b59404267e6f20feaa0521f99f |
|
BLAKE2b-256 | 8213c3b22a12d3739bd6bbf90955902386cd8d69a81c00fd9b0523f53010c4e0 |
Hashes for xxhash-1.4.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59b03a984b78300cfe6efe7575c8ca8c1bb65cd935b1e4104d3818743d062bc4 |
|
MD5 | 214be654992393ffaca80b8c92420dc3 |
|
BLAKE2b-256 | 992f8d6d224335c8c1de42fce93f61b2ed45771b834cb07f3a57e1bdb0b02eb8 |
Hashes for xxhash-1.4.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f492b1f8f0e695f1b562ac4476c2897fa179a4787d1c701d9fbe8b1e0cf20d8 |
|
MD5 | 45fa4138d367e8a3c3f79c931b394411 |
|
BLAKE2b-256 | 0370e93483b4d6d0575c793589cb3ab2e8974b667eab6d1bf4dae4a7e283984b |
Hashes for xxhash-1.4.1-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4fe42981d44dc6b36b7326ad1ff2b456f3ddc6ab6a2cb729278a6a56ebfea61 |
|
MD5 | da785d5f7a96c9da62117cc653285aa6 |
|
BLAKE2b-256 | 23221d0dbc2d5cb9b19b3211410e66fa444b3bbe906946d785a6730a4fdf336a |
Hashes for xxhash-1.4.1-cp37-cp37m-macosx_10_6_intel.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f728bd122ac08bc59651a5672d66b1d79becc1bd440c6b72d768255091401fb1 |
|
MD5 | 94cefa73315f3bebe0daa8d66256bc9f |
|
BLAKE2b-256 | 2111c7bdfce424520dc9053eebce7f9bfb683b842926d86f29feefa2224d3a9a |
Hashes for xxhash-1.4.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dfab08af15b7201ae7fb716aabaf07a54ca5edfe3e2af7746244a52f9af7c821 |
|
MD5 | b69f58443cb0924cf1f33d81ecda0b59 |
|
BLAKE2b-256 | cb0a94b378a32c972a7e1111303333d2d4329c53d1e5b4ce17ece642c9efddc4 |
Hashes for xxhash-1.4.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 731663ac550679fcd984e8f804b1f2ebcfda54bb9b1699f301723b00026126b4 |
|
MD5 | 8dc2745a3c4a13ed9dc02b467a7d2865 |
|
BLAKE2b-256 | f7e0b5ae9d849b2b33027137132a5d6f28f2203cd969ea4caa04631657de9e0f |
Hashes for xxhash-1.4.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ecb8011d329ecddd832798f041db08087a44423c5b5a0efa26d0fb2ef2442928 |
|
MD5 | 8a31aeadc0933a654e0447cd670ea6d1 |
|
BLAKE2b-256 | e7a973da7edaf9fea5cb8c8a94fa730faf31ea1c5edeabbae71889296b1c57e7 |
Hashes for xxhash-1.4.1-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2dbe77de27a2580e5c17e0495cba88e3b71d56233ff2114afa4ada28d123b738 |
|
MD5 | 63c78337e1dd94512e1b55a6fe8758cb |
|
BLAKE2b-256 | f1ac7cf7edf48b063a6f512900eea2c12183a4527a1dadcc4556359d2f895373 |
Hashes for xxhash-1.4.1-cp36-cp36m-macosx_10_6_intel.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dc4d8d3a2710bb97ef34ffece0eab269169d33d929ca60e67a0ed1d3689d7586 |
|
MD5 | 8e6adfe88db503a262e5f949b0e871f7 |
|
BLAKE2b-256 | 3f8a3ea9451c78e26c46870e257de448d1ee2dedbbe62dc229ac7d2c0b90f74d |
Hashes for xxhash-1.4.1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 85dec3fcffb90f5c0aa4ef8c241817003971b53d7d012de342a3699fe619fde6 |
|
MD5 | e083c61a438732661e92f721bfa011b7 |
|
BLAKE2b-256 | f864d00350ea0ac564e74b16372763fae9b2556b94be62c3e04af51ba3282f0c |
Hashes for xxhash-1.4.1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f03d3577488bc3f8da407df45f21dbdcd111ab21d132aa164b1ea29d099fae6 |
|
MD5 | ac54b12b19f19a564638a9e7fe84d805 |
|
BLAKE2b-256 | fa61666ec1da1003f1cde3724863cd2ebf2ccd6b235c68b27fe8bae19efaa039 |
Hashes for xxhash-1.4.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed8f34a2c7fe0eb3e2ec7b97e62b1fd1a74fcf01510156d9e3660a946801cfd3 |
|
MD5 | 65325a754acc4fc6cbe4b59ad67f424d |
|
BLAKE2b-256 | 865fdb0c6afcc2d46c1933737fb10804185cbc8394dac1a6eba6c3606baa7e58 |
Hashes for xxhash-1.4.1-cp35-cp35m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c559930ab30e37c231d2f1fecaee44ac1d4231fc3e9870500c175fc22c404631 |
|
MD5 | 9e5ae17baf25cc0cd5a012a8e7f5fa85 |
|
BLAKE2b-256 | 8944dce9ebfababe77900641f1239966be39335be251534cc4ddf1d1852e37a1 |
Hashes for xxhash-1.4.1-cp35-cp35m-macosx_10_6_intel.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a520aabd91fe8226f2ae58956f47a237c9a0889603c91b0ff44b7743c9e8f08 |
|
MD5 | 632079bb0aa75091da60388e93060830 |
|
BLAKE2b-256 | 79fa3cb2414d213ba4bcb4000e0ee622d25a4ebbf5919026d8b9b4a079a46ef5 |
Hashes for xxhash-1.4.1-cp34-cp34m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19f7c8563bbf08bae48fb3bdf202c4003ec1738e8b91926c3515f6dfc89cefb9 |
|
MD5 | ba54a3de82ce5fd506e6c669dbaa4d1a |
|
BLAKE2b-256 | 509195b73ef3471c4d100e78a0df48f4598ebf2be0b15cff02c3a618439f1ccf |
Hashes for xxhash-1.4.1-cp34-cp34m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ae71d5832fd0654717b64accc0eeee028895e2dff4c4b51185f59527bdc47be1 |
|
MD5 | dfb0d8e5555236fa2acbf0b0f7c86672 |
|
BLAKE2b-256 | ed51233f93034f6fb404a046fe5609297b57d4c9c9a64e9bc1937a6282de51bf |
Hashes for xxhash-1.4.1-cp34-cp34m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a7f9dc0d33095e56cd38f689e1b28a16def9d29de52612b8fdfb9f2001642a75 |
|
MD5 | aaefdbfaab025129f1fbe4109f631daa |
|
BLAKE2b-256 | 9acfea2247ddb84be2743585f47f604fe9b4926bb05f9a8df6dfe079e17b4725 |
Hashes for xxhash-1.4.1-cp34-cp34m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e859cafa09ed8dc4e7a184d056b37fe644b63457ca9675ac718cc81721c45c70 |
|
MD5 | 7930f83c39e4f73e236eb64117338df9 |
|
BLAKE2b-256 | 1519941c5a7201b18a812a7fbe56081699b48e868e5fb3ec3ae0a0837af15558 |
Hashes for xxhash-1.4.1-cp34-cp34m-macosx_10_6_intel.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 736c95e21d7949e1d68d32c463e64fd7b96dd4131416f1c2617da255456e9b95 |
|
MD5 | cc392ebfcbc5959fd4f12174be883dc9 |
|
BLAKE2b-256 | 41b17e24733713c44ae425e156efa5177abc2b5c61ef83df4bf553c5d5d4f96c |
Hashes for xxhash-1.4.1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4be5018b30f908d184cfb093f5a13cc000aa44e01c15fd14beb9422399b7b382 |
|
MD5 | eff5da7ccf55719f59eae73043b1c6a5 |
|
BLAKE2b-256 | 8040d59d37d551e7e325c37f465e897c15d2117477121fc76903c27e234526b6 |
Hashes for xxhash-1.4.1-cp27-cp27mu-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a79f9cd38c2ff2f72824de51d0863853b2ae239f18fa8c9c15dfcb880538ca4 |
|
MD5 | 549f419942e45c590824eda6fa7f4500 |
|
BLAKE2b-256 | 1831da249f66a83d2fe2f4ea6ee5884a4392341727657926d36d598d14cc70ac |
Hashes for xxhash-1.4.1-cp27-cp27m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 796be264ce21499229fdf479e4bc1e7eb26535572100b8c551bab39f047d100d |
|
MD5 | c9aa93a607079c7189fe06e7a04f9400 |
|
BLAKE2b-256 | d3da6dd1221d13cd81ba13eda60f7dfaed83929a7f67abae77e2e597addf3238 |
Hashes for xxhash-1.4.1-cp27-cp27m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11d53d8bb64615d891f847607c5c9be17814eaca0b7751bcd38b5e2559f0d825 |
|
MD5 | 2f460cc4314aa7da47e995f46e52fe49 |
|
BLAKE2b-256 | dd255b42fbb0ae9e9f7a40dcbfec3020cd39c6f9ae83d38c36dc5a6013575096 |
Hashes for xxhash-1.4.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 820c0363f22471a248b78c07c33885c779c0e6e8ee663deefc95ca104b9810b2 |
|
MD5 | 643c92a4b54e1329b2111df98ec5f851 |
|
BLAKE2b-256 | 1a9e7c60f784c7a643cb9297a455d97b83b8bc5cb4db6f5cc7f187c5ed20f5e6 |
Hashes for xxhash-1.4.1-cp27-cp27m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7fa9b550b2cefdda34f51729bd968441a69fb439c4c028588c505769960cadab |
|
MD5 | fd0a221ce90a71373485a53f562913b8 |
|
BLAKE2b-256 | 5986084242a0a31f27729286edebd7b7bf43b2236ceb252c874915c0014c6f96 |
Hashes for xxhash-1.4.1-cp27-cp27m-macosx_10_6_intel.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5722ef12d581a863005c2e257eb2d2e90fbf11596b9ddaa66aa70aa84eefb232 |
|
MD5 | 66bbb0ae4d78172a9b4491f54bc874e4 |
|
BLAKE2b-256 | d483fc5cfdf0a9e8ef8c0502611b8f89d1d3807e9fad0e9ec4e40c1ce18f1f98 |