Python bindings for CityHash and FarmHash
Project description
CityHash/FarmHash
Python wrapper for FarmHash and CityHash, a family of fast non-cryptographic hash functions.
Getting Started
To use this package in your program, simply type
pip install cityhash
This package exposes Python APIs for CityHash and FarmHash under cityhash
and
farmhash
namespaces, respectively. Each provides 32-, 64- and 128-bit
implementations.
Usage Examples
Stateless hashing
Usage example for FarmHash:
>>> from farmhash import FarmHash32, FarmHash64, FarmHash128
>>> FarmHash32("abc")
1961358185
>>> FarmHash64("abc")
2640714258260161385
>>> FarmHash128("abc")
76434233956484675513733017140465933893
Hardware-independent fingerprints
Fingerprints are seedless hashes which are guaranteed to be hardware- and platform-independent. This can be useful for networking applications which require persisting hashed values.
>>> from farmhash import Fingerprint128
>>> Fingerprint128("abc")
76434233956484675513733017140465933893
Incremental hashing
CityHash and FarmHash do not support incremental hashing and thus are not ideal for hashing of streams. If you require incremental hashing feature, use MetroHash or xxHash instead, which do support it.
Fast hashing of NumPy arrays
The Python Buffer Protocol allows Python objects to expose their data as raw byte arrays to other objects, for fast access without copying to a separate location in memory. Among others, NumPy is a major framework that supports this protocol.
All hashing functions in this packege will read byte arrays from objects that expose them via the buffer protocol. Here is an example showing hashing of a 4D NumPy array:
>>> import numpy as np
>>> from farmhash import FarmHash64
>>> arr = np.zeros((256, 256, 4))
>>> FarmHash64(arr)
1550282412043536862
The arrays need to be contiguous for this to work. To convert a non-contiguous
array, use NumPy's ascontiguousarray()
function.
SSE4.2 support
For Mac and Linux x86-64 platforms, the PyPi repository for this package includes wheels compiled with SSE4.2 support. Although it also includes wheels that work on Windows, the Windows wheels were not compiled with SSE4.2 support---pull requests that address this are welcome!
The 32- and 64-bit FarmHash variants both significantly benefit from SSE4.2 instructions. The 128-bit version, unfortunately, does not exhibit speed up after compiling with SSE4.2 support.
The vanilla CityHash fucntions (under cityhash
module) do not take advantage
of SSE4.2. Instead, the cityhashcrc
module provided with this package (Mac
and Linux platforms only) exposes 128- and 256-bit CRC functions which do
harness SSE4.2. These functions are very fast, and beat FarmHash128
on speed
(FarmHash does not include a 256-bit function). Since FarmHash is the intended
successor of CityHash, I would be careful before using the CityHash-CRC
functions, however, and would verify whether they provide sufficient randomness
for your intended application.
Development
Local workflow
For those who want to contribute, here is a quick start using some makefile commands:
git clone https://github.com/escherba/python-cityhash.git
cd python-cityhash
make env # create a Python virtualenv
make test # run Python tests
make cpp-test # run C++ tests
make shell # enter IPython shell
The Makefiles provided have self-documenting targets. To find out which targets are available, type:
make help
Distribution
The wheels are built using cibuildwheel and are distributed to PyPI using GitHub actions using this workflow. The wheels contain compiled binaries and are available for the following platforms: windows-amd64, ubuntu-x86, linux-x86_64, linux-aarch64, and macosx-x86_64.
See Also
For other fast non-cryptographic hash functions available as Python extensions, see MetroHash, MurmurHash, and xxHash.
Authors
The original CityHash Python bindings are due to Alexander [Amper] Marshalov. These were rewritten in Cython by Eugene Scherba, who also added the FarmHash bindings. The CityHash and FarmHash algorithms and their C++ implementation are by Google.
License
This software is licensed under the MIT License. See the included LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for cityhash-0.3.5.post4-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f04158fad9b6f02f73f1b9ebcb7ad7a7faea4cd1eaf21a25b0f428ce4533e35 |
|
MD5 | b0220769f0a1b5c688fd31337cf893db |
|
BLAKE2b-256 | 43c5ea728cdb6e1cf9c630411ded20bdfa3e7faa8027341f3cc242ae5b6b1733 |
Hashes for cityhash-0.3.5.post4-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8438965fbc829c92c14a2f6e89bd268931ee63adfb1e15ea7647869b57bc929 |
|
MD5 | 15d876b8a64835d3b5691671f24e2e99 |
|
BLAKE2b-256 | cc89f4a459dd192e748ab1cf1b78791d0dd527e9bf7101d5bb16d88824d778c8 |
Hashes for cityhash-0.3.5.post4-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b52fb005258c09059902a6f7427fbbddcfc3ed29355054f4aa2292024284c7e6 |
|
MD5 | 7cbc337cbcb825396d362f2d1784755a |
|
BLAKE2b-256 | d883268de1dda925e158165c30f644485670c4d99dd2c4f79bae04f782735947 |
Hashes for cityhash-0.3.5.post4-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 125ccf3c67c072b072beb7ffd71692d0a82e865eb9ee71efb530b8f957464df4 |
|
MD5 | 122d2347bec2ab2d132f17a13e659f60 |
|
BLAKE2b-256 | 57e0d07d9a886aaa9ce11b936c1e5d495871f0a2d0cd4caa1dae1ba2fd4e66d7 |
Hashes for cityhash-0.3.5.post4-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a134035feffbb4914f914bffecbe1395aeffea687d7eac6222c26e507bc038b |
|
MD5 | 82eb066c979b02a250ae5ec1eab3db12 |
|
BLAKE2b-256 | aa0d6bc7464f7c5c57cb54a47cea732f8e59ec4fcf9ab364cf274df32d4944c9 |
Hashes for cityhash-0.3.5.post4-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 096d14e8a8278014355d6e4f98abd18097570d76f078c3645fb01abc934a48ed |
|
MD5 | aeb9707fd8a18217d5899c8dc0774165 |
|
BLAKE2b-256 | 6a9919a2636ce0b975e9814e0c83bbaef8d57a8bac8cc694d8db6bd7a1494d6c |
Hashes for cityhash-0.3.5.post4-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 33ac39dadb955850430eb622ffae6d92d09d73e94fb16b48cb073c2c56042420 |
|
MD5 | 34b959f5994082f1e0c375658b5dac33 |
|
BLAKE2b-256 | d3429f2484dc48c6f34cbdc0ff4493d436b995bcaf3c6a1f504662ea7d75cbb4 |
Hashes for cityhash-0.3.5.post4-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1cd979be73464812de34a55b4632f82d1b3eb4ffd9b7f531dfe470b9f72dee0a |
|
MD5 | 2db3bbd5013ac09c877e4fe8f28e624d |
|
BLAKE2b-256 | d9de83f1ff6d5cb95affdda696b328d37111b4481ddfd5a9864354df71ddec96 |
Hashes for cityhash-0.3.5.post4-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d88b14c2f04a205bbf8fbb50444922732ba5d51039ad2154f5b5ee67bd5aad73 |
|
MD5 | a8b235ff0be4ae832beda1fcb469ae43 |
|
BLAKE2b-256 | e4853630b739cc658607930824a64942aba8d92ece8f60832a21557e3f5ffbff |
Hashes for cityhash-0.3.5.post4-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 22dca9beaeb826a1da6ee295edf2cd16a786adf4499b83e7d83f53944cb7f8af |
|
MD5 | a4e427168affa79b6e188da0c1b0de15 |
|
BLAKE2b-256 | ea36b7d4a888c47f0899f7414969c4762d89c7055216d89b0f6ee26b55375eba |
Hashes for cityhash-0.3.5.post4-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3f7b4f46108f3e8e434c67229fad553745292f445220075459623a44ebd96adc |
|
MD5 | 79ffd58ae46360ff6ae8748122fd03c6 |
|
BLAKE2b-256 | 1addeb0e049d244d3f1e6a26272710a3859a5f1e5f0e1d6d7469b57d56bd0463 |
Hashes for cityhash-0.3.5.post4-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0b5bea62367c7687ac02af176147a5ddb227610aa86fb86df5b038ce7ff17d0 |
|
MD5 | 637e66a145d46a4c113bdc3d3abaa475 |
|
BLAKE2b-256 | ad6b92e1dc524bf1f9809d0bfb8689a118be1287ffe74143b38ed6db156487b4 |
Hashes for cityhash-0.3.5.post4-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fc1eba3623a9f4c26ed58cae080a83e417d86e24662f466ccdc20e855291498b |
|
MD5 | 8b3e9d6771be924b7c597b9d7d79cb04 |
|
BLAKE2b-256 | 5d3a664f613d8f142fee23ac976870eeb7b26adb9f9a2997976a9a62563f4e16 |
Hashes for cityhash-0.3.5.post4-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a2d4cd1d4dac74f42270d96738b70fe34e0693eb5179c083791c14831f08b65 |
|
MD5 | 1371d2d83cfd3d591d497c2c781e55e7 |
|
BLAKE2b-256 | 345b9fbbd43624adb015dc6c84ea16d3ca1643b165b64418db9f2f815ecf769e |
Hashes for cityhash-0.3.5.post4-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0fe99393ff22f4fbe61ae5bd5df1a6dbfeb2f31087666f1ffb241fbb95560d2f |
|
MD5 | 72f73299bc7b3b4cd3dc20a1b91a551c |
|
BLAKE2b-256 | 529cb12e780975aa6730e875305a9c4ad2b1b8e09b3036459d6ce8392a28f1a9 |
Hashes for cityhash-0.3.5.post4-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9fdcc26970b45faf1c9692c47136784e7521be78e92460cc6ba3b54a4eb6c6f9 |
|
MD5 | b331344784a525b8fdeaf217cf53e300 |
|
BLAKE2b-256 | f17272bce87f6c8256ebb321d16f8b5dc1a66fb482690aa8dc090bc24d299132 |
Hashes for cityhash-0.3.5.post4-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4767129c0f198a7be9d2e1d03467f21fc306a1295b230a8ffbb3453acbe344c6 |
|
MD5 | 61a02320a82deed8586c1fd931df8645 |
|
BLAKE2b-256 | 27aa122b376ba7ac29d877e0840431bf741cbb0eae1f6fd07c5639426a753955 |
Hashes for cityhash-0.3.5.post4-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3deecf777d582eb39ac99b3bd78e5ef954f57c9249e5b7c632cc6f0034aa3e8 |
|
MD5 | abb151bcc2304f155bb053a55dfa4d35 |
|
BLAKE2b-256 | 7019986201bcd3dc9e8bc3c2b1bc90d5334dc604800efbee9df065a93d7846b3 |
Hashes for cityhash-0.3.5.post4-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd585eff532c8daf8e6adb74d374b350b5d305ca9eff84f592db9a0af1ce8822 |
|
MD5 | 2da232f330f6eb5b3cb69ab553f5b078 |
|
BLAKE2b-256 | f7315c206bfecd3705036f4959cf040c1a67c9e24940a465f7325ccf210bc197 |
Hashes for cityhash-0.3.5.post4-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef5712f5dc81a70e7ad8fef562e5c7ccbbbb88526f32ac3d80fbbcce809ba984 |
|
MD5 | 75a09634ac072a20f8345425d64618af |
|
BLAKE2b-256 | 9385406b9c8d6b5a3796481dd1ca84126302e7cbcba1ebc43be68ca22b14a106 |
Hashes for cityhash-0.3.5.post4-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 418922c919debc99fb3e870ac392d6df60d35112949cf4b5403a4ddb3aa76c5b |
|
MD5 | 12934bd9d02e14e59202d5ee1bc458a9 |
|
BLAKE2b-256 | 029a82b5bc0956807768929b66dbbef1a658bb2232e1c9c68452e7dc4e8e8c20 |
Hashes for cityhash-0.3.5.post4-cp36-cp36m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 997c0bad78c91bb1b8f4c3d9b60b0f4ca23c9e542e2415ba718c58902a688181 |
|
MD5 | d7aae4839cdd349e37513fb807744ac9 |
|
BLAKE2b-256 | 666dd689ed9bee2f338dce79c4cadf4f37b9bde1859ec2e730d75f1beac018ed |
Hashes for cityhash-0.3.5.post4-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2fa3a2b5740ad8855acc00a449dcf8c106b76c353121b4fa41fb6216bdcd6425 |
|
MD5 | 37a266a75f012ec3015df30cf0dc2fde |
|
BLAKE2b-256 | 0620af67b1e6476d58c7a732d44d1444b69f5254c96f4b8e81416ad9d77f8e46 |
Hashes for cityhash-0.3.5.post4-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 62782f35138383875e3c3846de9415949c8c4da6917d781a0da899747b623019 |
|
MD5 | a6e22a0cca92d8f000a9bb4fa5d6b896 |
|
BLAKE2b-256 | 53f2d208f7c36b570c8f908ce2e5487ed7f27cc9109f48c247b9872d1668ac71 |
Hashes for cityhash-0.3.5.post4-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c85b774c889754052a882ad911acf6998337d703674a3432ca15ebc1eba3f604 |
|
MD5 | c02005e223253081c2d501b6fb6efea5 |
|
BLAKE2b-256 | 2ba8cbea368deaf10c5ae831440f36c6c8a0066cf28e37cbcb819d5e500c4e58 |