hashcodecs

Total Downloads Monthly Downloads

SIMD-accelerated Base64, MurmurHash3, and XXH3 for Python and Rust.

Move byte-heavy work into Rust without changing your Python inputs. hashcodecs accepts bytes, bytearray, and memoryview, selects the best available SIMD backend, and exposes batch and reusable-buffer APIs.

Features

Base64 encode and decode with standard, URL-safe, padded, unpadded, wrapped, and canonical modes.
MurmurHash3 x86-32, x86-128, and x64-128 with one-shot and incremental APIs.
Bit-for-bit compatible XXH3-64 and XXH3-128 with one-shot and native batch APIs.
Caller-managed *_into outputs for allocation-sensitive workloads.
Runtime dispatch across AVX-512, AVX2, SSE4.1, SSSE3, NEON, and scalar implementations where applicable.
Direct CPython buffer handling for bytes, bytearray, and memoryview inputs.
Install wheels for CPython 3.10 through 3.15 and free-threaded CPython 3.14t and 3.15t on Linux, macOS, and Windows.

Installation

pip3 install hashcodecs

Python

The Base64 module follows familiar Python conventions while adding explicit padding, canonical validation, batch, and reusable-buffer operations.

import hashcodecs.base64 as base64
from hashcodecs import murmur3_32, xxh3_64, xxh3_128_batch

assert base64.b64encode(b'hello') == b'aGVsbG8='
assert base64.b64decode(b'aGVsbG8=') == b'hello'
assert base64.urlsafe_b64encode(b'hello', padded=False) == b'aGVsbG8'

assert murmur3_32(b'hello') == 0x248BFA47
assert xxh3_64(b'') == 0x2D06800538D394C2
assert xxh3_128_batch([b'hello', b'world']) == [
    0xB5E9C1AD071B3E7FC779CFAA5E523818,
    0xFA0D38A9B38280D0891E4985BDB2583E,
]

Reusable outputs

*_into functions write into caller-managed bytearray storage and return the number of bytes written. XXH3 batch outputs are packed little-endian digests.

import hashcodecs.base64 as base64
from hashcodecs import xxh3_64_batch_into

payload = b'hello'
encoded = bytearray(4 * ((len(payload) + 2) // 3))
encoded_len = base64.b64encode_into(payload, encoded)
assert encoded[:encoded_len] == b'aGVsbG8='

hashes = bytearray(2 * 8)
written = xxh3_64_batch_into([b'hello', b'world'], hashes, seed=42)
assert written == 16

Incremental hashing

from hashcodecs import murmur3_x64_128

hasher = murmur3_x64_128(seed=42)
hasher.update(b'hello')
snapshot = hasher.copy()
hasher.update(b' world')

assert snapshot.hexdigest() == snapshot.digest().hex()
assert hasher.digest() != snapshot.digest()

Rust

The Rust API exposes the same core algorithms without the Python binding layer.

cargo add hashcodecs

let encoded = hashcodecs::base64::b64encode(b"hello");
assert_eq!(encoded, "aGVsbG8=");

let mut output = [0_u8; 8];
let written = hashcodecs::base64::b64encode_into(b"hello", &mut output).unwrap();
assert_eq!(&output[..written], b"aGVsbG8=");

assert_eq!(
    hashcodecs::murmur3::murmur3_x86_32(b"hello", 0),
    0x248b_fa47
);
assert_eq!(
    hashcodecs::xxhash::xxh3_64(b"", 0),
    0x2d06_8005_38d3_94c2
);

Architecture

The Rust core owns algorithm behavior and SIMD dispatch. A substantial CPython layer handles argument parsing, buffer ownership, reusable outputs, and GIL decisions; root-level Python modules provide typed public exports without adding per-call wrappers.

Each Rust algorithm exposes a small public façade. Base64 groups internals by encode and decode operation and places ISA kernels such as encode/avx2.rs and decode/ssse3.rs under their operation. MurmurHash3 groups code by canonical variant. XXH3 uses processing-stage modules, with long-input ISA kernels under xxhash/long/.

See docs/ARCHITECTURE.md for the module layout, dispatch model, algorithm data flows, CPython boundary, and safety invariants.

Benchmarks

Run the suite on Windows 10 x64 with an Intel Core Ultra 7 265K. Pin one logical CPU and run each case in one thread. Collect 50 Rust samples and 15 Python samples. Higher throughput wins.

Base64: Rust

MurmurHash3: Rust

XXH3: Rust

Link hashcodecs with xxHash 0.8.3 through xxhash-c-sys. Build the C baseline with AVX2. Batch cases pass 32 equal-size inputs and include result-vector allocation.

Base64: Python

Pass bytes to Rust without an input copy. Python decoding uses validate=True.

MurmurHash3: Python

XXH3: Python

Pass 32 equal-size inputs to each batch case. Compare one native hashcodecs call with a loop over the upstream xxhash extension.

Read the focused cases, commands, and values in BENCHMARK.md. Read raw chart values in docs/benchmarks/results.csv.

Reproduce

Comparison crates and Python packages are development-only dependencies and are not included in consumer builds.

cargo bench --bench base64
cargo bench --bench murmur3
cargo bench --bench xxhash

uv sync --group benchmark --no-install-project
uv run --no-project --with . python benchmarks/python_base64.py
uv run --no-project --with . python benchmarks/python_base64_batch.py
uv run --no-project --with . python benchmarks/python_calls.py
uv run --no-project --with . python benchmarks/python_murmur3.py
uv run --no-project --with . python benchmarks/python_xxhash.py

The Python benchmarks expose focused modes such as --into, --lenient, --bytearray-input, --memoryview-input, --sliced-memoryview-input, --incremental, --large, and --hashcodecs-only. All scripts also accept --samples and --minimum-sample-seconds; use --help on a benchmark script for its supported modes and defaults.

For the same-ISA Windows XXH3 comparison shown above, rebuild the C baseline with:

$env:CFLAGS='/O2 /arch:AVX2'
cargo clean -p xxhash-c-sys
cargo bench --bench xxhash

Performance snapshot

In the full 2026-08-23 hashcodecs-only run on the benchmark host, hashcodecs.xxh3_64 processes a 1 MiB input at 79.20 GiB/s. With 256 B items in batches of 64, the Base64 batch API reaches 8.80 GiB/s for encode and 7.58 GiB/s for decode. The run pins one logical CPU and uses 15 samples with a 0.2-second minimum per sample. Read the benchmark details and raw comparison results.

Development

Build the Python wheel and source distribution:

uv build

Run the primary local checks:

cargo fmt --check
cargo clippy --all-targets --features python -- -D warnings
cargo test --features python
uv run --frozen --no-sync ruff check . --no-cache
uv run --frozen --no-sync ruff format --check .
uv run --frozen --no-sync pytest tests --cov=hashcodecs --cov-branch --cov-fail-under=100

Optimized paths are also checked with differential fuzzing, Kani, strict-provenance Miri, AddressSanitizer, and MemorySanitizer in CI.

Compatibility

Version 1.x keeps the documented Python API stable under the compatibility policy. Release wheels target CPython 3.10 through 3.15 on manylinux x86-64, macOS 11+ ARM64, and Windows x86-64. CPython 3.14t and 3.15t receive free-threaded wheels on the same platforms.

The Rust crate and Python package share one release version. The documented Python API follows the compatibility policy below. See Security Policy for vulnerability reporting.

References

The Base64 SIMD implementation follows the approach described in Faster Base64 Encoding and Decoding using AVX2 Instructions, extended with runtime-selected AVX-512 VBMI and AArch64 NEON backends.

Citation

If you use hashcodecs, cite CITATION.cff.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hashcodecs-1.2.1.tar.gz (159.3 kB view details)

Uploaded Aug 26, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

hashcodecs-1.2.1-cp315-cp315t-win_amd64.whl (349.0 kB view details)

Uploaded Aug 26, 2026 CPython 3.15tWindows x86-64

hashcodecs-1.2.1-cp315-cp315t-manylinux_2_28_x86_64.whl (480.0 kB view details)

Uploaded Aug 26, 2026 CPython 3.15tmanylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp315-cp315t-macosx_11_0_arm64.whl (403.4 kB view details)

Uploaded Aug 26, 2026 CPython 3.15tmacOS 11.0+ ARM64

hashcodecs-1.2.1-cp315-cp315-win_amd64.whl (342.4 kB view details)

Uploaded Aug 26, 2026 CPython 3.15Windows x86-64

hashcodecs-1.2.1-cp315-cp315-manylinux_2_28_x86_64.whl (473.3 kB view details)

Uploaded Aug 26, 2026 CPython 3.15manylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp315-cp315-macosx_11_0_arm64.whl (398.4 kB view details)

Uploaded Aug 26, 2026 CPython 3.15macOS 11.0+ ARM64

hashcodecs-1.2.1-cp314-cp314t-win_amd64.whl (349.1 kB view details)

Uploaded Aug 26, 2026 CPython 3.14tWindows x86-64

hashcodecs-1.2.1-cp314-cp314t-manylinux_2_28_x86_64.whl (479.9 kB view details)

Uploaded Aug 26, 2026 CPython 3.14tmanylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp314-cp314t-macosx_11_0_arm64.whl (403.3 kB view details)

Uploaded Aug 26, 2026 CPython 3.14tmacOS 11.0+ ARM64

hashcodecs-1.2.1-cp314-cp314-win_amd64.whl (342.5 kB view details)

Uploaded Aug 26, 2026 CPython 3.14Windows x86-64

hashcodecs-1.2.1-cp314-cp314-manylinux_2_28_x86_64.whl (472.9 kB view details)

Uploaded Aug 26, 2026 CPython 3.14manylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp314-cp314-macosx_11_0_arm64.whl (398.6 kB view details)

Uploaded Aug 26, 2026 CPython 3.14macOS 11.0+ ARM64

hashcodecs-1.2.1-cp313-cp313-win_amd64.whl (329.7 kB view details)

Uploaded Aug 26, 2026 CPython 3.13Windows x86-64

hashcodecs-1.2.1-cp313-cp313-manylinux_2_28_x86_64.whl (471.3 kB view details)

Uploaded Aug 26, 2026 CPython 3.13manylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp313-cp313-macosx_11_0_arm64.whl (397.1 kB view details)

Uploaded Aug 26, 2026 CPython 3.13macOS 11.0+ ARM64

hashcodecs-1.2.1-cp312-cp312-win_amd64.whl (329.2 kB view details)

Uploaded Aug 26, 2026 CPython 3.12Windows x86-64

hashcodecs-1.2.1-cp312-cp312-manylinux_2_28_x86_64.whl (471.0 kB view details)

Uploaded Aug 26, 2026 CPython 3.12manylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp312-cp312-macosx_11_0_arm64.whl (396.7 kB view details)

Uploaded Aug 26, 2026 CPython 3.12macOS 11.0+ ARM64

hashcodecs-1.2.1-cp311-cp311-win_amd64.whl (334.1 kB view details)

Uploaded Aug 26, 2026 CPython 3.11Windows x86-64

hashcodecs-1.2.1-cp311-cp311-manylinux_2_28_x86_64.whl (476.2 kB view details)

Uploaded Aug 26, 2026 CPython 3.11manylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp311-cp311-macosx_11_0_arm64.whl (400.7 kB view details)

Uploaded Aug 26, 2026 CPython 3.11macOS 11.0+ ARM64

hashcodecs-1.2.1-cp310-cp310-win_amd64.whl (334.3 kB view details)

Uploaded Aug 26, 2026 CPython 3.10Windows x86-64

hashcodecs-1.2.1-cp310-cp310-manylinux_2_28_x86_64.whl (476.4 kB view details)

Uploaded Aug 26, 2026 CPython 3.10manylinux: glibc 2.28+ x86-64

hashcodecs-1.2.1-cp310-cp310-macosx_11_0_arm64.whl (401.0 kB view details)

Uploaded Aug 26, 2026 CPython 3.10macOS 11.0+ ARM64

File details

Details for the file hashcodecs-1.2.1.tar.gz.

File metadata

Download URL: hashcodecs-1.2.1.tar.gz
Upload date: Aug 26, 2026
Size: 159.3 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for hashcodecs-1.2.1.tar.gz
Algorithm	Hash digest
SHA256	`e71f9181af4f3a9d7f5c202ae5d1fe163f7b94aa46b439f040b0488fc60ccc4b`
MD5	`c1aecf717fc013bfb0bc90921e847e0f`
BLAKE2b-256	`fca9b1adf31d7dc9e2dae10963979421477486397224601475e4d622e69a6a14`

Algorithm	Hash digest
SHA256	`8e0aee60b50c8767982ac6116c1d0ffb6d9dfe35f73170a428ef5d9a8738952f`
MD5	`b1fe1f97f9f3015f76ac979c70c40037`
BLAKE2b-256	`ffc2d755b9774326b31bf7a6a4ae26b7f0a41e04fba67b13d044a9d37e7b4eb1`

Algorithm	Hash digest
SHA256	`58c337c4c2163305cf0aa4ad743c486f59c28967f78e98b71fdb332f1797f64c`
MD5	`a3a8cfb2765999f7692cb351ffc24a21`
BLAKE2b-256	`8d379219ee585a493b825a50bcadff5757c01e602564a16fc963a19c3fe7c246`

Algorithm	Hash digest
SHA256	`36e3fae127952522feea81ee4b82b1943b65b39fb087bf3bc6b0d9cbf52b5b3c`
MD5	`2e649ed44b671dcaee594f9315c7ec50`
BLAKE2b-256	`d0dcaebb1fb8235fd4c4210f03bc377dfd752e6684cd2f3f420cdfd718f5fc01`

Algorithm	Hash digest
SHA256	`90e8e53830081304b561e353893a48cbffbf861c7116af940295a22ae98c213b`
MD5	`d88fc8f2b960240faf647002f8f21e99`
BLAKE2b-256	`1dc80761efe984b261a85b04b5b523a4e21fa69e70d870341aad95b13ebf3fa3`

Algorithm	Hash digest
SHA256	`ec9605955d4e1eb762dbd08ad09f0ada3c83dd288fc839fec25a17644dea00a4`
MD5	`054d6778559b81624a8c86a775ce3aaf`
BLAKE2b-256	`c06fb9e228411933acddc18fd675d926a9548e87494d41221c9c64a614b11377`

Algorithm	Hash digest
SHA256	`8d08b2e40348fcc72207e37f184f57b5237fff3c8d8948dff33607f3b5813fff`
MD5	`8c2f33ec76da94277561f9fbfc34bbb3`
BLAKE2b-256	`280251c7243b77b884fe7c5d15f04d4febe69fd51530e84d99268f384734fba1`

Algorithm	Hash digest
SHA256	`e07f0fc8762ff3b290c5baa7706e77e3eb48ef213e32308012860bb153d68ba6`
MD5	`c9819ad9fd2f5ba8a60b64a9d34ec5ae`
BLAKE2b-256	`ed608386dbf54af906b6987ab9af468353a3d00690e327d36d9df42d454d8e22`

Algorithm	Hash digest
SHA256	`4beeaccb71e645aff75a867eafe677bde695e69d9aa40a0ae72591e1250f1c6c`
MD5	`895ce036fd2df68ca808bec37ebfb2c3`
BLAKE2b-256	`6638f70429824f72477cd928032d019339d737bbcd61a18748a74d0c2e4b2553`

Algorithm	Hash digest
SHA256	`6a6d324b1734dbebc571e1eec5447a1c062251d8eb9eefe910e7e53fc4dd9ce3`
MD5	`61bef8532aa83cb27c30712cbb43f25e`
BLAKE2b-256	`7df16a9557c1e872e89bbee6534e251dcf531302a72d4da54a1f848d2bc4373c`

Algorithm	Hash digest
SHA256	`e9a7219b35503b185cec39d837234cf5bd7bdd8139d52a14648b64dffb319526`
MD5	`c95de232d336b8c2e264efc6a82f1ae6`
BLAKE2b-256	`5694ca6749a6896cec24b377316592d962333b8906b7a1fb5af22bf8810a9fd6`

Algorithm	Hash digest
SHA256	`1e8b978ee84d1368454d46bbaace98717750790af5308a289be8b1617dea2735`
MD5	`bff068bcffd830f8d2d283d3617d009b`
BLAKE2b-256	`6ac9120d70b558e2a1b7ac4134488e58c6204e419ec026e3f1dd1648a6ec661c`

Algorithm	Hash digest
SHA256	`ad86503fc8893b0c71ac59f1eeffd2d2171abc2999a9039fa7e45df870b89010`
MD5	`9d16124213efe7495c82eb68b28721d9`
BLAKE2b-256	`5514d9c2bcd7f72fc59db61a07eb49038fde75225607cf04547b5fb7d2f3d845`

Algorithm	Hash digest
SHA256	`65e321475258563c71e9c7b4d3ea6c3f551a2bca09fcc1b5515b78708436b65f`
MD5	`22d2e287a55526f65d05464eb0533d47`
BLAKE2b-256	`d8c81542e65875a17412467a3838dcab3a48abe4bd0863e65c0d35760cc2516b`

Algorithm	Hash digest
SHA256	`18bb20d06224918bb3cde9ca816db1b962a2f0af08173a2e898eb15f161534fb`
MD5	`cacbaa8c86183d13ac0b40b4387bc110`
BLAKE2b-256	`dffc19e5dd7fbd55cf877e40c493b6cdf985bf7289552b90af180415d05b4c32`

Algorithm	Hash digest
SHA256	`5740369f10e89c3f419e17746ab4fc960a3e0dc1c559b78197db3fd162ac8c25`
MD5	`f14b5baf3ad6a909af21ab10376154c7`
BLAKE2b-256	`ac3df518d162e86220df4e997d1809243de1587bd285ca0eec7517937a81bc87`

Algorithm	Hash digest
SHA256	`fb9753a78968fc1a0391eaa3fc0f9e3f4f5a8c944f5b53c12ed8461fb836ca1b`
MD5	`fc7a50e9864dae047ebde5232f62e84f`
BLAKE2b-256	`c167e2e5cdae38fb32632e599f20c3d148459d2eb1436f35cd4d93ae1e232397`

Algorithm	Hash digest
SHA256	`ceadb222c37b12fb404bc95b5ce93932e04f523beea009d2c07030c1bf320822`
MD5	`90a4db5d3e4225b5c365fee32c54e526`
BLAKE2b-256	`65083761c5090b28b3d2719f60e18d533f187290e55265f75a8ab6a332f8ad65`

Algorithm	Hash digest
SHA256	`1bdd53288a5349aaf476383dc173de690eb3e4608d63eebcacb266a5209def43`
MD5	`b81e58e4860102228e4b22333bdc65fe`
BLAKE2b-256	`7db01a8ec97a2b2c5c71f3b1d1f26a008a0178648ef6a3e6b85d3e4155cd2dd3`

Algorithm	Hash digest
SHA256	`990377be7bf1ad6f87583de01ae051c528f3e410e4f15fc095d0a455a4dfaa49`
MD5	`863069c25969ea00916f5acbe7961a3c`
BLAKE2b-256	`57bd2cf06f8e71d55f8db0694f0ed8f60ae4dc21736cbac258c06ed52e9e6afa`

Algorithm	Hash digest
SHA256	`af26a72db31fa26ba8618dbe830ebf5abb55477baddc0d4de8eed8d5631ec386`
MD5	`18bc990648a1691fc8c7a683f21b31ae`
BLAKE2b-256	`727a0413cf550ea1f7d69040ed1bff014b9aa2b991ae8a7e6848275ab094deb0`

hashcodecs 1.2.1

hashcodecs

Features

Installation

Python

Reusable outputs

Incremental hashing

Rust

Architecture

Benchmarks

Base64: Rust

MurmurHash3: Rust

XXH3: Rust

Base64: Python

MurmurHash3: Python

XXH3: Python

Reproduce

Performance snapshot

Development

Compatibility

References

Citation

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License expression

Requires

Tags

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

Algorithm	Hash digest
SHA256	`8d6fed65cdb965021f3f2338cc5a22de1b3fa9fa79be000778c7fe8a1de847cf`
MD5	`92782164f1e3b9a292fc9f21591c2113`
BLAKE2b-256	`af2dff70d6a5e414f880aa6f180de46b1a6f416dac2212d7a152eb28b9086e04`

Algorithm	Hash digest
SHA256	`e9f5dd6be39af87ca3384b302a39cef7e2c29a6ab180a90eead2343ddb4df82c`
MD5	`16228740e3eae3202b3730fa6212567b`
BLAKE2b-256	`9b9e61663b6443492c5b49fbdabe68fed5a38985f3c78a500d3bc5fbdddd9a6a`

Algorithm	Hash digest
SHA256	`947593d77991e7c9b70d6009f575fef673ecbb4806f6bd81537bc221b5e89718`
MD5	`33b04eb4a7c7b77d4905aa92693dab81`
BLAKE2b-256	`40323c93ede315c2d9d231abe724050f620c0905b500753bb36eaab9c0b80dc0`

Algorithm	Hash digest
SHA256	`f1bc3c0af64f48c00d6790e3438d456f5b74db18262ca5a3a5494613aa7298b6`
MD5	`2e5854830d68de3db5e5156481ff6dd0`
BLAKE2b-256	`17adbc7e2d61de747b7e4055bcb16628f5aeb13a06ff38df62745b5981506750`