Skip to main content

A python package implementing the crc32c algorithm in hardware and software

Project description

https://github.com/ICRAR/crc32c/workflows/Build%20and%20release%20to%20PyPI/badge.svg?branch=master https://badge.fury.io/py/crc32c.svg

This package implements the crc32c checksum algorithm. It automatically chooses between a hardware-based implementation (using the CRC32C SSE 4.2 instruction of Intel CPUs, and the crc32* instructions on ARMv8 CPUs), or a software-based one when no hardware support can be found.

Because crc32c is in PyPI, you can install it with:

pip install crc32c

Supported platforms are Linux and OSX using the gcc and clang compilers, and Windows using the Visual Studio compiler. Other compilers in Windows (MinGW for instance) might work. Binary wheels are also provided in PyPI for major platforms/architectures.

Usage

The only method exposed by this module is crc32c(data, [crc]). It computes the CRC32C checksum of data starting with an initial crc checksum, similarly to how the built-in binascii.crc32 works. It can thus be used like this:

print(crc32c.crc32c(b'hello world'))
# 3381945770
crc = crc32c.crc32c(b'hello')
print(crc32c.crc32c(b' world', crc))
# 3381945770

In older versions, the function exposed by this module was called crc32. That name is still present but deprecated, and will be removed in new versions of the library.

Additionally one can consult the following module-level values:

  • hardware_based indicates if the algorithm in use is software- or hardware-based.

  • big_endian indicates whether the platform is big endian or not.

Implementation details

By default, if your CPU doesn’t have CRC32C hardware support, the package will fallback to use a software implementation of the crc32c checksum algorithm. This behavior can be changed by setting the CRC32C_SW_MODE environment variable to one of the following values:

  • auto: same as if unset, will eventually be discontinued.

  • force: use software implementation regardless of hardware support.

  • none: fail to import the module with an ImportError if no hardware support is found (old 1.x default behavior).

The software algorithm is based on Intel’s slice-by-8 package, with some adaptations done by Evan Jones and packaging provided by Ferry Toth. Further adaptations were required to make the code more portable and fit for inclusion within this python package.

The Intel SSE 4.2 algorithm is based on Mark Adler’s code, with some modifications required to make the code more portable and fit for inclusion within this python package.

The ARMv8 hardware implementation is based on Google’s crc32c C++ library.

License

This package is licensed under the LGPL-2.1 license.

The original slice-by-8 software algorithm is licensed under the 2-clause BSD licence.

Further modifications to the slice-by-8 software algorithm are licensed under a 3-clause BSD licence

The original Intel SSE 4.2 crc32c algorithm’s code is licensed under a custom license embedded in the crc32c_adler.c file.

The original crc32c ARMv8 hardware code is licensed under a 3-clause BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crc32c-2.3.post0.tar.gz (38.5 kB view hashes)

Uploaded source

Built Distributions

crc32c-2.3.post0-cp311-cp311-win_amd64.whl (32.2 kB view hashes)

Uploaded cp311

crc32c-2.3.post0-cp311-cp311-win32.whl (30.9 kB view hashes)

Uploaded cp311

crc32c-2.3.post0-cp310-cp310-win_amd64.whl (32.2 kB view hashes)

Uploaded cp310

crc32c-2.3.post0-cp310-cp310-win32.whl (30.9 kB view hashes)

Uploaded cp310

crc32c-2.3.post0-cp39-cp39-win_amd64.whl (32.2 kB view hashes)

Uploaded cp39

crc32c-2.3.post0-cp39-cp39-win32.whl (30.9 kB view hashes)

Uploaded cp39

crc32c-2.3.post0-cp38-cp38-win_amd64.whl (32.2 kB view hashes)

Uploaded cp38

crc32c-2.3.post0-cp38-cp38-win32.whl (30.9 kB view hashes)

Uploaded cp38

crc32c-2.3.post0-cp37-cp37m-win_amd64.whl (32.2 kB view hashes)

Uploaded cp37

crc32c-2.3.post0-cp37-cp37m-win32.whl (30.9 kB view hashes)

Uploaded cp37

crc32c-2.3.post0-cp36-cp36m-win_amd64.whl (32.1 kB view hashes)

Uploaded cp36

crc32c-2.3.post0-cp36-cp36m-win32.whl (30.8 kB view hashes)

Uploaded cp36

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page