Skip to main content

A python wrapper of the C library 'Google CRC32C'

Project description

google-crc32c

Python / CFFI wrapper for google/crc32c

Checking Out

Be sure to check out all submodules:

$ git clone --recursive https://github.com/googleapis/python-crc32c

Prerequisites

On Linux:

  • docker
  • python3.7

On OS X:

  • make
  • Official python.org Python 2.7, 3.5, 3.6 and 3.7

On Windows:

  • cmake
  • Official python.org Python 3.5, 3.6 and 3.7
  • Visual Studio 15 2017 (just the compiler toolchain)

Unfortunately, libcrc32c relies on many C++11 features, so building a Python 2.7 extension with the Visual C++ Compiler for Python 2.7 is infeasible.

Building Wheels

On Linux:

./scripts/manylinux/build.sh

On OS X:

./scripts/osx/build.sh

On Windows: see .appveyor.yml.

Verify Wheels

On Linux (i.e. a host OS, not a docker container):

$ ./scripts/manylinux/check-37.sh
...
+ venv/bin/python check_cffi_crc32c.py
_crc32c_cffi: <module 'crc32c._crc32c_cffi' from '.../py-crc32c/venv/lib/python3.7/site-packages/crc32c/_crc32c_cffi.abi3.so'>
_crc32c_cffi.lib: <Lib object for 'crc32c._crc32c_cffi'>
dir(_crc32c_cffi.lib): ['crc32c_extend', 'crc32c_value']
+ unzip -l wheels/google_crc32c-0.0.1-cp37-cp37m-manylinux1_x86_64.whl
Archive:  wheels/google_crc32c-0.0.1-cp37-cp37m-manylinux1_x86_64.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
    26120  2018-10-25 00:09   crc32c/_crc32c_cffi.abi3.so
      765  2018-10-24 23:57   crc32c/__init__.py
    29552  2018-10-25 00:09   crc32c/.libs/libcrc32c-f865a225.so
      109  2018-10-25 00:09   google_crc32c-0.0.1.dist-info/WHEEL
      766  2018-10-25 00:09   google_crc32c-0.0.1.dist-info/METADATA
      652  2018-10-25 00:09   google_crc32c-0.0.1.dist-info/RECORD
        1  2018-10-25 00:09   google_crc32c-0.0.1.dist-info/zip-safe
        7  2018-10-25 00:09   google_crc32c-0.0.1.dist-info/top_level.txt
---------                     -------
    57972                     8 files
...

On OS X:

$ ./scripts/osx/check.sh
...
+ venv37/bin/python .../py-crc32c/check_cffi_crc32c.py
_crc32c_cffi: <module 'crc32c._crc32c_cffi' from '.../py-crc32c/venv37/lib/python3.7/site-packages/crc32c/_crc32c_cffi.abi3.so'>
_crc32c_cffi.lib: <Lib object for 'crc32c._crc32c_cffi'>
dir(_crc32c_cffi.lib): ['crc32c_extend', 'crc32c_value']
+ /Library/Frameworks/Python.framework/Versions/3.7/bin/delocate-listdeps --all --depending .../py-crc32c/wheels/google_crc32c-0.0.1-cp37-cp37m-macosx_10_6_intel.whl
/usr/lib/libSystem.B.dylib:
    crc32c/_crc32c_cffi.abi3.so
    crc32c/.dylibs/libcrc32c.dylib
/usr/lib/libc++.1.dylib:
    crc32c/.dylibs/libcrc32c.dylib
@loader_path/.dylibs/libcrc32c.dylib:
    crc32c/_crc32c_cffi.abi3.so
...

On Windows: TBD.

Installing locally for testing

Initialize the submodules and build the main libcrc32c.so shared library using cmake / make:

$ cd py-crc32c
$ git submodule update --init --recursive
$ python -m virtualenv venv
$ venv/bin/pip install --upgrade setuptools pip wheel
$ venv/bin/pip install cmake
$ mkdir usr
$ export CRC32C_INSTALL_PREFIX=$(pwd)/usr
$ mkdir crc32c/build
$ cd crc32c/build
$ ../../venv/bin/cmake \
>   -DCRC32C_BUILD_TESTS=no \
>   -DCRC32C_BUILD_BENCHMARKS=no \
>   -DBUILD_SHARED_LIBS=yes \
>   -DCMAKE_INSTALL_PREFIX:PATH=${CRC32C_INSTALL_PREFIX} \
>   ..
$ make all install
$ cd ../..

Now, run the tests:

$ venv/bin/pip install -e .[testing]
$ venv/bin/py.test tests/
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: ..., inifile:
collected 9 items

tests/test___init__.py .........                                         [100%]

=========================== 9 passed in 0.03 seconds ===========================

Project details


Release history Release notifications

This version

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for google-crc32c, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size google_crc32c-0.0.1-cp35-cp35m-macosx_10_6_intel.whl (20.8 kB) File type Wheel Python version cp35 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp35-cp35m-manylinux1_x86_64.whl (29.4 kB) File type Wheel Python version cp35 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp35-cp35m-manylinux2010_x86_64.whl (29.4 kB) File type Wheel Python version cp35 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp35-cp35m-win32.whl (26.1 kB) File type Wheel Python version cp35 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp35-cp35m-win_amd64.whl (31.0 kB) File type Wheel Python version cp35 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp36-cp36m-macosx_10_9_x86_64.whl (19.1 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp36-cp36m-manylinux1_x86_64.whl (28.9 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp36-cp36m-manylinux2010_x86_64.whl (28.9 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp36-cp36m-win32.whl (26.0 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp36-cp36m-win_amd64.whl (30.9 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp37-cp37m-macosx_10_9_x86_64.whl (19.1 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp37-cp37m-manylinux1_x86_64.whl (28.9 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp37-cp37m-manylinux2010_x86_64.whl (28.9 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp37-cp37m-win32.whl (26.0 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp37-cp37m-win_amd64.whl (30.9 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp38-cp38-manylinux1_x86_64.whl (29.4 kB) File type Wheel Python version cp38 Upload date Hashes View hashes
Filename, size google_crc32c-0.0.1-cp38-cp38-manylinux2010_x86_64.whl (29.4 kB) File type Wheel Python version cp38 Upload date Hashes View hashes
Filename, size google-crc32c-0.0.1.tar.gz (5.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page