Faster zlib and gzip compatible compression and decompression by providing python bindings for the ISA-L library.
Project description
python-isal
Faster zlib and gzip compatible compression and decompression by providing Python bindings for the ISA-L library.
This package provides Python bindings for the ISA-L library. The Intel(R) Intelligent Storage Acceleration Library (ISA-L) implements several key algorithms in assembly language. This includes a variety of functions to provide zlib/gzip-compatible compression.
python-isal provides the bindings by offering an isal_zlib and igzip module which are usable as drop-in replacements for the zlib and gzip modules from the stdlib (with some minor exceptions, see below).
Usage
Python-isal has faster versions of the stdlib’s zlib and gzip module these are called isal_zlib and igzip respectively.
They can be imported as follows
from isal import isal_zlib
from isal import igzip
isal_zlib and igzip are meant to be used as drop in replacements so their api and functions are the same as the stdlib’s modules. Except where ISA-L does not support the same calls as zlib (See differences below).
A full API documentation can be found on our readthedocs page.
python -m isal.igzip implements a simple gzip-like command line application (just like python -m gzip).
Installation
Installation with pip
pip install isal
Installation is supported on Linux, MacOS and Windows. On x86-64 (amd64) platforms wheels are provided, so installation should be almost instantaneous. The installation will include a staticallly linked version of ISA-L. If a wheel is not provided for your system the installation will build ISA-L first in a temporary directory. Please check the ISA-L homepage for the build requirements.
The latest development version of python-isal can be installed with:
pip install git+https://github.com/rhpvorderman/python-isal.git
This requires having the build requirements installed. If you wish to link dynamically against a version of libisal installed on your system use:
PYTHON_ISAL_LINK_DYNAMIC=true pip install isal --no-binary isal
ISA-L is available in numerous Linux distro’s as well as on conda via the conda-forge channel. Checkout the ports documentation on the ISA-L project wiki to find out how to install it. It is important that the development headers are also installed.
On Debian and Ubuntu the ISA-L libraries (including the development headers) can be installed with:
sudo apt install libisal-dev
Installation via conda
Python-isal can be installed via conda, for example using the miniconda installer with a properly setup conda-forge channel. When used with bioinformatics tools setting up bioconda provides a clear set of installation instructions for conda.
python-isal is available on conda-forge and can be installed with:
conda install python-isal
This will automatically install the ISA-L library dependency as well, since it is available on conda-forge.
Differences with zlib and gzip modules
Compression level 0 in zlib and gzip means no compression, while in isal_zlib and igzip this is the lowest compression level. This is a design choice that was inherited from the ISA-L library.
Compression levels range from 0 to 3, not 1 to 9. isal_zlib.Z_DEFAULT_COMPRESSION has been aliased to isal_zlib.ISAL_DEFAULT_COMPRESSION (2).
isal_zlib only supports NO_FLUSH, SYNC_FLUSH, FULL_FLUSH and FINISH_FLUSH. Other flush modes are not supported and will raise errors.
zlib.Z_DEFAULT_STRATEGY, zlib.Z_RLE etc. are exposed as isal_zlib.Z_DEFAULT_STRATEGY, isal_zlib.Z_RLE etc. for compatibility reasons. However, isal_zlib only supports a default strategy and will give warnings when other strategies are used.
zlib supports different memory levels from 1 to 9 (with 8 default). isal_zlib supports memory levels smallest, small, medium, large and largest. These have been mapped to levels 1, 2-3, 4-6, 7-8 and 9. So isal_zlib can be used with zlib compatible memory levels.
isal_zlib methods have a data argument which is positional only. In isal_zlib this is not enforced and it can also called as keyword argument. This is due to implementing isal_zlib in cython and maintaining backwards compatibility with python 3.6.
igzip.open returns a class IGzipFile instead of GzipFile. Since the compression levels are not compatible, a difference in naming was chosen to reflect this. igzip.GzipFile does exist as an alias of igzip.IGzipFile for compatibility reasons.
Contributing
Please make a PR or issue if you feel anything can be improved. Bug reports are also very welcome. Please report them on the github issue tracker.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for isal-0.8.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5661a315ea72e159e5636dceffe135f8a56869c8c72b5e8d3e7abdf767e841e5 |
|
MD5 | 42c3f4fc90855f8febb24c20337ad5a6 |
|
BLAKE2b-256 | 7a857abd8d048a4f4d1b233b0e30c07d5d4732d17efb35d5d945782a1863a96a |
Hashes for isal-0.8.1-cp39-cp39-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 443c1a76f5fe1f6cf49ee0c3656fd03b2d64f66800164a6bb1e0b2cc204b1ac6 |
|
MD5 | 4c924bebab0c54ea66aa4fb879498b05 |
|
BLAKE2b-256 | bc022b6313e10f9e4a59a4154deb7251a188af7b56985e99f07a3999751b2450 |
Hashes for isal-0.8.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f92b5e861a35a14f651e7031948646ad17678949b57696d749b7b9048caff61d |
|
MD5 | 662d88b4f8be2af1a8820dceb1eb1fd3 |
|
BLAKE2b-256 | c3e88f8e03d1c20e604af46f74e6222a9509ba42e48e48dbd4d5bc8c09d2ca11 |
Hashes for isal-0.8.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 33a250b3f040110a22e63652d2ed1804dcd4a3972325bec064662df68a07c5d0 |
|
MD5 | ed22f22920de0c39453515705801f73c |
|
BLAKE2b-256 | 2902a370c05a05e731a512c74e0fca83a31149aa4e956114071b2e5ec7d5c47a |
Hashes for isal-0.8.1-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7131e434dddaee010c802b2482bc0bccb78f1fd67666cbbaaf33bb5a3e928b74 |
|
MD5 | 0b705677408731bc09ce39b12361202c |
|
BLAKE2b-256 | 7e7e5245e72b6dd3bc96d25024fc978fc2c88c57eb47ae6cb000a076a45c324e |
Hashes for isal-0.8.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4190a94ab6297c74b77fb65bd9bede057edd676f3972697fb7a16528eddd3a62 |
|
MD5 | d88a65e53fe944f184d2489c25662edd |
|
BLAKE2b-256 | 641f495c8a8687e15750a384649edf3ef6b46613df98984b4c6bf698693cb9f0 |
Hashes for isal-0.8.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64a3fe5ac6349b8afb4d1a86813627642b8c9d031af58273e4fab7b66e85535d |
|
MD5 | 2bb44df0123dca43998ea9f7c7227599 |
|
BLAKE2b-256 | 19b8bd9a22a6497f3eb664646a3ceff5b153ba98e9d9547a2ef7f521e862f6fd |
Hashes for isal-0.8.1-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88d3eeaa74db3a269662acbcb17edf56fcc53147fc0f7394880085ff044e676e |
|
MD5 | 81e09836a22564316dbc5192546789dd |
|
BLAKE2b-256 | cd9484d3dbff956daef3ae3abb28867bb019f607d964b5b6e7ab8841781b1b36 |
Hashes for isal-0.8.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f649ef250348b18a1a1503b3393fda3411f9328d1768fb1b4b972c7f96b39471 |
|
MD5 | d7667f425373926252b9171c15008ea5 |
|
BLAKE2b-256 | 9d23fe61192322c413bca318486a35f9d0ec0d156f2882dc3c85ceb60f0c561a |
Hashes for isal-0.8.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b64004b36b5d1a863dd274b040591128fa560ad384fa9700891cdea8e848e955 |
|
MD5 | d9a8cfeb83532ba8a7809c17eca6d631 |
|
BLAKE2b-256 | 190ff6f5758685b6a0af7b205abd94cc0444716d197c9e5d59c0514366ca05de |
Hashes for isal-0.8.1-cp36-cp36m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6cf9814f47cdd78d66dd138108cf3ff255eb167cbb4d36e1a0da4e9250a3f1f |
|
MD5 | 45a209769184d9ebee3e92e62e10cfb3 |
|
BLAKE2b-256 | e245a999e467226a0f7f6a0ec134f71d32578347bdcc607bd17b7c216842a60b |
Hashes for isal-0.8.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfd19f0fd4c560582519a1a34745631bcfa0e1283e9f2fe635fbfcae82963440 |
|
MD5 | abc016619d9bd288977e9a3862bd583d |
|
BLAKE2b-256 | 5f5ef7725b07198802021c00373f1a8099d48b56133c9e5a327c373ced3487d2 |