Faster zlib and gzip compatible compression and decompression by providing python bindings for the ISA-L library.
Reason this release was yanked:
Does not work correctly with concatenated gzips
Project description
python-isal
Faster zlib and gzip compatible compression and decompression by providing Python bindings for the ISA-L library.
This package provides Python bindings for the ISA-L library. The Intel(R) Intelligent Storage Acceleration Library (ISA-L) implements several key algorithms in assembly language. This includes a variety of functions to provide zlib/gzip-compatible compression.
python-isal provides the bindings by offering an isal_zlib and igzip module which are usable as drop-in replacements for the zlib and gzip modules from the stdlib (with some minor exceptions, see below).
Usage
Python-isal has faster versions of the stdlib’s zlib and gzip module these are called isal_zlib and igzip respectively.
They can be imported as follows
from isal import isal_zlib
from isal import igzip
isal_zlib and igzip are meant to be used as drop in replacements so their api and functions are the same as the stdlib’s modules. Except where ISA-L does not support the same calls as zlib (See differences below).
A full API documentation can be found on our readthedocs page.
python -m isal.igzip implements a simple gzip-like command line application (just like python -m gzip).
Installation
Installation with pip
pip install isal
Installation is supported on Linux, MacOS and Windows. On x86-64 (amd64) platforms wheels are provided, so installation should be almost instantaneous. The installation will include a staticallly linked version of ISA-L. If a wheel is not provided for your system the installation will build ISA-L first in a temporary directory. Please check the ISA-L homepage for the build requirements.
The latest development version of python-isal can be installed with:
pip install git+https://github.com/rhpvorderman/python-isal.git
This requires having the build requirements installed. If you wish to link dynamically against a version of libisal installed on your system use:
PYTHON_ISAL_LINK_DYNAMIC=true pip install isal --no-binary isal
ISA-L is available in numerous Linux distro’s as well as on conda via the conda-forge channel. Checkout the ports documentation on the ISA-L project wiki to find out how to install it. It is important that the development headers are also installed.
On Debian and Ubuntu the ISA-L libraries (including the development headers) can be installed with:
sudo apt install libisal-dev
Installation via conda
Python-isal can be installed via conda, for example using the miniconda installer with a properly setup conda-forge channel. When used with bioinformatics tools setting up bioconda provides a clear set of installation instructions for conda.
python-isal is available on conda-forge and can be installed with:
conda install python-isal
This will automatically install the ISA-L library dependency as well, since it is available on conda-forge.
Differences with zlib and gzip modules
Compression level 0 in zlib and gzip means no compression, while in isal_zlib and igzip this is the lowest compression level. This is a design choice that was inherited from the ISA-L library.
Compression levels range from 0 to 3, not 1 to 9. isal_zlib.Z_DEFAULT_COMPRESSION has been aliased to isal_zlib.ISAL_DEFAULT_COMPRESSION (2).
isal_zlib only supports NO_FLUSH, SYNC_FLUSH, FULL_FLUSH and FINISH_FLUSH. Other flush modes are not supported and will raise errors.
zlib.Z_DEFAULT_STRATEGY, zlib.Z_RLE etc. are exposed as isal_zlib.Z_DEFAULT_STRATEGY, isal_zlib.Z_RLE etc. for compatibility reasons. However, isal_zlib only supports a default strategy and will give warnings when other strategies are used.
zlib supports different memory levels from 1 to 9 (with 8 default). isal_zlib supports memory levels smallest, small, medium, large and largest. These have been mapped to levels 1, 2-3, 4-6, 7-8 and 9. So isal_zlib can be used with zlib compatible memory levels.
isal_zlib methods have a data argument which is positional only. In isal_zlib this is not enforced and it can also called as keyword argument. This is due to implementing isal_zlib in cython and maintaining backwards compatibility with python 3.6.
igzip.open returns a class IGzipFile instead of GzipFile. Since the compression levels are not compatible, a difference in naming was chosen to reflect this. igzip.GzipFile does exist as an alias of igzip.IGzipFile for compatibility reasons.
Contributing
Please make a PR or issue if you feel anything can be improved. Bug reports are also very welcome. Please report them on the github issue tracker.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for isal-0.8.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 46a444ba26aa64d2cc09c998ad9d44bf22841b359c33a19db274f7abda6ebb35 |
|
MD5 | 7d6ecf7fd96a90824c696a4c849d7466 |
|
BLAKE2b-256 | 8551468e88704ed8550620ca846cea9c6f525f685313724433101596c1ebaa48 |
Hashes for isal-0.8.0-cp39-cp39-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 977f6fe2d40b79f7331db9d2dcf2ba3655a79ea6e3ddb809780bad217ea28826 |
|
MD5 | 92dc9c2a2914732af8100791e8edfc7e |
|
BLAKE2b-256 | 6f43da18db5083b233e8b249ee4cc37a553dbbfd4f61c3ab10e1f7344e084241 |
Hashes for isal-0.8.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7fdd3466455d8093989a40576f27853adc7a78110622f031187ddb209859f200 |
|
MD5 | 228284dc0bcdef42adcc09a009060c0a |
|
BLAKE2b-256 | 8f2c70e7ed65d6f733f3680a72b2c3de806255fa1359c79cb2536f6e1dd79224 |
Hashes for isal-0.8.0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79a975f31aebb94b2817c2611a0e6ec3ce50c95019877cc886c2716c54a449e6 |
|
MD5 | 491a8fc60df0124f59e85e9cb1e4011d |
|
BLAKE2b-256 | 1e1257b647d6680940acd5d8e28795ead04f808be9ae952d6d7797db92d489c4 |
Hashes for isal-0.8.0-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 773684bd0bb516c951d527df53282270eeafb71119d3323e861fce5f751e4a3c |
|
MD5 | 900324a1e3caff561c627e40a2fd95b4 |
|
BLAKE2b-256 | 9184d79927ae5597d356778a16046d602f05beca506f6d3c711475719185f13e |
Hashes for isal-0.8.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c81b6957cf2166015dc0805d6dc5a0a614fe82aee17f578ba21571c111d9d395 |
|
MD5 | b729d9c63249ed7ccf6af1c5e3d07951 |
|
BLAKE2b-256 | e0f13782b7a297282dde40bd62362bebbe341df2f08a618b3afb13939ea1b042 |
Hashes for isal-0.8.0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de31cb9df96ad5d1ef514c393e0197d8f97d381097f10f3f1b5456a1429431f7 |
|
MD5 | e548c816311d85cfd6db2561ab785919 |
|
BLAKE2b-256 | 682d3dc87568f5160e19c7cdf5b8635c6230f8795052c6ea61cad00fa88ec517 |
Hashes for isal-0.8.0-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0bca5b5870b4627ef450ff4120f85e7c0958f1594f54f6cb27a3b3d4135ecfe |
|
MD5 | 06bad50a91fb7a2865f97ba1b5b519f4 |
|
BLAKE2b-256 | ce930baf411668ee5acdd47ff483dad779920a7254312a213b3ad72c5c592747 |
Hashes for isal-0.8.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 599f3f891143a38ea0120548f450fe5bab619d653d3db8f11c3c243b217531b3 |
|
MD5 | 32e0bd8368cf35f99d2ba5d7c9ae8afb |
|
BLAKE2b-256 | 5e8f1eeb61a5d7946f5191bce359be271d7090d895a4c5f50256b14f4727a9c3 |
Hashes for isal-0.8.0-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2fcb63bf305339901bb720139468108abf6be3c6c415fc86ee105f8118fa1c0e |
|
MD5 | 7a3763822438fd0d2898828e5d0c4109 |
|
BLAKE2b-256 | d8fd2c35138b6bcd57825f5c124cbab1adc084e458a99e614be62cdae1fad79c |
Hashes for isal-0.8.0-cp36-cp36m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ec2a7ee33f188062f9000b846ba4e72ef454e83c04edccc275471a63fbc5fd6 |
|
MD5 | 8a017605e3da1aa0de2b980310a4dbf6 |
|
BLAKE2b-256 | d0f9d2067b60a552e3fdb0876d7e9869b47894e5a9f914c4a27cc9dedc7e07d8 |
Hashes for isal-0.8.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a355c583ece64b803ab316bf865eba6ce75137d9cae5c8d898ce16a473c0e602 |
|
MD5 | 06184d2c152ca833451dc519c9d7d09e |
|
BLAKE2b-256 | 775db0ffa363e11628f37abcf6acc23ac085a784ab25749b4a871f0191410def |