Faster zlib and gzip compatible compression and decompression by providing python bindings for the isa-l library.
Project description
python-isal
Faster zlib and gzip compatible compression and decompression by providing Python bindings for the ISA-L library.
This package provides Python bindings for the ISA-L library. The Intel Infrastructure Storage Acceleration Library (ISA-L) implements several key algorithms in assembly language. This includes a variety of functions to provide zlib/gzip-compatible compression.
python-isal provides the bindings by offering an isal_zlib and igzip module which are usable as drop-in replacements for the zlib and gzip modules from the stdlib (with some minor exceptions, see below).
Installation
Installation with pip
Linux and MacOS: pip install isal. Wheels are provided, so installation should be almost instantaneous.
Windows: Installation is not supported yet.
The installation will include a staticallly linked version of isa-l. On Linux and MacOS, wheels are provided. If a wheel is not provided for your system the installation will build ISA-L first in a temporary directory. Please check the ISA-L homepage for the build requirements.
The latest development version of python-isal can be installed with:
pip install git+https://github.com/rhpvorderman/python-isal.git
This requires having the build requirements installed. If you wish to link dynamically against a version of libisal installed on your system use:
PYTHON_ISAL_LINK_DYNAMIC=true pip install isal
ISA-L is available in numerous Linux distro’s as well as on conda via the conda-forge channel. Checkout the ports documentation on the ISA-L project wiki to find out how to install it. It is important that the development headers are also installed.
On Debian and Ubuntu the ISA-L libraries (including the development headers) can be installed with:
sudo apt install libisal-dev
Installation via conda
Python-isal can be installed via conda, for example using the miniconda installer with a properly setup conda-forge channel. When used with bioinformatics tools setting up bioconda provides a clear set of installation instructions for conda.
python-isal is available on conda-forge and can be installed with
conda install python-isal
This will automatically install the isa-l library dependency as well, since it is available on conda-forge.
Usage
Python-isal has faster versions of the stdlib’s zlib and gzip module these are called isal_zlib and igzip respectively.
They can be imported as follows
from isal import isal_zlib
from isal import igzip
isal_zlib and igzip were meant to be used as drop in replacements so their api and functions are the same as the stdlib’s modules. Except where isa-l does not support the same calls as zlib (See differences below).
A full API documentation can be found on our readthedocs page.
python -m isal.igzip implements a simple gzip-like command line application (just like python -m gzip).
Differences with zlib and gzip modules
Compression level 0 in zlib and gzip means no compression, while in isal_zlib and igzip this is the lowest compression level. This is a design choice that was inherited from the ISA-L library.
Compression levels range from 0 to 3, not 1 to 9.
isal_zlib.crc32 and isal_zlib.adler32 do not support negative numbers for the value parameter.
zlib.Z_DEFAULT_STRATEGY, zlib.Z_RLE etc. are exposed as isal_zlib.Z_DEFAULT_STRATEGY, isal_zlib.Z_RLE etc. for compatibility reasons. However, isal_zlib only supports a default strategy and will give warnings when other strategies are used.
zlib supports different memory levels from 1 to 9 (with 8 default). isal_zlib supports memory levels smallest, small, medium, large and largest. These have been mapped to levels 1, 2-3, 4-6, 7-8 and 9. So isal_zlib can be used with zlib compatible memory levels.
isal_zlib only supports FLUSH, SYNC_FLUSH and FULL_FLUSH FINISH is aliased to FULL_FLUSH (and works correctly as such).
isal_zlib has a compressobj and decompressobj implementation. However, the unused_data and unconsumed_tail for the Decompress object, only work properly when using gzip compatible compression. (25 <= wbits <= 31).
The flush implementation for the Compress object behavious differently from the zlib equivalent.
Contributing
Please make a PR or issue if you feel anything can be improved. Bug reports are also very welcome. Please report them on the github issue tracker.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for isal-0.4.0-cp39-cp39-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e22af17fd613fb7ccbe394de6ee72fbf99f84cb538b487636e4c85d20ed5a7b6 |
|
MD5 | dc5888bd49e30d339d09b3569511330b |
|
BLAKE2b-256 | 7317af88a6005ca79f47bb34a460ca4e107d4c690699ed17d13deab9e7d3f1c6 |
Hashes for isal-0.4.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35ba17818ddf850bce61257268ab803fbb44ebdd5732cf51590b6e21e225f385 |
|
MD5 | d37185df8cddc3347a1d390a8184681f |
|
BLAKE2b-256 | 9601b27cc307828631b5d48ebaf701fd4199a201753bdd91b087e4aa7edb9f16 |
Hashes for isal-0.4.0-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3eab41f8f0ecf091c5047cb0e77df32ca1e06801ff53ab4396b77b3461ac0295 |
|
MD5 | e92ced7004f2ce5e04ba7a3550410875 |
|
BLAKE2b-256 | 319375693c3958946bdd9e9f560576beb0192d302b78ae5f4a22dcb9d3e1cfa4 |
Hashes for isal-0.4.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e68b0c45c51e48892abfbcfb400142b00f296b569a3353d6ceaebf2307d99b4b |
|
MD5 | 73f9577f541a6f57972508e543d40bad |
|
BLAKE2b-256 | 1e8306f54398a21c093bb02be05465fcdc6172cac6ccb1fc2b4ac4ca01736c20 |
Hashes for isal-0.4.0-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | db68b7b04526ad42a65190f3e8355d5c1ff01394c35c036532a1903142feb596 |
|
MD5 | 95d9e2dfb9f249b0afc0fd1a61683b50 |
|
BLAKE2b-256 | c781bc99805864eb456042e376279d59b9e798b457bf14f4fbfd2ebda26b5f4e |
Hashes for isal-0.4.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8150f68f407f211d10efb35939c6026a778657a29b2644a3d0e45645bcfadbc |
|
MD5 | 9ccb9283497cd3fed6eb282287592314 |
|
BLAKE2b-256 | 32ac6c1210c4cbdf4ef8b3ffd3a08a39ba56ebfc7fb04d76cc0a3b0dd575fac1 |
Hashes for isal-0.4.0-cp36-cp36m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f15ce28c4045a76dcadb9f75c23456da369b40953d958afeda3e0c97831bcf66 |
|
MD5 | b2ed8b779dc1db87b347853a92467695 |
|
BLAKE2b-256 | 7a87e68d1a54eca2a31b66f7e34e5b1dd3175d7443a54b9962260eee9942141b |
Hashes for isal-0.4.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f35b53789940ebe6860e6c64a9d271f0969c9a763ab340de66b71c130571d3a9 |
|
MD5 | e3715f106236d548144fbb3a4ae966d0 |
|
BLAKE2b-256 | 0b77f883234f313e3ae287d8baf4db0c7c75176d4414332e9c054373c1bf87db |