Faster zlib and gzip compatible compression and decompression by providing python bindings for the ISA-L library.
Reason this release was yanked:
Does not work correctly with concatenated gzips
Project description
python-isal
Faster zlib and gzip compatible compression and decompression by providing Python bindings for the ISA-L library.
This package provides Python bindings for the ISA-L library. The Intel(R) Intelligent Storage Acceleration Library (ISA-L) implements several key algorithms in assembly language. This includes a variety of functions to provide zlib/gzip-compatible compression.
python-isal provides the bindings by offering an isal_zlib and igzip module which are usable as drop-in replacements for the zlib and gzip modules from the stdlib (with some minor exceptions, see below).
Usage
Python-isal has faster versions of the stdlib’s zlib and gzip module these are called isal_zlib and igzip respectively.
They can be imported as follows
from isal import isal_zlib
from isal import igzip
isal_zlib and igzip are meant to be used as drop in replacements so their api and functions are the same as the stdlib’s modules. Except where ISA-L does not support the same calls as zlib (See differences below).
A full API documentation can be found on our readthedocs page.
python -m isal.igzip implements a simple gzip-like command line application (just like python -m gzip).
Installation
Installation with pip
pip install isal
Installation is supported on Linux, MacOS and Windows. On x86-64 (amd64) platforms wheels are provided, so installation should be almost instantaneous. The installation will include a staticallly linked version of ISA-L. If a wheel is not provided for your system the installation will build ISA-L first in a temporary directory. Please check the ISA-L homepage for the build requirements.
The latest development version of python-isal can be installed with:
pip install git+https://github.com/rhpvorderman/python-isal.git
This requires having the build requirements installed. If you wish to link dynamically against a version of libisal installed on your system use:
PYTHON_ISAL_LINK_DYNAMIC=true pip install isal --no-binary isal
ISA-L is available in numerous Linux distro’s as well as on conda via the conda-forge channel. Checkout the ports documentation on the ISA-L project wiki to find out how to install it. It is important that the development headers are also installed.
On Debian and Ubuntu the ISA-L libraries (including the development headers) can be installed with:
sudo apt install libisal-dev
Installation via conda
Python-isal can be installed via conda, for example using the miniconda installer with a properly setup conda-forge channel. When used with bioinformatics tools setting up bioconda provides a clear set of installation instructions for conda.
python-isal is available on conda-forge and can be installed with:
conda install python-isal
This will automatically install the ISA-L library dependency as well, since it is available on conda-forge.
Differences with zlib and gzip modules
Compression level 0 in zlib and gzip means no compression, while in isal_zlib and igzip this is the lowest compression level. This is a design choice that was inherited from the ISA-L library.
Compression levels range from 0 to 3, not 1 to 9. isal_zlib.Z_DEFAULT_COMPRESSION has been aliased to isal_zlib.ISAL_DEFAULT_COMPRESSION (2).
isal_zlib only supports NO_FLUSH, SYNC_FLUSH, FULL_FLUSH and FINISH_FLUSH. Other flush modes are not supported and will raise errors.
zlib.Z_DEFAULT_STRATEGY, zlib.Z_RLE etc. are exposed as isal_zlib.Z_DEFAULT_STRATEGY, isal_zlib.Z_RLE etc. for compatibility reasons. However, isal_zlib only supports a default strategy and will give warnings when other strategies are used.
zlib supports different memory levels from 1 to 9 (with 8 default). isal_zlib supports memory levels smallest, small, medium, large and largest. These have been mapped to levels 1, 2-3, 4-6, 7-8 and 9. So isal_zlib can be used with zlib compatible memory levels.
isal_zlib methods have a data argument which is positional only. In isal_zlib this is not enforced and it can also called as keyword argument. This is due to implementing isal_zlib in cython and maintaining backwards compatibility with python 3.6.
igzip.open returns a class IGzipFile instead of GzipFile. Since the compression levels are not compatible, a difference in naming was chosen to reflect this. igzip.GzipFile does exist as an alias of igzip.IGzipFile for compatibility reasons.
Contributing
Please make a PR or issue if you feel anything can be improved. Bug reports are also very welcome. Please report them on the github issue tracker.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for isal-0.7.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5393e4c05ff4417939cc5807b51a3863f0382f9d3b9377fe330a985be536e167 |
|
MD5 | bbe0c250f41f0e2e2d46ad4aa82f1035 |
|
BLAKE2b-256 | 375026a4f68251d48382d513ed1e1f7319c61e3c90d2779e312c39fd37a40c10 |
Hashes for isal-0.7.0-cp39-cp39-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9647cde0d8cb95020fd8746d1867f588ead876490fcc51961ea83a91807ab862 |
|
MD5 | 523a933bca2f97125f28ef10b6fce9d6 |
|
BLAKE2b-256 | bdb8f38b076df87f79ea097fd26d560a3046fc1ea06d905264a025e8130bc731 |
Hashes for isal-0.7.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 954f5767ab3019e0edc973dd44983ce2f79936490df1490edd0f502dc974f801 |
|
MD5 | 995a5266a8cfd85cced7bac41c2788ef |
|
BLAKE2b-256 | 559fe2e985532c23aef466dab94c1b42968c2439dddd989b162117a01fd7c63c |
Hashes for isal-0.7.0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4dbfbaaff90f8c2f2ec2dc01b34b332ee6d01a4c972f5b8a3a9c1e5581093c07 |
|
MD5 | d0fabbe1aea6e45495b41c5dd983417d |
|
BLAKE2b-256 | a84d454f520ab86f56475b3169749ff1406052dc91aa843897b710d5bfb097ae |
Hashes for isal-0.7.0-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | afe472925f5f0b1a011baf84a452a33155ae1de3b17884c5280cf80a550595df |
|
MD5 | 8af7886613c598fa086dde953e72cedd |
|
BLAKE2b-256 | 8814bd5b6153431141529ae9c630af0916f88c175b71b739ef3ddd5d81c0224a |
Hashes for isal-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 01c8733a00e157ba956372aaefd02045de055567b017706ae1a0db81453edfdb |
|
MD5 | 7ff1166fc1dd33ea59ec7e6aa2a1c0d6 |
|
BLAKE2b-256 | ffb0e826a6003114b8d18914f1885785f14465715eac7ef435fb6788eb8dfeb2 |
Hashes for isal-0.7.0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 80cca0afbca0b916e16a59ebb410e03807817b41939e259b8917f68ad247c64e |
|
MD5 | 8a266685256eb291398962059fa6f8fa |
|
BLAKE2b-256 | bfbddf9ab349db769866b3fda33b1ef7900208f30a87c158259a6e2fd3fdd1b4 |
Hashes for isal-0.7.0-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 06cc8560192b4046c207051f559933024805a319abb9594fe3d214a6d1a90341 |
|
MD5 | b41958d9721538545b64b8e40e14225f |
|
BLAKE2b-256 | 70d7039ab547e9c9bf4fb5496fad2e5d5aca5eb9f33ab78ede7e4ce044451b9a |
Hashes for isal-0.7.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 363b5779f789feb6e1af34fad47ba5f63437c80549e2da2b4a6928f49a3b0d2f |
|
MD5 | 84f81d451a78364acabf1909f3cf6d6f |
|
BLAKE2b-256 | b62b1cf43beaf7779ab0e1dc03f96d2b560806ecad070b79be9a589b212bd360 |
Hashes for isal-0.7.0-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 36f4a41e6de6c4d3e72589ddd2c0e01c3cf16f234c799db6311d4b84fe22ec6c |
|
MD5 | 8f9e6e9d7f4faf927b328b083c5749b5 |
|
BLAKE2b-256 | 42df9239f12aa39758448e99dbf8e2e391b1e7440b18f043c3a12b46be7b5e6b |
Hashes for isal-0.7.0-cp36-cp36m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 469797d658df8419fe8ee471a4124e46eaa3f96dec9cbd1454fa63cac869ecf8 |
|
MD5 | 2913f7c7f261eee0f93f27c7bcdf1b3c |
|
BLAKE2b-256 | 288007d1443a2995a527fe484528bf1d2da4e69af2271853959d9da7012f2e38 |
Hashes for isal-0.7.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ea916c6e5d73e1cf7d037775db65ae4d0cecd6ee86e408cf98ff5c8d794e364 |
|
MD5 | 491783ac97118dd6f10249c9f8bc00a6 |
|
BLAKE2b-256 | 61335f31582c273486f2ad767e6e5ecbd6a6fba9d903b6a28234687638d29b65 |