Faster zlib and gzip compatible compression and decompression by providing python bindings for the ISA-L library.
Project description
python-isal
Faster zlib and gzip compatible compression and decompression by providing Python bindings for the ISA-L library.
This package provides Python bindings for the ISA-L library. The Intel(R) Intelligent Storage Acceleration Library (ISA-L) implements several key algorithms in assembly language. This includes a variety of functions to provide zlib/gzip-compatible compression.
python-isal provides the bindings by offering an isal_zlib and igzip module which are usable as drop-in replacements for the zlib and gzip modules from the stdlib (with some minor exceptions, see below).
Usage
Python-isal has faster versions of the stdlib’s zlib and gzip module these are called isal_zlib and igzip respectively.
They can be imported as follows
from isal import isal_zlib
from isal import igzip
isal_zlib and igzip are meant to be used as drop in replacements so their api and functions are the same as the stdlib’s modules. Except where ISA-L does not support the same calls as zlib (See differences below).
A full API documentation can be found on our readthedocs page.
python -m isal.igzip implements a simple gzip-like command line application (just like python -m gzip).
Installation
Installation with pip
pip install isal
Installation is supported on Linux, MacOS and Windows. On x86-64 (amd64) platforms wheels are provided, so installation should be almost instantaneous. The installation will include a staticallly linked version of ISA-L. If a wheel is not provided for your system the installation will build ISA-L first in a temporary directory. Please check the ISA-L homepage for the build requirements.
The latest development version of python-isal can be installed with:
pip install git+https://github.com/rhpvorderman/python-isal.git
This requires having the build requirements installed. If you wish to link dynamically against a version of libisal installed on your system use:
PYTHON_ISAL_LINK_DYNAMIC=true pip install isal --no-binary isal
ISA-L is available in numerous Linux distro’s as well as on conda via the conda-forge channel. Checkout the ports documentation on the ISA-L project wiki to find out how to install it. It is important that the development headers are also installed.
On Debian and Ubuntu the ISA-L libraries (including the development headers) can be installed with:
sudo apt install libisal-dev
Installation via conda
Python-isal can be installed via conda, for example using the miniconda installer with a properly setup conda-forge channel. When used with bioinformatics tools setting up bioconda provides a clear set of installation instructions for conda.
python-isal is available on conda-forge and can be installed with:
conda install python-isal
This will automatically install the ISA-L library dependency as well, since it is available on conda-forge.
Differences with zlib and gzip modules
Compression level 0 in zlib and gzip means no compression, while in isal_zlib and igzip this is the lowest compression level. This is a design choice that was inherited from the ISA-L library.
Compression levels range from 0 to 3, not 1 to 9. isal_zlib.Z_DEFAULT_COMPRESSION has been aliased to isal_zlib.ISAL_DEFAULT_COMPRESSION (2).
isal_zlib only supports NO_FLUSH, SYNC_FLUSH, FULL_FLUSH and FINISH_FLUSH. Other flush modes are not supported and will raise errors.
zlib.Z_DEFAULT_STRATEGY, zlib.Z_RLE etc. are exposed as isal_zlib.Z_DEFAULT_STRATEGY, isal_zlib.Z_RLE etc. for compatibility reasons. However, isal_zlib only supports a default strategy and will give warnings when other strategies are used.
zlib supports different memory levels from 1 to 9 (with 8 default). isal_zlib supports memory levels smallest, small, medium, large and largest. These have been mapped to levels 1, 2-3, 4-6, 7-8 and 9. So isal_zlib can be used with zlib compatible memory levels.
isal_zlib methods have a data argument which is positional only. In isal_zlib this is not enforced and it can also called as keyword argument. This is due to implementing isal_zlib in cython and maintaining backwards compatibility with python 3.6.
igzip.open returns a class IGzipFile instead of GzipFile. Since the compression levels are not compatible, a difference in naming was chosen to reflect this. igzip.GzipFile does exist as an alias of igzip.IGzipFile for compatibility reasons.
Contributing
Please make a PR or issue if you feel anything can be improved. Bug reports are also very welcome. Please report them on the github issue tracker.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for isal-0.9.0-pp37-pypy37_pp73-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d181085d8cbd1cedc4a15501f3fdcebe0b2c7f6317a174b159f60f9a12e3ad9 |
|
MD5 | 3f2e8accd3ef9eb0c768c86204ebec79 |
|
BLAKE2b-256 | e2e57eae72bcf542ed67eb6de82fb21b0c809b52f9bead5d0ad53a35932d66ac |
Hashes for isal-0.9.0-pp37-pypy37_pp73-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1decbe9ea88fc096946cbd135ba15d322320ea1bd816ec461f1214f429ff2c09 |
|
MD5 | 61bf6b38801dc91a8112321dc6077889 |
|
BLAKE2b-256 | e1cde492cec14534bb9c05941391d34d51d27072972dfb4c886032528f959833 |
Hashes for isal-0.9.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e911a340671b24fc66e4365be1ed7dda246db384bd6272004ca77093ac348e01 |
|
MD5 | 9aa50a3dc532204322b82c10c034210c |
|
BLAKE2b-256 | 071661a7f08349436b72aa411d8a9f7c45df9dd38d00517b2e04914bc33e43aa |
Hashes for isal-0.9.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c99c396822d058fd90f8065d6a9f5b9e559d394107d88a878657ca5f01df5c06 |
|
MD5 | f668fa706311b66abdab20d9460767fa |
|
BLAKE2b-256 | 8ed7c10362b8f63feb1edd0ae2dee44c3091387a47c9c8a1e168b8e8d34c9af7 |
Hashes for isal-0.9.0-pp36-pypy36_pp73-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98e9bb3031987b23eb94c69f2eab28752a8923a72c9d19cef542aa22cc8d7b74 |
|
MD5 | 81063636dd2aa06d0071f7a752462feb |
|
BLAKE2b-256 | e9e8d82c0c9380f9ee09c14a63821aaad83142b2bf44ec303b0dd74ec91acc2e |
Hashes for isal-0.9.0-pp36-pypy36_pp73-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 481edf99f0858ac808ca9727c80ef838d97b5f03fe000042973e03b5fdd56bf0 |
|
MD5 | 5a51d007781c605dabcc93e3a8f0045e |
|
BLAKE2b-256 | 4ecd1622c55fa3248eb4dcc967cd6781720a635b133b3d7244e48ca6397e4049 |
Hashes for isal-0.9.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4edcb7d8c06c62786e3b387aac3ece9f7c4df61ea236e0c60c92baab1256961 |
|
MD5 | acc85321e72ddf89502623818c93771c |
|
BLAKE2b-256 | c51ca717bbb022f8f9dc21984291977afae104ae8ac6fb105d17e4dc457d1f4f |
Hashes for isal-0.9.0-cp39-cp39-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 523edaf2b591e54e357eaacc470a2c22a6c5ced88fb9a01987befcef8347cdb9 |
|
MD5 | 21053f5b34a01cc073278806fb95f557 |
|
BLAKE2b-256 | e0b3e607ada74d6ac2bdeb97496dc352f45ca6d20a426a14f9f0a2b849b9fc05 |
Hashes for isal-0.9.0-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ddb0ca396a166855dc4021eddb3279f0cfff40abafda726073ebff5b9f7e13e |
|
MD5 | c3a229188c86c60ae41a0e3e258da60b |
|
BLAKE2b-256 | 191a7076ab53ee45ae1161b02ed4b11a3d956248e96b80026e3059c03ff0e04b |
Hashes for isal-0.9.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51e347d6675619fc61a67cebe971b2f2b9fe8567220afa2f6b21d68986d62419 |
|
MD5 | bb1f998ebdb999014d4c633eddfa941d |
|
BLAKE2b-256 | bb9a422d63004e9f14dead85312a3614d68eac0598367a908365d9e800550431 |
Hashes for isal-0.9.0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee95cbc854293b7019467f38d82b63dec228d0e8e94cf9c9a70aa1d3466a5b20 |
|
MD5 | 221bea80e4d6cb7fb9cd3ffef3422d77 |
|
BLAKE2b-256 | 40af5c6f58d784f7bc7d3ac89657b7d130c261d5e5d089cc5a17766f33230b93 |
Hashes for isal-0.9.0-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | da9163c8392a47f361d1126f37386fd8c240ba059e179520666dcb3f1441cc12 |
|
MD5 | 6bcae907a397476212597875d5415917 |
|
BLAKE2b-256 | 31ae18466dacb2c76ed1fa0a44e5ff17914e70f6f339de90d0f6a530cb671547 |
Hashes for isal-0.9.0-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89a8c29c653e6fcf1cf216362ea09e313813e443cff3e82495a537b9af365154 |
|
MD5 | 958c6cc7412b00a2b6993fdad1a20b81 |
|
BLAKE2b-256 | e8bc4f7ba66bdbd070faa855c7a5eff6e05b85c57f22e1095ad4cd7d2e0bd3e2 |
Hashes for isal-0.9.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93b0e7af041a0e9cb3c9395ac8c0e790aa156e8b353cc9023e9d3c24b8590e28 |
|
MD5 | 477a79568d420580648fbeedab5406d9 |
|
BLAKE2b-256 | 75b462eb8b6382d7533e220a672a25687a732b29c2111929565b795db404a6a4 |
Hashes for isal-0.9.0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 37c8b646056dfa0e5b47e00cf38b47ae50224035a126e6d527098d85dcc72e77 |
|
MD5 | fbddde02b08d41717ff42b5c5eaf543e |
|
BLAKE2b-256 | 8a3003d0abc2fa7630bde489096d3493a36cc4a0013067e31ac728e3017c760b |
Hashes for isal-0.9.0-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bfa4e49cc92d7ed74d57ba2901f1b67cd748d983dd67d63c9b47b9412ea2b098 |
|
MD5 | 6c8e6d43762af8c9dea2c71a633e62b7 |
|
BLAKE2b-256 | ab2ca0c2e725a1593d9726ec49b9c33e5bf2964da99082ca78d9140913afde80 |
Hashes for isal-0.9.0-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8737f45a1320f1b7347c78aec5c4070d3aaf98637abbcb08d64dff07ee8b53b |
|
MD5 | 7770baddeb19eed53c3fe47870801f83 |
|
BLAKE2b-256 | d717fd615fbd786fcc7c22c270667d261de6afeecee73a1418ff8ed23181f488 |
Hashes for isal-0.9.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f9b049489988a6b2be7da38ac59067ccf2f9b4b9c8093d206fb7d90f9edfb80 |
|
MD5 | a1cfc21e26c11d0b8d65aa8e578724b1 |
|
BLAKE2b-256 | bf64a582c9d3d50e048e8c68abce72d749594a74ba37de2da56a700401d3c3ec |
Hashes for isal-0.9.0-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0f5a77c6a1e9e3d19e3d4db9c3c374d8aabea6596bf6eae28de24ec91b68a9a |
|
MD5 | 97e6dc9bfab38bfe6fa494bc888d9c6b |
|
BLAKE2b-256 | 0263a17a1a4b3973efb236a128caad643568ee41032a6a28e2c609223a921246 |
Hashes for isal-0.9.0-cp36-cp36m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cf8d84b486fe8030060a20e58fd0e6800c8fa6056117f67a7e8064568e62c8bc |
|
MD5 | 83802ef9ed272f3c8236ac31889959e0 |
|
BLAKE2b-256 | 17737fc60c4b9fd6a5a587e8bcd96b4300d0460dcc046e8d76ebf544797c0693 |
Hashes for isal-0.9.0-cp36-cp36m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 43766d614688d07f38fc4b137709c23ace527d19251d75b6d694b8913f100ac6 |
|
MD5 | 1212505bbc0b884ae283f2ea21aba200 |
|
BLAKE2b-256 | 3d73cb862c3881d54db2eeb57d38923e3ec2ff9156b19b59f7fb417563b59576 |
Hashes for isal-0.9.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ae1ab9470d4720b8f020622abdf4c8d9044ce89a39d7d5d93a97dc6e1adb4736 |
|
MD5 | b8d55f5bb874320a20f29ea19e7be2dc |
|
BLAKE2b-256 | d7746911190bdbcc074aa671cf09983f277974e59dea95ca759b62397e2f3581 |