LZ4Frame library for Python (via C bindings)
Installing / packaging
# To get from PyPI pip3 install py-lz4framed # To only build extension modules inline (e.g. in repository) python3 setup.py build_ext -i # To build & install globally python3 setup.py install
import lz4framed compressed = lz4framed.compress(b'binary data') uncompressed = lz4framed.decompress(compressed)
To iteratively compress (to a file or e.g. BytesIO instance):
with open('myFile', 'wb') as f: # Context automatically finalises frame on completion, unless an exception occurs with Compressor(f) as c: try: while (...): c.update(moreData) except Lz4FramedNoDataError: pass
To decompress from a file-like object:
with open('myFile', 'rb') as f: try: for chunk in Decompressor(f): decoded.append(chunk) except Lz4FramedNoDataError: # Compress frame data incomplete - error case ...
See also lz4framed/__main__.py for example usage.
import lz4framed print(lz4framed.__version__, lz4framed.LZ4_VERSION, lz4framed.LZ4F_VERSION) help(lz4framed)
python3 -mlz4framed USAGE: lz4framed (compress|decompress) (INFILE|-) [OUTFILE] (De)compresses an lz4 frame. Input is read from INFILE unless set to '-', in which case stdin is used. If OUTFILE is not specified, output goes to stdout.
python3 -m unittest discover -v .
The only existing lz4-frame interoperable implementation I was aware of at the time of writing (lz4tools) had the following limitations:
- Incomplete implementation in terms of e.g. reference & memory leaks on failure
- Lack of unit tests
- Not thread safe
- Does not release GIL during low level (de)compression operations
- Did not address the requirements for an external project