Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

LZ4Frame library for Python (via C bindings)

Project Description

Overview

This is an LZ4-frame compression library for Python v3.2+ (and 2.7+), bound to Yann Collet’s LZ4 C implementation.

Installing / packaging

# To get from PyPI
pip3 install py-lz4framed

# To only build extension modules inline (e.g. in repository)
python3 setup.py build_ext -i

# To build & install globally
python3 setup.py install

Notes

Usage

Single-function operation:

import lz4framed

compressed = lz4framed.compress(b'binary data')

uncompressed = lz4framed.decompress(compressed)

To iteratively compress (to a file or e.g. BytesIO instance):

with open('myFile', 'wb') as f:
    # Context automatically finalises frame on completion, unless an exception occurs
    with Compressor(f) as c:
        try:
            while (...):
               c.update(moreData)
        except Lz4FramedNoDataError:
            pass

To decompress from a file-like object:

with open('myFile', 'rb') as f:
    try:
        for chunk in Decompressor(f):
           decoded.append(chunk)
    except Lz4FramedNoDataError:
        # Compress frame data incomplete - error case
        ...

See also lz4framed/__main__.py for example usage.

Documentation

import lz4framed
print(lz4framed.__version__, lz4framed.LZ4_VERSION, lz4framed.LZ4F_VERSION)
help(lz4framed)

Command-line utility

python3 -mlz4framed
USAGE: lz4framed (compress|decompress) (INFILE|-) [OUTFILE]

(De)compresses an lz4 frame. Input is read from INFILE unless set to '-', in
which case stdin is used. If OUTFILE is not specified, output goes to stdout.

Tests

Static

This library has been checked using flake8 and pylint, using a modified configuration - see pylint.rc and flake8.cfg.

Unit

python3 -m unittest discover -v .

Why?

The only existing lz4-frame interoperable implementation I was aware of at the time of writing (lz4tools) had the following limitations:

  • Incomplete implementation in terms of e.g. reference & memory leaks on failure
  • Lack of unit tests
  • Not thread safe
  • Does not release GIL during low level (de)compression operations
  • Did not address the requirements for an external project
Release History

Release History

This version
History Node

0.10.0

History Node

0.9.7

History Node

0.9.6

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
py-lz4framed-0.10.0.tar.gz (88.6 kB) Copy SHA256 Checksum SHA256 Source Sep 5, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting