Skip to main content

ZSTD Bindings for Python

Project description

https://travis-ci.org/sergey-dryabzhinsky/python-zstd.svg?branch=master

Simple python bindings to Yann Collet ZSTD compression library

Zstd, short for Zstandard, is a new lossless compression algorithm,
which provides both good compression ratio and speed for your standard compression needs. “Standard” translates into everyday situations which neither look for highest possible ratio (which LZMA and ZPAQ cover) nor extreme speeds (which LZ4 covers).

It is provided as a BSD-license package, hosted on GitHub.

WARNING!!!

If you setup 1.0.0.99.1 version - remove it manualy to able to update. PIP matching version strings not tuple of numbers.

Result generated by versions prior to 1.0.0.99.1 is not compatible with orignial Zstd by any means. It generates custom header and can be read only by zstd python module.

As of 1.0.0.99.1 version it uses standard Zstd output, not modified.

To prevent data loss there is two functions now: `compress_old` and `decompress_old`. They are works just like in old versions prior to 1.0.0.99.1.

As of 1.1.4 version module build without them by default.

As of 1.3.4 version these functions are deprecated and will be removed in future releases.

As of 1.5.0 version these functions are removed.

DISCLAIMER

These python bindings are kept simple and blunt.

Support of dictionaries and streaming is not planned.

Build from source

>>> $ git clone https://github.com/sergey-dryabzhinsky/python-zstd
>>> $ git submodule update --init
>>> $ apt-get install python-dev python3-dev python-setuptools python3-setuptools
>>> $ python setup.py build_ext clean
>>> $ python3 setup.py build_ext clean

Note: legacy format support disabled by default. To build with Zstd legacy versions support - pass --legacy option to setup.py script:

>>> $ python setup.py build_ext --legacy clean

Note: PyZstd legacy format support disabled by default. To build with python-zstd legacy format support (pre 1.1.2) - pass --pyzstd-legacy option to setup.py script:

>>> $ python setup.py build_ext --pyzstd-legacy clean

If you want to build with existing distribution of libzstd just add --external option. But beware! Legacy formats support is unknown in this case. And if your version not equal with python-zstd - tests may not pass.

>>> $ python setup.py build_ext --external clean

If paths to header file zstd.h and libraries is uncommon - use common build params: –libraries –include-dirs –library-dirs.

>>> $ python setup.py build_ext --external --include-dirs /opt/zstd/usr/include --libraries zstd --library-dirs /opt/zstd/lib clean

Install from pypi

>>> # for Python 2.7+
>>> $ pip install zstd
>>> # or for Python 3.4+
>>> $ pip3 install zstd

API

Error
Standard python Exception for zstd module
ZSTD_compress (data[, level, threads]): string|bytes

Function, compress input data block via mutliple threads, return compressed block, or raises Error.

Params:

  • data: string|bytes - input data block, length limited by 2Gb by Python API
  • level: int - compression level, ultra-fast levels from -100 (ultra) to -1 (fast) available since zstd-1.3.4, and from 1 (fast) to 22 (slowest), 0 or unset - means default (3). Default - 3.
  • threads: int - how many threads to use, from 0 to 200, 0 or unset - auto-tune by cpu cores count. Default - 0. Since: 1.4.4.1

Aliases: compress(…), dumps(…)

Since: 0.1

ZSTD_uncompress (data): string|bytes

Function, decompress input compressed data block, return decompressed block, or raises Error.

Params:

  • data: string|bytes - input compressed data block, length limited by 2Gb by Python API

Aliases: decompress(…), uncompress(…), loads(…)

Since: 0.1

version (): string|bytes

Returns this module doted version string.

Since: 1.3.4.3

ZSTD_version (): string|bytes

Returns ZSTD library doted version string.

Since: 1.3.4.3

ZSTD_version_number (): int

Returns ZSTD library version in format: MAJOR*100*100 + MINOR*100 + RELEASE.

Since: 1.3.4.3

ZSTD_external (): int

Returns 0 of 1 if ZSTD library build as external.

Since: 1.5.0.2

Removed

ZSTD_compress_old (data[, level]): string|bytes

Function, compress input data block, return compressed block, or raises Error.

DEPRECATED: Returns not compatible with ZSTD block header

REMOVED: since 1.5.0

Params:

  • data: string|bytes - input data block, length limited by 2Gb by Python API
  • level: int - compression level, ultra-fast levels from -5 (ultra) to -1 (fast) available since zstd-1.3.4, and from 1 (fast) to 22 (slowest), 0 or unset - means default (3). Default - 3.

Since: 1.0.0.99.1

ZSTD_uncompress_old (data): string|bytes

Function, decompress input compressed data block, return decompressed block, or raises Error.

DEPRECATED: Accepts data with not compatible with ZSTD block header

REMOVED: since 1.5.0

Params:

  • data: string|bytes - input compressed data block, length limited by 2Gb by Python API

Since: 1.0.0.99.1

Use

Module has simple API:

>>> import zstd
>>> dir(zstd)
['Error', 'ZSTD_compress', 'ZSTD_uncompress', 'ZSTD_version', 'ZSTD_version_number', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'compress', 'decompress', 'dumps', 'loads', 'uncompress', 'version']
>>> zstd.version()
'1.5.0.2'
>>> zstd.ZSTD_version()
'1.5.0'
>>> zstd.ZSTD_version_number()
10500
>>> zstd.ZSTD_external()
0

In python2

>>> data = "123456qwert"

In python3 use bytes

>>> data = b"123456qwert"
>>> cdata = zstd.compress(data, 1)
>>> data == zstd.decompress(cdata)
True
>>> cdata_mt = zstd.compress(data, 1, 4)
>>> cdata == cdata_mt
True
>>> data == zstd.decompress(cdata_mt)
True

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for zstd, version 1.5.0.2
Filename, size File type Python version Upload date Hashes
Filename, size zstd-1.5.0.2.tar.gz (528.9 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page