pyzstd 0.15.10

Python bindings to Zstandard (zstd) compression library.

Introduction

Pyzstd module provides classes and functions for compressing and decompressing data, using Facebook’s Zstandard (or zstd as short name) algorithm.

The API style is similar to Python’s bz2/lzma/zlib modules.

• Includes zstd v1.5.5 source code

• Can also dynamically link to zstd library provided by system, see this note.

• Has a CFFI implementation that can work with PyPy

• Support sub-interpreter on CPython 3.12+

• ZstdFile class has C language level performance

• Supports Zstandard Seekable Format

• Has a command line interface: python -m pyzstd --help

Links

Documentation: https://pyzstd.readthedocs.io/

GitHub: https://github.com/Rogdham/pyzstd

Release note

0.15.10 (Mar 24, 2024)

1. Fix SeekableZstdFile class can’t open new file in appending mode.

2. Support sub-interpreter on CPython 3.12+, can utilize per-interpreter GIL.

3. On CPython(3.5~3.12)+Linux, use another output buffer code that can utilize the mremap mechanism.

4. Change repository URL and maintainer following the deletion of the GitHub account of the original author, Ma Lin (animalize) <https://github.com/Rogdham/pyzstd/issues/1>.

0.15.9 (Jun 24, 2023)

ZstdFile class related changes:

1. Add SeekableZstdFile class, it’s a subclass of ZstdFile, supports Zstandard Seekable Format.

2. Add mode argument to ZstdFile.flush() method, now it can flush a zstd frame.

3. Add read_size and write_size arguments to ZstdFile.__init__() method, can work with Network File Systems better.

4. Optimize ZstdFile performance to C language level.

0.15.7 (Apr 21, 2023)

ZstdDict class changes:

1. Fix these advanced compression parameters may be ignored when loading a dictionary: windowLog, hashLog, chainLog, searchLog, minMatch, targetLength, strategy, enableLongDistanceMatching, ldmHashLog, ldmMinMatch, ldmBucketSizeLog, ldmHashRateLog, and some non-public parameters.

2. When compressing, load undigested dictionary instead of digested dictionary by default. Loading again an undigested is slower, see differences.

3. Add .as_prefix attribute. Can use zstd as a patching engine.

0.15.6 (Apr 5, 2023)

Upgrade zstd source code from v1.5.4 to v1.5.5.

0.15.4 (Feb 24, 2023)

1. Upgrade zstd source code from v1.5.2 to v1.5.4. v1.5.3 is a non-public release.

2. Support pyproject.toml build mechanism (PEP-517). Note that specifying build options in old way may be invalid, see build commands.

3. Support “multi-phase initialization” (PEP-489) on CPython 3.11+, can work with CPython sub-interpreters in the future. Currently this build option is disabled by default.

4. Add a command line interface (CLI).

0.15.3 (Aug 3, 2022)

Fix ZstdError object can’t be pickled.

0.15.2 (Jan 22, 2022)

Upgrade zstd source code from v1.5.1 to v1.5.2.

0.15.1 (Dec 25, 2021)

1. Upgrade zstd source code from v1.5.0 to v1.5.1.

2. Fix ZstdFile.write() / train_dict() / finalize_dict() may use wrong length for some buffer protocol objects.

3. Two behavior changes:

   • Setting CParameter.nbWorkers to 1 now means “1-thread multi-threaded mode”, rather than “single-threaded mode”.

   • If the underlying zstd library doesn’t support multi-threaded compression, no longer automatically fallback to “single-threaded mode”, now raise a ZstdError exception.

4. Add a module level variable zstd_support_multithread.

5. Add a setup.py option --avx2, see build options.

0.15.0 (May 18, 2021)

1. Upgrade zstd source code from v1.4.9 to v1.5.0.

2. Some improvements, no API changes.

0.14.4 (Mar 24, 2021)

1. Add a CFFI implementation that can work with PyPy.

2. Allow dynamically link to zstd library.

0.14.3 (Mar 4, 2021)

Upgrade zstd source code from v1.4.8 to v1.4.9.

0.14.2 (Feb 24, 2021)

1. Add two convenient functions: compress_stream(), decompress_stream().

2. Some improvements.

0.14.1 (Dec 19, 2020)

1. Upgrade zstd source code from v1.4.5 to v1.4.8.

   • v1.4.6 is a non-public release for Linux kernel.

   • v1.4.8 is a hotfix for v1.4.7.

2. Some improvements, no API changes.

0.13.0 (Nov 7, 2020)

1. ZstdDecompressor class: now it has the same API and behavior as BZ2Decompressor / LZMADecompressor classes in Python standard library, it stops after a frame is decompressed.

2. Add an EndlessZstdDecompressor class, it accepts multiple concatenated frames. It is renamed from previous ZstdDecompressor class, but .at_frame_edge is True when both the input and output streams are at a frame edge.

3. Rename zstd_open() function to open(), consistent with Python standard library.

4. decompress() function:

   • ~9% faster when: there is one frame, and the decompressed size was recorded in frame header.

   • raises ZstdError when input or output data is not at a frame edge. Previously, it only raise for output data is not at a frame edge.

0.12.5 (Oct 12, 2020)

No longer use Argument Clinic, now supports Python 3.5+, previously 3.7+.

0.12.4 (Oct 7, 2020)

It seems the API is stable.

0.2.4 (Sep 2, 2020)

The first version upload to PyPI.

Includes zstd v1.4.5 source code.