py7zr 0.17.0

Pure python 7-zip library

py7zr – a 7z library on python

py7zr is a library and utility to support 7zip archive compression, decompression, encryption and decryption written by Python programming language.

Install

You can install py7zr as usual other libraries using pip.

$ pip install py7zr

OR, alternatively using conda:

$ conda install -c conda-forge py7zr

Documents

User manuals

• User Guide for latest version.

• API Guide for latest version.

• Manual for stable version.

Developer guide

• Contributor guide for one want to contribute the project.

• 7z file specification

CLI Usage

You can run command script py7zr like as follows;

• List archive contents

$ py7zr l test.7z

• Extract archive

$ py7zr x test.7z

• Extract archive with password

$ py7zr x -P test.7z
  password?: ****

• Create and compress to archive

$ py7zr c target.7z test_dir

• Create multi-volume archive

$ py7zr c -v 500k target.7z test_dir

• Test archive

$ py7zr t test.7z

• Append files to archive

$ py7zr a test.7z test_dir

• Show information

$ py7zr i

• Show version

$ py7zr --version

SevenZipFile Class Usage

py7zr is a library which can use in your python application.

Decompression/Decryption

Here is a code snippet how to decompress some file in your application.

import py7zr

archive = py7zr.SevenZipFile('sample.7z', mode='r')
archive.extractall(path="/tmp")
archive.close()

You can also use ‘with’ block because py7zr provide context manager(v0.6 and later).

import py7zr

with py7zr.SevenZipFile('sample.7z', mode='r') as z:
    z.extractall()

with py7zr.SevenZipFile('target.7z', 'w') as z:
    z.writeall('./base_dir')

py7zr also supports extraction of single or selected files by ‘extract(targets=[‘file path’])’. Note: if you specify only a file but not a parent directory, it will fail.

import py7zr
import re

filter_pattern = re.compile(r'<your/target/file_and_directories/regex/expression>')
with SevenZipFile('archive.7z', 'r') as archive:
    allfiles = archive.getnames()
    selective_files = [f if filter_pattern.match(f) for f in allfiles]
    archive.extract(targets=selective_files)

py7zr support an extraction of password protected archive.(v0.6 and later)

import py7zr

with py7zr.SevenZipFile('encrypted.7z', mode='r', password='secret') as z:
    z.extractall()

Compression/Encryption

Here is a code snippet how to produce archive.

import py7zr

with py7zr.SevenZipFile('target.7z', 'w') as archive:
    archive.writeall('/path/to/base_dir', 'base')

To create encrypted archive, please pass a password.

import py7zr

with py7zr.SevenZipFile('target.7z', 'w', password='secret') as archive:
    archive.writeall('/path/to/base_dir', 'base')

To create archive with algorithms such as zstandard, you can call with custom filter.

import py7zr

my_filters = [{"id": py7zr.FILTER_ZSTD}]
another_filters = [{"id": py7zr.FILTER_ARM}, {"id": py7zr.FILTER_LZMA2, "preset": 7}]
with py7zr.SevenZipFile('target.7z', 'w', filters=my_filter) as archive:
    archive.writeall('/path/to/base_dir', 'base')

shutil helper

py7zr also support shutil interface.

from py7zr import pack_7zarchive, unpack_7zarchive
import shutil

# register file format at first.
shutil.register_archive_format('7zip', pack_7zarchive, description='7zip archive')
shutil.register_unpack_format('7zip', ['.7z'], unpack_7zarchive)

# extraction
shutil.unpack_archive('test.7z', '/tmp')

# compression
shutil.make_archive('target', '7zip', 'src')

Requirements

py7zr uses a python3 standard lzma module for extraction and compression. The standard lzma module uses liblzma that support core compression algorithm of 7zip.

Minimum required version is Python 3.6.

py7zr tested on Linux, macOS, Windows and Ubuntu aarch64.

It hopefully works on M1 Mac too.

Recommended versions are:

• CPython 3.7.5, CPython 3.8.0 and later.

• PyPy3.6-7.3.2 and later.

• CPython 3.10.0 is supported from py7zr v0.17.0

Following fixes are included in these versions, and it is not fixed on python3.6.

• BPO-21872: LZMA library sometimes fails to decompress a file

• PyPy3-3090: lzma.LZMADecomporessor.decompress does not respect max_length

• PyPy3-3242: ‘_lzma_cffi’ has no function named ‘lzma_stream_encoder’

Following improvements are included in CPython 3.10

• BPO-41486: Faster bz2/lzma/zlib via new output buffering

Compression Methods supported

‘py7zr’ supports algorithms and filters which lzma module and liblzma support. It also support BZip2 and Deflate that are implemented in python core libraries, and ZStandard with third party libraries.

Supported algorithms are:

• compress

  • LZMA2

  • LZMA

  • Bzip2

  • Deflate

  • Copy

  • PPMd

  • ZStandard

  • Brotli

• crypt

  • 7zAES

• Filters

  • Delta

  • BCJ(X86,ARMT,ARM,PPC,SPARC,IA64)

• No support

  • BCJ2

  • Deflate64

• A feature handling symbolic link is basically compatible with ‘p7zip’ implementation, but not work with original 7-zip because the original does not implement the feature.

• ZStandard and Brotli is not default methods of 7-zip, so these archives are considered not to be compatible with original 7-zip on windows/p7zip on linux/mac.

• liblzma, which Python’s standard lzma module depends, does not provide BCJ2 filter.

• Deflate64 is proprietary algorithm.

Dependencies

There are several dependencies to support algorithms and CLI expressions.

Package	Purpose
Pycryptodomex	7zAES encryption
PyZstd	ZStandard compression
PyPPMd	PPMd compression
Brotli	Brotli compression (CPython)
BrotliCFFI	Brotli compression (PyPy)
pybcj	BCJ filter
multivolumefile	Multi-volume archive read/write
texttable	CLI formatter

Performance

You can find a compression and decompression benchmark results at [Github issue](https://github.com/miurahr/py7zr/issues/297) and [wiki page](https://github.com/miurahr/py7zr/wiki/Benchmarks)

py7zr works well, but slower than 7-zip and p7zip C/C++ implementation by several reasons. When compression/decompression speed is important, it is recommended to use these alternatives through subprocess.run python interface.

Use Cases

• aqtinstall Another (unofficial) Qt (aqt) CLI Installer on multi-platforms.

• PreNLP Preprocessing Library for Natural Language Processing

• mlox a tool for sorting and analyzing Morrowind plugin load order

License

• Copyright (C) 2019-2021 Hiroshi Miura

• pylzma Copyright (c) 2004-2015 by Joachim Bauch

• 7-Zip Copyright (C) 1999-2010 Igor Pavlov

• LZMA SDK Copyright (C) 1999-2010 Igor Pavlov

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

WARNING

Test archive data, which affected a malware, have been existed from Aug, 2020 - 20, Jan, 2021!

All the git history is re-writed, so please remove your local and fork copy of the git repository, and clone again(if necessary)!

Problematic file is named issue_218.7z and issue_218_2.7z.

There is NO affected in library itself. and the test execution also does not extract the malware file. There is no problem when you install py7zr with pip command.

Release that has a clean source:

• v0.11.3 and later

• v0.10.2

• v0.9.10

• v0.9.4 and before

Py7zr ChangeLog

All notable changes to this project will be documented in this file.

Unreleased

v0.17.0

Fixed

• Extraction: overwrite a symbolic link sometimes failed(#383)

• Allow creation of archive without any write call(#369,#372)

• Type check configuration update (#384)

• Adjust for type check errors (#384)

v0.16.4

Fixed

• Win32 file namespace convention doesn’t work on Cygwin(#380,#381)

• Win32 file namespace convention doesn’t work for network path(#380)

v0.16.3

Fixed

• Reduce memory consumptions and fix memory_error on 32bit python (#370,#373,#374,#375)

Added

• Add CI test for python 3.10 (#371)

v0.16.2

Added

• Bundle type hint data

• README: Add conda recipe(#342)

Changed

• Use PyBCJ instead of bcj-cffi.(#368)

• Docs: change recommended python versions

• CI: benchmark on python 3.10

• Test expectation for python 3.10 change

• Improve exceptions and error messages

• Docs: add description of ArchiveInfo class

• Docs: fix typo on shutil integration(#353)

• Bump pyzstd@0.15.0

• Bump pyppmd@0.17.0

Fixed

• Docs: specification error of signature header data types.

• Fix infinite loop in extract(#354)

v0.16.1

Added

• type hint for mypy

v0.16.0

Added

• Add Brotli compression.

• CI: Test on AArch64.

Changed

• CLI: support multi-volume archive without making temporary file(#311)

• Filter parameter: PPMd: mem is now accept int or “<val>{m|k|b}” as same as 7-zip command line option. int value is recognized as “1 << val” ie. 24 means 4MB.

• Dependency: PyPPMd v0.14.0+

• Dependency PyCryptodome to PyCryptodomex that changes package name from PyCrypto to PyCryptodome(#334)

v0.15.2

Added

• CLI: create sub-command(c) has -P or –password option.(#332)

Fixed

• Fix not to produce directory when memory extraction mode.(#323)

Changed

• Use PyPPMd v0.12.1 or later for ppmd compression instead of ppmd-cffi(#322)

• Add minimum version requirement for PyCryptodome (#329)

• Bump setuptools_scm @6.0.1

v0.15.1

Changed

• Update release automation script.

• Bump ppmd-cffi and bcj-cffi versions(#320)

v0.15.0

Added

• Add option to specify multiprocessing instead of multi-threading. (#306)

Changed

• Change Property Borg class to constant class(#319)

• Reformat whole code with black.

• Merge pyzstdfilter into compressor.py.

• Lint codes by flake8/black.

Fixed

• README: description of dependencies.

• ZStandard decompression on PyPy3

v0.14.1

Fixed

• Fix of empty file archive(#305,#310)

v0.14.0

Added

• Introduce writed() method that accept dict[name, BinaryIO](#302)

Changed

• READ_BLOCKSIZE configurable on constructor(#307)

• Use pyzstd for zstandard algorithm on CPython(#304)

• Use bcj-cffi library for lzma+bcj performance(#303)

• CLI: Fix getting module_name on 3.6.13(#308)

v0.13.0

Added

• Add writestr() and writef() methods in SevenZipFile class.(#290,#293)

• Add benchmark tests for compression algorithms(#295)

• Track benchmark results on Github issue(#296)

Changed

• Refactoring BCF Filter classes, and move to individual module.(#292)

v0.12.0

Changed

• PPMd and ZStandard is now one of default algorithms(#289)

• Increment copyright year

Fixed

• Crash when append files to an empty files archive(#286)

v0.11.3

Fixed

• Fix test failure when running on pypi source(#279)

Security

• Drop issue_218.7z test data wihch is reported a blackmoon trojan(#285)

v0.11.1

Changed

• Improve BCJ filter performance with LZMA1, ZStd compressions.

Fixed

• Fix to allow writing encrypted header(#280)

• Avoid crash when creationtime is wrong or Unix epoch. (#275,#276)

v0.11.0

Changed

• PPMd: Use stream encoder/decoder instead of buffered one.

• PPMd: Use ppmd-cffi@v0.3.1 and later.(#268)

Added

• PPMd compression/decompression support.(#255)

• New API to set methods to set header encode mode, encode or encrypted.(#259)

• Support Python 3.9.(#261)

• Support arm64/aarch64 architecture on Linux.(#262)

Fixed

• Append mode cause error when target archive use LZMA2+BCJ.(#266)

• Fix zstandard compression/decompression.(#258)

Deprecated

• Drop support for python 3.5 which become end-of-line in Sept. 2020.