py7zr 0.11.0b1

Pure python 7-zip library

py7zr – a 7z library on python

py7zr is a library and utility to support 7zip archive compression, decompression, encryption and decryption written by Python programming language.

Install

You can install py7zr as usual other libraries using pip.

$ pip install py7zr

When you want to handle extra codecs (ZStandard) then add extra requirements to command line

$ pip install py7zr[zstd]

Documents

User manuals

• User Guide for latest version.

• API guide for latest version.

• Manual for stable version.

Developer guide

• Contributor guide for one want to contribute the project.

• 7z file specification

CLI Usage

You can run command script py7zr like as follows;

• List archive contents

$ py7zr l test.7z

• Extract archive

$ py7zr x test.7z

• Extract archive with password

$ py7zr x -P test.7z
  password?: ****

• Create and compress to archive

$ py7zr c target.7z test_dir

• Create multi-volume archive

$ py7zr c -v 500k target.7z test_dir

• Test archive

$ py7zr t test.7z

• Append files to archive

$ py7zr a test.7z test_dir

• Show information

$ py7zr i

• Show version

$ py7zr --version

SevenZipFile Class Usage

py7zr is a library which can use in your python application.

Decompression/Decryption

Here is a code snippet how to decompress some file in your application.

import py7zr

archive = py7zr.SevenZipFile('sample.7z', mode='r')
archive.extractall(path="/tmp")
archive.close()

You can also use ‘with’ block because py7zr provide context manager(v0.6 and later).

import py7zr

with py7zr.SevenZipFile('sample.7z', mode='r') as z:
    z.extractall()

with py7zr.SevenZipFile('target.7z', 'w') as z:
    z.writeall('./base_dir')

py7zr also supports extraction of single or selected files by ‘extract(targets=[‘file path’])’. Note: if you specify only a file but not a parent directory, it will fail.

import py7zr
import re

filter_pattern = re.compile(r'<your/target/file_and_directories/regex/expression>')
with SevenZipFile('archive.7z', 'r') as archive:
    allfiles = archive.getnames()
    selective_files = [f if filter_pattern.match(f) for f in allfiles]
    archive.extract(targets=selective_files)

py7zr support an extraction of password protected archive.(v0.6 and later)

import py7zr

with py7zr.SevenZipFile('encrypted.7z', mode='r', password='secret') as z:
    z.extractall()

Compression/Encryption

Here is a code snippet how to produce archive.

import py7zr

with py7zr.SevenZipFile('target.7z', 'w') as archive:
    archive.writeall('/path/to/base_dir', 'base')

To create encrypted archive, please pass a password.

import py7zr

with py7zr.SevenZipFile('target.7z', 'w', password='secret') as archive:
    archive.writeall('/path/to/base_dir', 'base')

To create archive with algorithms such as zstandard, you can call with custom filter.

import py7zr

my_filters = [{"id": py7zr.FILTER_ZSTD}]
another_filters = [{"id": py7zr.FILTER_ARM}, {"id": py7zr.FILTER_LZMA2, "preset": 7}]
with py7zr.SevenZipFile('target.7z', 'w', filters=my_filter) as archive:
    archive.writeall('/path/to/base_dir', 'base')

shutil helper

py7zr also support shutil interface.

from py7zr import pack_7zarchvie, unpack_7zarchive
import shutil

# register file format at first.
shutil.register_archive_format('7zip', pack_7zarchive, description='7zip archive')
shutil.register_unpack_format('7zip', ['.7z'], unpack_7zarchive)

# extraction
shutil.unpack_archive('test.7z', '/tmp')

# compression
shutil.make_archive('target', '7zip', 'src')

Required Python versions

py7zr uses a python3 standard lzma module for extraction and compression. The standard lzma module uses liblzma that support core compression algorithm of 7zip.

Minimum required version is Python 3.5. Two additional library is required only on Python3.5; contextlib2 and pathlib2.

Compression is supported on Python 3.6 and later. Multi-volume archive creation issupported on Python 3.7 and later.

There are other runtime requrements; texttable, pycryptodome

Version recommendations are:

• CPython 3.7.5, CPython 3.8.0 and later.

• PyPy3.6-7.2.0 and later.

Following fixes are included in these versions, and it is not fixed on python3.6.

• BPO-21872: LZMA library sometimes fails to decompress a file

• PyPy3-3088: lzma.LZMADecomporessor.decompress does not respect max_length

Compression Methods supported

‘py7zr’ supports algorithms and filters which lzma module and liblzma support. It also support BZip2 and Deflate that are implemented in python core libraries, and ZStandard with third party libraries. py7zr, python3 core lzma module and liblzma do not support some algorithms such as PPMd, BCJ2 and Deflate64.

Here is a table of algorithms.

#	Category	Algorithm combination
1	Compression Decompression	LZMA2 + Delta or BCJ(X86, ARM, PPC, IA64, ARMT, SPARC)
2		LZMA + BCJ(X86,ARMT,ARM,PPC,SPARC)
3		LZMA2, LZMA, Bzip2, Deflate, COPY
4		Bzip2,Deflate + BCJ(X86,ARM,PPC,ARMT,SPARC)
6	Encryption Decryption	7zAES + LZMA2 + Delta or BCJ
6		7zAES + LZMA
7		7zAES + Bzip2, Deflate
8	Compression only (experimental)	LZMA + BCJ(IA64)
9		PPMd
9	Unsupported	BCJ2, Deflate64
10		ZStandard

• A feature handling symbolic link is basically compatible with ‘p7zip’ implementation, but not work with original 7-zip because the original does not implement the feature.

• Decryption of filename encrypted archive is also supported.

• CAUTION: Specifying an unsupported algorithm combination may produce a broken archive.

• ZStandard support is under development, but not working yet.

Use Cases

• aqtinstall Another (unofficial) Qt (aqt) CLI Installer on multi-platforms.

• PreNLP Preprocessing Library for Natural Language Processing

• mlox a tool for sorting and analyzing Morrowind plugin load order

License

• Copyright (C) 2019,2020 Hiroshi Miura

• pylzma Copyright (c) 2004-2015 by Joachim Bauch

• 7-Zip Copyright (C) 1999-2010 Igor Pavlov

• LZMA SDK Copyright (C) 1999-2010 Igor Pavlov

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

Py7zr ChangeLog

All notable changes to this project will be documented in this file.

Unreleased

Added

Changed

Fixed

Deprecated

Removed

Security

Added

• Add PPMd compression support(experimental).

Fixed

• Fix zstandard compression/decompression.(#258)

Deprecated

• Drop support for python 3.5 which become end-of-line in Sept. 2020.

v0.10.1

Fixed

• Fix exception when reading header which size is larger than buffer size (#252)

v0.10.0

• Release is same as v0.10.0b3.

v0.10.0b3

Fixed

• Fix BCJ(x86) filter code with a missing logic which cause extraction error for certain data. (#249, #250)

v0.10.0b2

Added

• Compatibility test with python-libarchive-c/libarchive for compression(#247)

• Document: express how to handle multi-volume archive (#243)

Changed

• Calculate CRC32 of header without re-reading header from disk again.(#245)

v0.10.0b1

Added

• SevenZipFile.needs_password() method.(#208, #235)

• CLI: Support append mode command line.(#228)

• Support “APPEND” mode. User can open SevenZipFile() class with mode=’a’ (#227)

Changed

• read(), extract(): improve performance when specifying parts of archived file, by skipping rest of arcvhive when target file has extracted.(#239,#242)

• read(), extract(): improve performance when specifying parts of archived file, by not running threads for unused compression blocks(folders).(#239,#242)

• docs: improve API documentation.(#244)

• setup: set minimum required python version as >=3.5

• Compression will be happened when call write() not close() (#222, #226)

• Handle file read/write in SevenZipCompressor/Decompressor class (#213)

Fixed

• Raise PasswordRequired when encrypted header without passing password (#234, #237)

• CLI: don’t raise exception when password is wrong or not given.(#229)

• Fix specification typo.

• Catch exception in threading extraction(#218,#219)

v0.9.2

Changed

• Utilize max_length argument for each decompressor.(#210, #211)

• Change READ_BUFFER_SIZE 32768 for python 3.7.5 and before.

• Extend Buffer size when necessary.(#209)

v0.9.1

Changed

• Improve DecompressionChain.decompress() logics.(#207)

Fixed

• Fix BCJ filter for decompression that can cause infinite loop or wrong output.(#204,#205,#206)

v0.9.0

Added

• BCJ Decoder/Encoder written by python.(#198, #199)

• Support Bzip2, Defalte + BCJ(X86, PPC, ARM, ARMT, SPARC) (#199)

• Add Copy method as an extraction only support.(#184)

Changed

• Use large(1MB) read blocksize for Python 3.7.5 and later and PyPy 7.2.0 and later.

• Set ZStandard compression as unsupported because of a bug with unknown reason.(#198)

• Manage compression methods to handle whether decompressor requires coder[‘property’] or not.

Fixed

• Significantly improve decompress performance which is as same speed as v0.7.*. by updating buffer handling.

• Fix decompression max_size to pass lzma module. Now it is as same as out_remaining.

• Support LZMA+BCJ(X86, PPC, ARM, ARMT, SPARC) with alternative BCJ filter.(#198, #199)

• Fix packinfo crc read and write (#187, #189)

• Accept archive which Method ID is NULL(size=0)(#181, #182)

• CLI: Does not crash when trying extract archive which use unsupported method(#183)

v0.8.0

Added

• test: add test for #178 bug report the case of LZMA+BCJ as xfails.

• File format specification: add ISO/IEC standard style specification document.

• Support extra methods for archiveinfo() method.(#150)

• test: unit tests for Sparc, ARMT and IA64 filters.

• Support for PPC and ARM filters.

• Support encryption(#145)

• Export supported filter constants, such as FILTER_ZSTD(#145)

Changed

• Improve README, documents and specifications.

• Update password handling and drop get_password() helper (#162)

• Enable encoded header and add more test with 7zip compatibility.(#164)

• Refactoring SevenZipFile class internals. (#160)

• Refactoring classes in compressor module. (#161)

• Add ‘packinfo.crcs’ field digests data when creating archive.(#157) It help checking archive integrity without extraction.

• CLI: help option to show py7zr version and python version.

• Use importlib for performance improvement instead of pkg_resources module.

• Documents: additional methods, filter examples.

• CI configurations: Manage coverage with Coveralls.

• Refactoring decompression classes to handle data precisely with folder.unpacksizes(#146)

• Default compression mode is LZMA2+BCJ which is as same as 7zip and p7zip(#145)

• Enhance encryption strength, IV is now 16 bytes, and generated with cryptodom.random module.(#145)

• Refactoring compression algorythm related modules.

Fixed

• Now return correct header size by archiveinfo() method.(#169)

• Disable adding CRC for encoded header packinfo.(#164)

• Fix password leak/overwrite among SevenZipFile objects in a process.(#159) This can cause decryption error or encryption with unintended password.

• Release password on close()

• SevenZipFile.test() method now working properly. (#155)

• Fix extraction error on python 3.5.(#151)

• Support combination of filters(#145)

• Compression of Delta, BZip2, ZStandard, and Deflate(#145)

• Fix archived head by multiple filter specified.

• Fix delta filter.

• Working with BCJ filter.

• Fix archiveinfo to provide proper names.

Removed

• test: Drop some test case with large files.

• Drop ArchiveProperty class: A field has already deprecated or not used.(#170)

• Drop AntiFile property: a property has already deprecated or not used.

• remove final_header definition.