py7zr 0.6b7

Pure python 7-zip decompression(restricted) library

py7zr

Pure python 7-zip implementation

Dependency

It uses a standard lzma module and its API supported. It also uses type definitions introduced in Python3.6.

Minimum required version is Python 3.5.

Version recommendations are:

• CPython 3.7.5, CPython 3.8.0 and later.

• PyPy3.6-7.2.0 and later.

Following fixes are included in above versions and python3.6 series is not fixed it.

• BPO-21872: LZMA library sometimes fails to decompress a file

• PyPy3-3088: lzma.LZMADecomporessor.decompress does not respect max_length

Document

Here is a readthedocs manual document.

Usage

You can run command script py7zr like as follows;

$ py7zr l test.7z
$ py7zr x test.7z
$ py7zr w target.7z test_dir

py7zr is a library which can use in your pyhton application. Here is a code snippet how to decompress some file in your applicaiton.

import py7zr

archive = py7zr.SevenZipFile('sample.7z', mode='x')
archive.extractall(path="/tmp")
archive.close()

Here is a code snippet how to produce archive.

import py7zr

archvie = py7zr.SevenZipFile('target.7z', 'w')
archive.writeall('./base_dir')
archive.close()

py7zr also support shutil interface.

from py7zr import pack_7zarchvie, unpack_7zarchive
import shutil

# register file format at first.
shutil.register_archive_format('7zip', pack_7zarchive, description='7zip archive')
shutil.register_unpack_format('7zip', ['.7z'], unpack_7zarchive)

# extraction
shutil.unpack_archive('test.7z', '/tmp')

# compression
shutil.make_archive('target', '7zip', 'src')

Use Cases

• aqtinstall Another (unofficial) Qt (aqt) CLI Installer on multi-platforms.

• PreNLP Preprocessing Library for Natural Language Processing

• mlox a tool for sorting and analyzing Morrowind plugin load order

License

• Copyright (C) 2019 Hiroshi Miura

• Copyright (c) 2004-2015 by Joachim Bauch

• 7-Zip Copyright (C) 1999-2010 Igor Pavlov

• LZMA SDK Copyright (C) 1999-2010 Igor Pavlov

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

Py7zr ChangeLog

All notable changes to this project will be documented in this file.

Unreleased

Added

Changed

Fixed

Deprecated

Removed

Security

v0.6b7

Added

• Test: add benchmarking test.

Changed

• Concurrency strategy: change to threading instead of multiprocessing. (#92)

v0.6b6

Fixed

• Make pywin32 a regular runtime dependency

v0.6b5

Added

• Add concurrent extraction test.

• Add remote data test for general application test.

• Add class for multi volume header.

• Add readlink helper function for windows.

Changed

• Release process is done by Github Actions

• Temporary disable to measure coverage, which is not working with threading.

• Tox: now pass PYTEST_ADDOPTS environment variable.

Fixed

• Build with pep517 utility.

• Fix race condition for changing current working directory of caller, which cause failures in multithreading.(#80,#82)

Security

• CLI: Use ‘getpass’ standard library to input password.(#59)

v0.6b4

Changed

• extract: decompression is done as another process in default.

• extract: default multiprocessing mode is spawn

• extract: single process mode for password protected archive.

Fixed

• extract: catch UnsupportedMethod exception properly when multiprocessing.

v0.6b3

Added

• Test: download and extract test case as a show case.

• setup.cfg: add entry-point configuration.

Changed

• Use spawn multiprocessing mode for all platforms.

• Use self context for multiprocessing.

Removed

• Static py7zr binary. Now it is generated by python installer.

v0.6b2

Changed

• Concurrency implementation changes to use multiprocessing.Process() instead of concurrency.futures to avoid freeze or deadlock with application usage of it.(#70)

• Stop checking coverage because coverage.py > 5.0.0 produce error when multiprocessing.Process() usage.

• Drop handlers, NullHandler, BufferHnalder, and FileHander.

Known Issues

• Extraction of encrypted archive which has multiple compression folders fails when multiprocessing mode is not ‘fork’, that is python3.8 and later on MacOS, and on Windows. see. test_extract_encrypted_2()

v0.6b1

Fixed

• Fixed extraction of 7zip file with BZip2 algorithm.(#66)

v0.6a2

Added

• Support filtering a target of extracted files from archive (#64)

Fixed

• Fix symbolic link extraction with relative path target directory.(#67)

v0.6a1

Added

• Support decryption (#55)

• Add release note automation workflow with Github actions.

• COPY decompression method.(#61)

Fixed

• Fix retrieving Folder header information logics for codecs.(#62)

Removed

• Test symlink on windows.(#60)

v0.5b6

Fixed

• Fix extraction of archive which has zero size files and directories(#54).

• Revert zero size file logic(#47).

v0.5b5

Fixed

• Revert zero size file logic which break extraction by 7zip.

v0.5b4

Fixed

• Support for making archive with zero size files(#47).

• Produced broken archive when target has many directorires(#48).

• Reduce test warnings, fix annotations.

• Fix coverage error on test.

v0.5b3

Fixed

• Support for making archive with symbolic links.

v0.5b2

Changed

• Update documents.

Fixed

• Fix write logics (#42)

• Fix read FilesInfo block.

v0.5b1

Support making a 7zip archive.

Added

• Support for compression and archiving.

• Support encoded(compressed) header and set as default.(#39)

• SevenZipFile: accept pathlib.Path as a file argument.

• Unit test: read and write UTF-16LE string for filename.

• Support for shutil.register_archive_format() and shutil.make_archive() by exposing pack_7zarchive()

• Support custom filters for compression.

Fixed

• Skip rare case when directory already exist, that can happen multiple process working in same working directory.

• Write: Produce a good archive file for multiple target files.

• SignatureHeader function: write nextheaderofs and nextheadersize as real_uint64.

• docs: description of start header structure.

Removed

• Drop py7zr.properties.FileAttributes; please use stat.FILE_ATTRIBUTES_*

Changed

• Test: Use tmp_path fixture which is pytest default one.

• Move setuptools configurations in setup.py into setup.cfg.

v0.4

Added

• Support for pypy3 (pypy3.5-7.0) and later(pypy3.6-7.1 or later).

• unit test for NullHandler, BufferHandler, FileHandler.

• Update document to add 7zformat descriptions.

Changed

• NullHandler, BufferHandler, FileHandler: open() now takes mode argument.

• Upper limit of max_length of decompress() call is now io.DEFAULT_BUFFER_SIZE. - PyPy issue: https://bitbucket.org/pypy/pypy/issues/3088/lzmalzmadecompressordecompress-data

• Drop padding logic introduced in v0.3.5 that may be cuased by python core bug, when max_length > io.DEFAULT_BUFFER_SIZE. - PyPy Issue: https://bitbucket.org/pypy/pypy/issues/3090/lzma-sometimes-decompresses-data - bpo-21872: https://bugs.python.org/issue21872 - Fix: https://github.com/python/cpython/pull/14048

• Remove print functions from API and moves CLI

  • API should not output anything other than error message. * Introduce FileInfo class to represent file attributes inside archive. * Introduce ArchiveInfo class to represent archive attributes. * provide archiveinfo() method to provide ArchiveInfo object. * now list() method returns List[FileInfo]

  • Every print things moves to Cli class.

• Update tests according to API change.

• Update documents to refrect API changes.

Fixed

• Update README to indicate supported python version as 3.5 and later, pypy3 7.1 and later.

v0.3.5

Changed

• Use seek&truncate for padding trailer if needed.

v0.3.4

Added

• Docs: class diagram, design note, 7z formats and presentations.

• Test for a target includes padding file.

Changed

• Test file package naming.

Fixed

• Fix infinite loop when archive file need padding data for extraction.

v0.3.3

Added

• Add test for zerofile with multi-foler archive.

Fixed

• Fix zerofile extraction error with multithread mode(#24, thanks @Arten013)

v0.3.2

Added

• typing hints

• CI test with mypy

• Unit test: SignatureHeader.write() method.

• Unit test: unknown mode for SevenZipFile constructor.

• Unit test: SevenZipFile.write() method.

Changed

• Conditional priority not likely to be external in header.

• Refactoring read_uint64().

Fixed

• SignatureHeader.write(): fix exception to write 7zip version.

v0.3.1

Added

• CLI i subcommand: show codec information.

• Decompression performance test as regression test.

• Add more unit test for helper functions.

Changed

• List subcommand now do not show compressed file size in solid compression. This is as same behavior as p7zip command.

• Merge io.py into archiveinfo.py

• Drop internal intermediate queue, which is not used.

Fixed

• Always overwrite when archive has multiple file with same name.

v0.3

Added

• Add some code related to support write feature(wip).

• Static check for import order in python sources and MANIFEST.in

Changed

• Concurrent decompression with threading when an archive is in multi folder compression.

• Pytest configurations are set in tox.ini

Fixed

• Package now has test code and data.

v0.2.0

Fixed

• Detect race condition on os.mkdir

v0.1.6

Fixed

• Wrong file size when lzma+bcj compression.

v0.1.5

Fixed

• Suppress warning: not dequeue from queue length 0

v0.1.4

Changed

• When a directory exist for target, do not raise error, and when out of it raise exception

• Refactoring FileArchivesList and FileArchive classes.

v0.1.3

Changed

• When a directory exist for target, do not raise error, and when out of it raise exception

v0.1.2

Changed

• Refactoring CLI with cli package and class.

Fixed

• Archive with zero size file cause exception with file not found error(#4).

Removed

• Drop unused code chunks.

• Drop Digests class and related unit test.

v0.1.1

Added

• Add write(), close() and testzip() dummy methods which raises NotImplementedError.

• Add more unit tests for write functions.

Fixed

• Fix Sphinx error in documentation.

• SevenZipFile: Check mode before touch file.

• Fix write_boolean() when array size is over 8.

• Fix write_uint64() and read_uint64().

v0.1.0

Added

• Introduce compression package.

• Introduce SevenZipCompressor class.

• Add write() method for each header class.

• Add tests for write methods.

• Add method for registering shutil.

Changed

• Each header classes has __slots__ definitions for speed and memory optimization.

• Rename to ‘io’ package from ‘archiveio’

• Each header classes has classmethod ‘retrieve’ and constructor does not reading a archive file anymore.

• Change to internalize _read() method for each header classes.

• get_decompressor() method now become SevenZipDecompressor class.

• Each header classes initializes members to None in constructor.

• Method definitions map become an internal member of SevenZipDecompressor or SevenZipCompressor class.

• Add test package compress

Fixed

• Fix ArchiveProperties read function.

v0.0.8

Added

• Test for CLI.

Changed

• Improve main function.

• Improve tests, checks outputs with sha256

v0.0.7

Added

• CI test on AppVeyor.

Changed

• Worker class refactoring.

Fixed

• Fix test cases: bugzilla_16 and github_14.

• Test: set timezone to UTC on Unix and do nothing on Windows.

v0.0.6

Fixed

• Fix too many file descriptors opened error.

v0.0.5

Changed

• Test: check sha256 for extracted files

Fixed

• Fix decompressiong archive with LZMA2 and BCJ method

• Fix decompressing multi block archive

• Fix file mode on unix/linux.

v0.0.4

Added

• Set file modes for extracted files.

• More unit test.

Changed

• Travis-CI test on python 3.7.

Fixed

• Fix to set extracted files timestamp as same as archived.

v0.0.3

Added

• PyPi package index.

Fixed

• setup: set universal = 0 because only python 3 is supported.

v0.0.2

Changed

• refactoring all the code.