Python function to construct a ZIP archive with stream processing - without having to store the entire ZIP in memory or disk
Project description
stream-zip
Python function to construct a ZIP archive on the fly - without having to store the entire ZIP in memory or disk. This is useful in memory-constrained environments, or when you would like to start returning compressed data before you've even retrieved all the uncompressed data. Generating ZIPs on-demand in a web server is a typical use case for stream-zip.
Offers similar functionality to zipfly, but with a different API, and does not use Python's zipfile module under the hood.
To unZIP files on the fly try stream-unzip.
Installation
pip install stream-zip
Usage
from datetime import datetime
from stream_zip import ZIP64, NO_COMPRESSION, stream_zip
def unzipped_files():
modified_at = datetime.now()
perms = 0o600
def file_1_data():
yield b'Some bytes'
def file_2_data():
yield b'Some bytes'
# ZIP64 mode
yield 'my-file-1.txt', modified_at, perms, ZIP64, file_1_data()
# No compression
yield 'my-file-2.txt', modified_at, perms, NO_COMPRESSION, file_2_data()
for zipped_chunk in stream_zip(unzipped_files()):
print(zipped_chunk)
Limitations
It's not possible to completely stream-write ZIP files. Small bits of metadata for each member file, such as its name, must be placed at the end of the ZIP. In order to do this, stream-unzip buffers this metadata in memory until it can be output.
stream-unzip creates ZIP64 files to support sizes bigger than 4GB. Older software may not be able to open these.
No compression is supported via the NO_COMPRESSION
constant as in the above examples. However in this case the entire contents of these are buffered in memory, and so this should not be used for large files.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for stream_zip-0.0.12-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4cbb8622eb5e546927bd39c2c57eeef6bd913b4d7c38992b597ecd43b168919 |
|
MD5 | e0ffa594c0e1efbc4d114496f3a4f545 |
|
BLAKE2b-256 | 90c55ee24734dcba2916f7c0467616369c349e0769fe0f7c0064dfc47af7d707 |