Skip to main content

Zipfile generator that takes input files as well as streams

Project description

python-zipstream

zipstream.py is a zip archive generator based on python 3.3's zipfile.py. It was created to generate a zip file generator for streaming (ie web apps). This is beneficial for when you want to provide a downloadable archive of a large collection of regular files, which would be infeasible to generate the archive prior to downloading or of a very large file that you do not want to store entirely on disk or on memory.

The archive is generated as an iterator of strings, which, when joined, form the zip archive. For example, the following code snippet would write a zip archive containing files from 'path' to a normal file:

import zipstream

z = zipstream.ZipFile()
z.write('path/to/files')

with open('zipfile.zip', 'wb') as f:
    for data in z:
        f.write(data)

zipstream also allows to take as input a byte string iterable and to generate the archive as an iterator. This avoids storing large files on disk or in memory. To do so you could use something like this snippet:

def iterable():
    for _ in xrange(10):
        yield b'this is a byte string\x01\n'

z = zipstream.ZipFile()
z.write_iter('my_archive_iter', iterable())

with open('zipfile.zip', 'wb') as f:
    for data in z:
        f.write(data)

Of course both approach can be combined:

def iterable():
    for _ in xrange(10):
        yield b'this is a byte string\x01\n'

z = zipstream.ZipFile()
z.write('path/to/files', 'my_archive_files')
z.write_iter('my_archive_iter', iterable())

with open('zipfile.zip', 'wb') as f:
    for data in z:
        f.write(data)

Since recent versions of web.py support returning iterators of strings to be sent to the browser, to download a dynamically generated archive, you could use something like this snippet:

def GET(self):
    path = '/path/to/dir/of/files'
    zip_filename = 'files.zip'
    web.header('Content-type' , 'application/zip')
    web.header('Content-Disposition', 'attachment; filename="%s"' % (
        zip_filename,))
    return zipstream.ZipFile(path)

If the zlib module is available, zipstream.ZipFile can generate compressed zip archives.

Installation

pip install zipstream-new

Requirements

  • Python 2.6+, 3.2+, pypy

Examples

flask

from flask import Response

@app.route('/package.zip', methods=['GET'], endpoint='zipball')
def zipball():
    def generator():
        z = zipstream.ZipFile(mode='w', compression=zipstream.ZIP_DEFLATED)

        z.write('/path/to/file')

        for chunk in z:
            yield chunk

    response = Response(generator(), mimetype='application/zip')
    response.headers['Content-Disposition'] = 'attachment; filename={}'.format('files.zip')
    return response

# or

@app.route('/package.zip', methods=['GET'], endpoint='zipball')
def zipball():
    z = zipstream.ZipFile(mode='w', compression=zipstream.ZIP_DEFLATED)
    z.write('/path/to/file')

    response = Response(z, mimetype='application/zip')
    response.headers['Content-Disposition'] = 'attachment; filename={}'.format('files.zip')
    return response

# Partial flushing of the zip before closing

@app.route('/package.zip', methods=['GET'], endpoint='zipball')
def zipball():
    def generate_zip_with_manifest():
        z = zipstream.ZipFile(mode='w', compression=zipstream.ZIP_DEFLATED)

        manifest = []
        for filename in os.listdir('/path/to/files'):
            z.write(os.path.join('/path/to/files', filename), arcname=filename)
            yield from z.flush()
            manifest.append(filename)

        z.write_str('manifest.json', json.dumps(manifest).encode())

        yield from z

    response = Response(z, mimetype='application/zip')
    response.headers['Content-Disposition'] = 'attachment; filename={}'.format('files.zip')
    return response

django 1.5+

from django.http import StreamingHttpResponse

def zipball(request):
    z = zipstream.ZipFile(mode='w', compression=zipstream.ZIP_DEFLATED)
    z.write('/path/to/file')

    response = StreamingHttpResponse(z, content_type='application/zip')
    response['Content-Disposition'] = 'attachment; filename={}'.format('files.zip')
    return response

webpy

def GET(self):
    path = '/path/to/dir/of/files'
    zip_filename = 'files.zip'
    web.header('Content-type' , 'application/zip')
    web.header('Content-Disposition', 'attachment; filename="%s"' % (
        zip_filename,))
    return zipstream.ZipFile(path)

Running tests

With python version > 2.6, just run the following command: python -m unittest discover

Alternatively, you can use nose.

If you want to run the tests on all supported Python versions, run tox.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zipstream-new-1.1.8.tar.gz (9.4 kB view details)

Uploaded Source

Built Distribution

zipstream_new-1.1.8-py3-none-any.whl (20.2 kB view details)

Uploaded Python 3

File details

Details for the file zipstream-new-1.1.8.tar.gz.

File metadata

  • Download URL: zipstream-new-1.1.8.tar.gz
  • Upload date:
  • Size: 9.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/50.3.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.5

File hashes

Hashes for zipstream-new-1.1.8.tar.gz
Algorithm Hash digest
SHA256 b031fe181b94e51678389d26b174bc76382605a078d7d5d8f5beae083f111c76
MD5 acd41cc94496e08bd8e86a83b214d9e0
BLAKE2b-256 e5f31b5228576f215b200c7e922a280a92e4494df33baae6e0280a6f45371f13

See more details on using hashes here.

File details

Details for the file zipstream_new-1.1.8-py3-none-any.whl.

File metadata

  • Download URL: zipstream_new-1.1.8-py3-none-any.whl
  • Upload date:
  • Size: 20.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/50.3.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.5

File hashes

Hashes for zipstream_new-1.1.8-py3-none-any.whl
Algorithm Hash digest
SHA256 0662eb3ebe764fa168a5883cd8819ef83b94bd9e39955537188459d2264a7f60
MD5 5a50ff1a51e34a2161764455cb71f501
BLAKE2b-256 81f3d7b4c8c9b6657ff0db27b739894ed0665fa8f3c78a7452bf74d6447f6865

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page