Skip to main content

wraps gsutil, a command-line interface to Google Cloud Storage.

Project description


gsutilwrap wraps Google Storage gsutil command-line interface in order to simplify the deployment and backup tasks related to Google Cloud Storage. It provides a set of data manipulation commands including copying, reading, writing and hashing stored data.

We primarly needed something simple that can still leverage mutli-threading, has decent progress output and implements robust pattern matching. Since gsutil CLI already provides all this functionality, we decided to wrap it. The wrapper adds type-annotated arguments and provides code inspection and autocomplete feature in an IDE such as PyCharm.

Additionally, since gsutil lacked copying of multiple patterns to multiple targets, we created this extra feature in gsutilwrap.

If you need to transfer data from/to Google Cloud Storage in the core of your application, we would recommend you to use the library google-cloud-storage provided by Google itself. That library is much more sophisticated in terms of features and would not incur you the overhead of authorizing and spawning a process for each operation. However, it lacks pattern matching (except for matching the prefixes) and you have to manage multi-threading and progress output yourself.


import pathlib

import gsutilwrap

# list
lst =

lst = gsutilwrap.ls_many(

# if you need a listing with size and update time, use long_ls
entries = gsutilwrap.long_ls(

for entry in entries:
    print("File size and update time of {}: {} {}".format(
        entry.url, entry.size, entry.update_time))

# write/read text
    text='some text')

text = gsutilwrap.read_text(

# write/read bytes

data = gsutilwrap.read_bytes(

# copy


        ("gs://some-bucket/some-path/*.txt", "/some/dir"),
        ("gs://some-bucket/some-path/*.xml", "/some/other/dir")

# stat an object
stat = gsutilwrap.stat(
print("Modification time: {}".format(stat.file_mtime))
print("Size: {}".format(stat.content_length))
print("MD5: {}".format(stat.md5.hex()))


  • Create a virtual environment:

python3 -m venv venv3
  • Activate it:

source venv3/bin/activate
  • Install gsutilwrap with pip:

pip3 install gsutilwrap


  • Check out the repository.

  • In the repository root, create the virtual environment:

python3 -m venv venv3
  • Activate the virtual environment:

source venv3/bin/activate
  • Install the development dependencies:

pip3 install -e .[dev]
  • We provide a set of live tests. The live tests need an existing bucket in the Google Cloud Storage. You need to set the URL prefix which will be used for all the live tests via the environment variable TEST_GSUTILWRAP_URL_PREFIX.

    Mind that the live tests will use Google Cloud resources for which you will be billed. Always check that no resources are used after the tests finished so that you don’t incur an unnecessary cost!

  • We use tox for testing and packaging the distribution. Assuming that the virtual environment has been activated and the development dependencies have been installed, run:

  • We also provide a set of pre-commit checks that lint and check code for formatting. Run them locally from an activated virtual environment with development dependencies:

  • The pre-commit script can also automatically format the code:

./  --overwrite


We follow Semantic Versioning. The version X.Y.Z indicates:

  • X is the major version (backward-incompatible),

  • Y is the minor version (backward-compatible), and

  • Z is the patch version (backward-compatible bug fix).

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gsutilwrap-1.1.2.tar.gz (11.5 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page