Skip to main content

A minimalistic Google Storage client

Project description

gs is a command line interface (CLI) and Python library that provides a set of essential commands for Google Cloud Storage. It is modeled after the AWS CLI’s aws s3 command. Its features are:

  • Python 3 compatibility
  • A minimalistic set of dependencies
  • A tiny footprint
  • Intuitive convention-driven configuration of API credentials without browser login prompts
  • Checksum validation to ensure end-to-end data integrity in uploads and downloads
  • Progress bars for long-running upload and download operations
  • Resumable uploads and downloads
  • An attractive paging and table layout interface
  • A JSON object metadata output mode for feeding data to other utilities

Installation

pip install gs

Synopsis

Usage:
gs [OPTIONS] COMMAND [ARGS]...
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
gs configure Set gs config options, including the API key.
gs ls List buckets or objects in a bucket/prefix.
gs cp Copy files to, from, or between buckets.
gs mv Move files to, from, or between buckets.
gs mb Create a new Google Storage bucket.
gs rb Permanently delete an empty bucket.
gs rm Delete objects (files) from buckets.
gs sync Sync a directory of files with bucket/prefix.
gs api Use httpie to perform a raw HTTP API request.
gs presign Get a pre-signed URL for accessing an object.

Run gs configure to configure Google service account access credentials that will be used by the gs command. You can create a new service account key at https://console.cloud.google.com/iam-admin/serviceaccounts.

Credentials

Before making API calls, gs ingests API credentials in the following order of priority:

  • First, gs checks if a GOOGLE_APPLICATION_CREDENTIALS environment variable is set. If so, it attempts to load and use credentials from the service account credentials filename referenced by the variable.
  • If that varible is not set, gs attempts to load service account credentials previously configured with gs configure (stored in ~/.config/gs/config.json).
  • If that fails, gs attempts to load a service account API token from Google instance metadata.
  • If that fails, gs gives up and prints an error.

Using the Python library interface

from gs import GSClient
client = GSClient()
object_meta = client.get("b/my-bucket/o/my-object")
with client.get("b/my-bucket/o/my-object", params=dict(alt="media"), stream=True) as res:
    object_bytes = res.raw.read()
presigned_url = client.get_presigned_url("my-bucket", "my-object", expires_at=time.time()+3600)

Authors

  • Andrey Kislyuk

Bugs

Please report bugs, issues, feature requests, etc. on GitHub.

License

Licensed under the terms of the MIT License.

https://travis-ci.com/chanzuckerberg/gs.png https://img.shields.io/pypi/v/gs.svg https://img.shields.io/pypi/l/gs.svg https://codecov.io/gh/chanzuckerberg/gs/branch/master/graph/badge.svg

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
gs-0.4.0-py2.py3-none-any.whl (24.2 kB) Copy SHA256 hash SHA256 Wheel 2.7 Sep 29, 2018
gs-0.4.0.tar.gz (21.1 kB) Copy SHA256 hash SHA256 Source None Sep 29, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page