Cloud storage IO for humans
Project description
cloudstorageio
Storage agnostic IO interface for humans
Developed and tested on python 3.7+
Getting Started
These instructions will get you a copy of the project up and running on your local machine.
-
S3 confidropboxgs
pip install awscli --upgrade --user
set your aws configs
sudo apt install awscli
$ aws configure AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: us-west-2 Default output format [None]: json
click here for more info about configuration and installation
-
Google cloud storage configs
Provide authentication credentials to your application code by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS. Replace [FILE_NAME] with the file of the JSON file that contains your service account key
GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"
click here for more info about configuration and installation
-
Dropbox workspace configs
Provide app access token by setting the environment variable DROPBOX_TOKEN. Replace [TOKEN] with the generated token of your dropbox app
DROPBOX_TOKEN='TOKEN'
click here for more info about configuration and installation
Installation
pip install cloudstorageio
Usage
- Write and read file
from cloudstorageio import CloudInterface
google_file_path = 'gs://bucket-name/path-to-file/sample-file.txt'
s3_file_path = 's3://bucket-name/path-to-file/sample-pic.jpg'
ci = CloudInterface()
# Write text to Google cloud storage file
with ci.open(google_file_path, 'w') as f:
f.write("Lorem Ipsum is simply dummy text")
# Read picture from S3 storage
with ci.open(s3_file_path, 'rb') as f:
s3_output = f.read() # binary content of picture
- Remove, list, and check folder/file
from cloudstorageio import CloudInterface
dropbox_folder_path = 'dbx://bucket-name/folder'
s3_file_path = 's3://bucket-name/path-to-file/sample-pic.jpg'
ci = CloudInterface()
ci.isfile(s3_file_path) # returns True
ci.isdir(dropbox_folder_path) # returns True
ci.remove(s3_file_path) # removes file
ci.listdir(dropbox_folder_path) # lists folder content
- Copy file
from cloudstorageio import CloudInterface
dropbox_file_path = 'dbx://bucket-name/path-to-file/sample-pic.jpg'
google_file_path = 'gs://bucket-name/path-to-file/sample-file.txt'
ci = CloudInterface()
ci.copy(from_path=dropbox_file_path, to_path=google_file_path) # copies dropbox file to google
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for cloudstorageio-1.0.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4f2aac7600e8cad53a303b15602f9dd16abd36da117b82de9c3ce5dba0e38af |
|
MD5 | 28eeb02c9bcb0a1a18b4d8e7516b8a66 |
|
BLAKE2b-256 | ab2af73371e320a7177d75e0711fff2a4f8f7adc9e79ddd840623f9e361d1d2f |