Generic object storage interface and commands.
Project description
The Object IO Library and Command
Object stores (S3, Google, Azure, Minio, Swift, etc.) all have their own command line interfaces with their own conventions for accessing objects. This library and command provides a simple, uniform interface to object store facilities, both from the command line and from within Python.
Installation
$ pip install objectio
For the Github version:
$ pip install git+https://github.com/tmbdev/objectio.git
Documentation
Command Line Usage
$ obj cat az://container/blobname
$ obj cat gs://bucket/blobname
$ obj cat s3://bucket/blobname
$ obj cat file:/path
$ cat file | obj put gs://bucket/blobname
Python Usage
import objectio
with objectio.gopen("gs://bucket/blob", "rb") as stream:
contents = stream.read()
New Protocols
You can define new schemes by creating a ./objectio.yaml
or ~/.objectio.yaml
file. For example:
schemes:
random:
read:
cmd: ["cat", "/dev/random"]
Future Extensions
The intention is to keep this library simple and always allow command line programs to be configured for I/O as needed by end users.
Additional functionality:
- better error handling of non-zero return codes
obj dir az://container --format csv
obj del gs://container/blob
- possibly port to Go
Note that for Python, running I/O in a separate process is preferable to using Python-native libraries, since the latter do not run concurrently. For a Go language implementation, protocol implementations using built in native client libraries are potentially useful.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for objectio-0.2.29-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2577426a393d0e8a5b27370798e493b89f2d8c5619614199ea5521e4feee6f11 |
|
MD5 | 5c7a67b06614f937e9a3e8497c4ad223 |
|
BLAKE2b-256 | 86e3a132a91c4e9fd5e59c947263c7ef4e3415640fa151344f858e2def8c1726 |