Python utilities used for interacting with Minio Object Storage
Project description
Christopher H. Todd's PROJECT_STRING_NAME
The PROJECT_GIT_NAME project is responsible for ...
The library ...
Table of Contents
Dependencies
Python Packages
Libraries
minio_bucket_helpers.py
This library is used to interact with Minio object storage. Will handle functions used to interact with buckets (creating, downloading, finding, etc)
Functions:
def get_buckets(minio_client):
"""
Purpose:
Get a list of buckets that exist in the Minio Client
Args:
minio_client (minio client Obj): Client obj connection to Minio
Returns:
buckets (List of Bucket Objs): List of Bucket OBJs in Minio
"""
def get_bucket_names(minio_client):
"""
Purpose:
Get a list of buckets that exist in the Minio Client
Args:
minio_client (minio client Obj): Client obj connection to Minio
Returns:
bucket_names (List of Strings): List of Buckets in Minio
"""
def create_bucket(minio_client, bucket_name):
"""
Purpose:
Create a specified Bucket by name
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of bucket to create
Returns:
N/A
"""
def delete_bucket(minio_client, bucket_name):
"""
Purpose:
Delete a specified Bucket by name
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of bucket to delete
Returns:
N/A
"""
minio_client.py
MinioClient Class for interacting with minio object store. Objects will be created connected to Minio
Classes:
class MinioClient(object):
"""
MinioClient Class. Class objects will hold connection to the
Minio service and can be used to interact with buckets and objects
"""
minio_connection_helpers.py
This library is used to interact with Minio object storage. Functions establish a connection to the Minio service that can be used to interact with the service and pass to the other helper functions
Functions:
def connect_to_minio(minio_url, access_key=None, secret_key=None, secure=False):
"""
Purpose:
Connect to Minio and return the minio_client of minio lib
Args:
minio_url (String): URL of Minio
access_key (String): Access Key for Minio
secret_key (String): Secret Key for Minio
Returns:
minio_client (minio client Obj): Client obj connection to Minio
"""
def build_minio_url(minio_host, minio_port=9000):
"""
Purpose:
Create the Minio URL from host and port
Args:
minio_host (String): Host of Minio
minio_host (Int): Port of Minio (Defaults to 9000)
Returns:
minio_url (String): URL of Minio
"""
minio_exceptions.py
File for holding custom exception types that will be generated by the minio_helpers libraries
Exceptions:
class BucketAlreadyExists(Exception):
"""
Purpose:
The BucketAlreadyExists will be raised when attempting to create a bucket
that already exists
"""
class BucketDoesntExist(Exception):
"""
Purpose:
The BucketDoesntExist will be raised when attempting to delete a bucket
that doesn't exist in Minio
"""
class ObjectAlreadyExists(Exception):
"""
Purpose:
The ObjectAlreadyExists will be raised when attempting to create an object
that already exists
"""
class ObjectDoesntExist(Exception):
"""
Purpose:
The ObjectDoesntExist will be raised when attempting to pull an object
that doesn't exist in Minio
"""
class ObjectDecodingNotSupported(Exception):
"""
Purpose:
The ObjectDecodingNotSupported will be raised when attempting to decode a
filetype from Minio that is not yet supported (such as .avro)
"""
minio_general_helpers.py
This library is used to interact with Minio object storage.
Functions:
N/A
minio_object_helpers.py
Example executable Python scripts/modules for testing and interacting with the library. These show example use-cases for the libraries and can be used as templates for developing with the libraries or to use as one-off development efforts.
Functions:
def get_objects(minio_client, bucket_name):
"""
Purpose:
Get a list of objects that exist in the Minio Client
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to get objects for
Returns:
objects (List of Object Objs): List of Object OBJs in Minio
"""
def get_object_names(minio_client, bucket_name):
"""
Purpose:
Get a list of objects that exist in the Minio Client
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to get objects for
Returns:
object_names (List of Strings): List of Objects in Minio
"""
def is_object_in_bucket(minio_client, bucket_name, object_name):
"""
Purpose:
Check if Object exists in Bucket
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to check for object
object_name (String): Name of object to check for in Minio
Returns:
object_exists (Boolean): Boolean if the object exists or not
"""
def get_object_stats(minio_client, bucket_name, object_name):
"""
Purpose:
Get Stats of the Object
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to check for object
object_name (String): Name of object to get stats for in Minio
Returns:
object_stats (Dict):Dict of stats about the object
"""
def download_object_to_memory(minio_client, bucket_name, object_name, encoding="utf-8"):
"""
Purpose:
Download an Object from Mino into memory (if supported)
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to get object from
filename (String): Location (And Path) of file to upload
object_name (String): Name of object to upload in Minio
Returns:
parsed_object (Obj, depending on extension): Object parsed from Minio from the
extension of the file. Current supported = .txt -> str, .json -> Dict/JSON
"""
def download_object_to_file(minio_client, bucket_name, object_name, filename=None):
"""
Purpose:
Download a file from Minio to local storage
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to get object from
filename (String): Location (And Path) of file to upload
object_name (String): Name of object to upload in Minio
Returns:
N/A
"""
def upload_object(minio_client, bucket_name, filename, object_name=None):
"""
Purpose:
Uploading a local file to Minio
Args:
minio_client (minio client Obj): Client obj connection to Minio
bucket_name (String): Name of the bucket to get to upload object to
filename (String): Location (And Path) of file to upload
object_name (String): Name of object to upload in Minio
Returns:
N/A
"""
def delete_object(minio_client, bucket_name, object_name):
"""
Purpose:
Delete a specified Object by name
Args:
minio_client (minio client Obj): Client obj connection to Minio
object_name (String): Name of object in Minio to delete
Returns:
N/A
"""
Example Scripts
Example executable Python scripts/modules for testing and interacting with the library. These show example use-cases for the libraries and can be used as templates for developing with the libraries or to use as one-off development efforts.
connect_to_minio.py
Purpose:
Connecting to Minio
Steps:
- Connect to Minio
function call:python3 connect_to_minio {--access-key=access_key} \
{--secret-key=secret_key} {--minio-host=minio_host} {--minio-port=minio_port}
create_bucket_in_minio.py
Purpose:
Create Bucket in Minio
Steps:
- Connect to Minio
- Create a Bucket
function call:python3 create_bucket_in_minio.py {--access-key=access_key} \
{--secret-key=secret_key} {--minio-host=minio_host} {--minio-port=minio_port} \
{--bucket-name=bucket_name}
delete_bucket_from_minio.py
Purpose:
Delete Bucket from Minio
Steps:
- Connect to Minio
- Delete a Bucket
function call:python3 delete_bucket_from_minio.py {--access-key=access_key} \
{--secret-key=secret_key} {--minio-host=minio_host} {--minio-port=minio_port} \
{--bucket-name=bucket_name}
get_objects_from_bucket.py
Purpose:
Get all Objects (Or a Specific Object) From a Bucket in Minio
Steps:
- Connect to Minio
- Get the Bucket Obj
- Get objects from Minio and store locally
function call:python3 get_objects_from_Bucket.py {--access-key=access_key} \
{--secret-key=secret_key} {--minio-host=minio_host} {--minio-port=minio_port} \
{--bucket-name=bucket_name} {--object-name=object_name}
Notes
- Relies on f-string notation, which is limited to Python3.6. A refactor to remove these could allow for development with Python3.0.x through 3.5.x
TODO
- Unittest framework in place, but lacking tests
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ctodd-python-lib-minio-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 04ced019ab93f1ddb754e3ea0ec2e86cee78672aa8501eb4917fcf41e373ba5f |
|
MD5 | 9f8c84c2a63e15f5daa3683450ff09bd |
|
BLAKE2b-256 | a4de8c72bb9d42edf2deb6b06e38036776b25553e7213d3491e205edda606053 |
Hashes for ctodd_python_lib_minio-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d8eee6c0971458cbd2309aa06f89988ae47ccac1a92d33a31771963ad25f036 |
|
MD5 | b4b6b4d743105108484e83f08a9c35b9 |
|
BLAKE2b-256 | 35d6a2036ea6d8672aa83c60d6351b3e3a7ce8415fd33b918f5654ba0c9343e6 |