Skip to main content

A small tool to parse URLs for S3 compatible storage services

Project description

PyPI - Python Version PyPI PyPI - License https://coveralls.io/repos/github/marazmiki/s3-parse-url/badge.svg?branch=master Codacy grade

Parses S3 credentials from the given string and returns it in comfortable format to pass to popular clients like boto3.

About

This is a small utility to parse locations of buckets of S3-compatible storage (and, of course, the original Amazon’s S3 itself) given in the URL form like s3://bucket/.

It could be useful in our epoch of the 12-factor applications when it’s a good practice to store credentials inside of environment variables.

Also, these days, there are some notable S3-compatible storage services:

  • Selectel

  • MinIO (a self-hosted solution extremely handy for testing)

And dozens of others.

With s3-parse-url, you can use any one of these services with no doubts about configuration endpoints. For example, you can connect to your Selectel storage with boto3 just using selectel://my-bucket DSN.

That’s an example, what it was all about:

from s3_parse_url import s3_parse_url
from s3_parse_url.ext.clients import get_boto3_client

dsn = "s3://AKIA***OO:XP***@my-bucket/?region=us-east-2"

# It's a completely ready boto3 client instance to work with Selectel
s3_client = get_boto3_client(dsn)

Of course, in the code above we worked with Selectel (have you ever heard about it?). You can work this way with any S3 compatible storage. If you prefer unknown storage, you can easily create a plugin to add support for your favorite service. Or, if you are a pervert, you can use a universal S3:// scheme, but in this case, you should manage endpoints by yourself:

from s3_parse_url.ext.clients import get_boto3_client

# Also should work
dsn = "s3://my-minio-user:my-minio-pass@minio.example.com:9000/?region=us-east-1"

# A ready client to work with a minio instance
s3_client = get_boto3_client(dsn)

Supported providers

Currently we have support for these storages

  • Amazon S3

  • Selectel

  • Yandex

  • Mail.ru

  • MinIO

But you can easily add your own one.

License

This project is licensed under the terms of the MIT license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

s3_parse_url-0.4.0.tar.gz (6.0 kB view details)

Uploaded Source

Built Distribution

s3_parse_url-0.4.0-py3-none-any.whl (7.6 kB view details)

Uploaded Python 3

File details

Details for the file s3_parse_url-0.4.0.tar.gz.

File metadata

  • Download URL: s3_parse_url-0.4.0.tar.gz
  • Upload date:
  • Size: 6.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.9.19

File hashes

Hashes for s3_parse_url-0.4.0.tar.gz
Algorithm Hash digest
SHA256 c2aa8d80637f9ce7de4f4cc8f8eb385e38359270009ce0d7bf2f5d89d6b5bbea
MD5 3be0f4e6d2de813f247d2f694369a69c
BLAKE2b-256 1243dfc39eccabe0282f1c3b87bde91418b68ad30d7a4278403eb2e8d17da97f

See more details on using hashes here.

File details

Details for the file s3_parse_url-0.4.0-py3-none-any.whl.

File metadata

File hashes

Hashes for s3_parse_url-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d1af86f35f45b2b3460dae2a43c1d41b314504e3b0047b659ec8b7ef134472e8
MD5 92a96bed371f16061f2b93d489baad5c
BLAKE2b-256 4c92bfe0a7a1d01726b1d65a52164048a538c854092013271ed7b95fafba031d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page