Skip to main content

No project description provided

Project description

S3Path

Latest version Travis-CI

S3Path provide a Python convenient File-System/Path like interface for AWS S3 Service using boto3 S3 resource as a driver.

Like pathlib, but for S3 Buckets

AWS S3 is among the most popular cloud storage solutions. It’s object storage, is built to store and retrieve various amounts of data from anywhere.

Currently, Python developers use Boto3 as the default API to connect / put / get / list / delete files from S3.

S3Path blends Boto3’s ease of use and the familiarity of pathlib api.

Install:

$ pip install s3path

Basic use:

The following example assumes an s3 bucket setup as specified bellow:

$ aws s3 ls s3://pypi-proxy/

2018-04-24 22:59:59        186 requests/index.html
2018-04-24 22:59:57     485015 requests/requests-2.9.1.tar.gz
2018-04-24 22:35:01      89112 boto3/boto3-1.4.1.tar.gz
2018-04-24 22:35:02        180 boto3/index.html
2018-04-24 22:35:19    3308919 botocore/botocore-1.4.93.tar.gz
2018-04-24 22:35:36        188 botocore/index.html

Importing the main class:

>>> from s3path import S3Path

Listing “subdirectories” - s3 keys can be split like file-system with a / in s3path we:

>>> bucket_path = S3Path('/pypi-proxy/')
>>> [path for path in bucket_path.iterdir() if path.is_dir()]
[S3Path('/pypi-proxy/requests/'),
 S3Path('/pypi-proxy/boto3/'),
 S3Path('/pypi-proxy/botocore/')]

Listing html source files in this “directory” tree:

>>> bucket_path = S3Path('/pypi-proxy/')
>>> list(bucket_path.glob('**/*.html'))
[S3Path('/pypi-proxy/requests/index.html'),
 S3Path('/pypi-proxy/boto3/index.html'),
 S3Path('/pypi-proxy/botocore/index.html')]

Navigating inside a “directory” tree:

>>> bucket_path = S3Path('/pypi-proxy/')
>>> boto3_package_path = bucket_path / 'boto3' / 'boto3-1.4.1.tar.gz'
>>> boto3_package_path
S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz')

Querying path properties:

>>> boto3_package_path = S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz')
>>> boto3_package_path.exists()
True
>>> boto3_package_path.is_dir()
False
>>> boto3_package_path.is_file()
True

Opening a “file” (s3 key):

>>> botocore_index_path = S3Path('/pypi-proxy/botocore/index.html')
>>> with botocore_index_path.open() as f:
>>>     print(f.read())
"""
<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Package Index</title>
</head>
<body>
    <a href="botocore-1.4.93.tar.gz">botocore-1.4.93.tar.gz</a><br>
</body>
</html>
"""

Or Simply reading:

>>> botocore_index_path = S3Path('/pypi-proxy/botocore/index.html')
>>> botocore_index_path.read_text()
"""
<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Package Index</title>
</head>
<body>
    <a href="botocore-1.4.93.tar.gz">botocore-1.4.93.tar.gz</a><br>
</body>
</html>
"""

Requirements:

  • Python >= 3.4

  • boto3

For pathlib style documentation of all interfaces interface_docs.

For s3path vs boto3 comparison boto3_comparison.

For advance features (configurations/s3 parameters) boto3_advance.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

s3path-0.1.6.tar.gz (12.2 kB view details)

Uploaded Source

Built Distribution

s3path-0.1.6-py3-none-any.whl (11.5 kB view details)

Uploaded Python 3

File details

Details for the file s3path-0.1.6.tar.gz.

File metadata

  • Download URL: s3path-0.1.6.tar.gz
  • Upload date:
  • Size: 12.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5

File hashes

Hashes for s3path-0.1.6.tar.gz
Algorithm Hash digest
SHA256 e8b000f0c48ff9bef0837825f310b01cf6ae4172587da45b9316ee30d3395253
MD5 813fad1992611c5aa30e22c670c26c1d
BLAKE2b-256 10cfa2c6d604f099a616a03946c9cd62604d87013d44c7608f1f9ed7853ebdc8

See more details on using hashes here.

File details

Details for the file s3path-0.1.6-py3-none-any.whl.

File metadata

  • Download URL: s3path-0.1.6-py3-none-any.whl
  • Upload date:
  • Size: 11.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5

File hashes

Hashes for s3path-0.1.6-py3-none-any.whl
Algorithm Hash digest
SHA256 579fa479ea2c8b3fc014b180ac2513ab4bedd265b92dbff98081334f4f2ae02c
MD5 02e4f752d548918b667944f4899f2d41
BLAKE2b-256 a99b8f1eb5c7c77f1a08784c25e92bd62c91c3e24caa9f51abb1fa37a85a1acb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page