No project description provided

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for lior.miz from gravatar.com lior.miz

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Lior Mizrahi
  • Requires: Python >= 3.4
Classifiers
Report project as malware

Project description

S3Path

Latest version S3Path CI

S3Path provide a Python convenient File-System/Path like interface for AWS S3 Service using boto3 S3 resource as a driver.

Like pathlib, but for S3 Buckets

AWS S3 is among the most popular cloud storage solutions. It’s object storage, is built to store and retrieve various amounts of data from anywhere.

Currently, Python developers use Boto3 as the default API to connect / put / get / list / delete files from S3.

S3Path blends Boto3’s ease of use and the familiarity of pathlib api.

Install:

From PyPI:

$ pip install s3path

From Conda:

$ conda install -c conda-forge s3path

Basic use:

The following example assumes an s3 bucket setup as specified bellow:

$ aws s3 ls s3://pypi-proxy/

2018-04-24 22:59:59        186 requests/index.html
2018-04-24 22:59:57     485015 requests/requests-2.9.1.tar.gz
2018-04-24 22:35:01      89112 boto3/boto3-1.4.1.tar.gz
2018-04-24 22:35:02        180 boto3/index.html
2018-04-24 22:35:19    3308919 botocore/botocore-1.4.93.tar.gz
2018-04-24 22:35:36        188 botocore/index.html

Importing the main class:

>>> from s3path import S3Path

Listing “subdirectories” - s3 keys can be split like file-system with a / in s3path we:

>>> bucket_path = S3Path('/pypi-proxy/')
>>> [path for path in bucket_path.iterdir() if path.is_dir()]
[S3Path('/pypi-proxy/requests/'),
 S3Path('/pypi-proxy/boto3/'),
 S3Path('/pypi-proxy/botocore/')]

Listing html source files in this “directory” tree:

>>> bucket_path = S3Path('/pypi-proxy/')
>>> list(bucket_path.glob('**/*.html'))
[S3Path('/pypi-proxy/requests/index.html'),
 S3Path('/pypi-proxy/boto3/index.html'),
 S3Path('/pypi-proxy/botocore/index.html')]

Navigating inside a “directory” tree:

>>> bucket_path = S3Path('/pypi-proxy/')
>>> boto3_package_path = bucket_path / 'boto3' / 'boto3-1.4.1.tar.gz'
>>> boto3_package_path
S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz')

Querying path properties:

>>> boto3_package_path = S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz')
>>> boto3_package_path.exists()
True
>>> boto3_package_path.is_dir()
False
>>> boto3_package_path.is_file()
True

Opening a “file” (s3 key):

>>> botocore_index_path = S3Path('/pypi-proxy/botocore/index.html')
>>> with botocore_index_path.open() as f:
>>>     print(f.read())
"""
<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Package Index</title>
</head>
<body>
    <a href="botocore-1.4.93.tar.gz">botocore-1.4.93.tar.gz</a><br>
</body>
</html>
"""

Or Simply reading:

>>> botocore_index_path = S3Path('/pypi-proxy/botocore/index.html')
>>> botocore_index_path.read_text()
"""
<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Package Index</title>
</head>
<body>
    <a href="botocore-1.4.93.tar.gz">botocore-1.4.93.tar.gz</a><br>
</body>
</html>
"""

Requirements:

  • Python >= 3.4

  • boto3

  • smart-open

Further Documentation:

Similar Projects:

  • pathlib : Python standard library’s Object-oriented filesystem paths and this project base

  • pathy : Similar project which provides a Path interface for GoogleCloudStorage.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for lior.miz from gravatar.com lior.miz

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Lior Mizrahi
  • Requires: Python >= 3.4
Classifiers

Release history Release notifications | RSS feed

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.8

0.5.7

0.5.6

0.5.5

0.5.4

0.5.3

0.5.2

0.5.1

0.5.0

0.4.2

0.4.1

0.4.0

0.3.4

0.3.3

This version

0.3.2

0.3.1

0.3.0

0.2.205

0.2.204

0.2.203

0.2.202

0.2.201

0.2.101

0.1.101

0.1.93

0.1.92

0.1.91

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

s3path-0.3.2.tar.gz (14.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

s3path-0.3.2-py3-none-any.whl (14.3 kB view details)

Uploaded Python 3

File details

Details for the file s3path-0.3.2.tar.gz.

File metadata

  • Download URL: s3path-0.3.2.tar.gz
  • Upload date:
  • Size: 14.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.5

File hashes

Hashes for s3path-0.3.2.tar.gz
Algorithm Hash digest
SHA256 7e0a85015d2651c13019faf085cb4c0a3328f7cc41ac601874cfdc550af00a8e
MD5 ed5800a0a63485f311289eb5474973f1
BLAKE2b-256 db92166c66459f9ca850ee03cda15e78535e306149aba02bf49e0a745a28f702

See more details on using hashes here.

File details

Details for the file s3path-0.3.2-py3-none-any.whl.

File metadata

  • Download URL: s3path-0.3.2-py3-none-any.whl
  • Upload date:
  • Size: 14.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.5

File hashes

Hashes for s3path-0.3.2-py3-none-any.whl
Algorithm Hash digest
SHA256 4493f418543f9d79b4b4a9c32fbb2d3cede7071f51e24e3e5d83a9b1457b7a8a
MD5 28adcc21ff491e91172bfeb1602cb43e
BLAKE2b-256 1d117214d5503ee0bb6995dbef1c7704143527b0d3af98a382dae7f5b99cbf65

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page