Platforms like Heroku don’t allow for FUSE filesystem usage, so I had to get a bit creative.
Introducing, s3monkey, a library that mocks out all standard Python library system file operations, allowing you to use already–written code to interface with Amazon S3.
All standard library file operation modules are patched when using the provided context manager, including the built–in open, os, io, & pathlib.
If you’re interested in financially supporting Kenneth Reitz open source, consider visiting this link. Your support helps tremendously with sustainability of motivation, as Open Source is no longer part of my day job.
Potential Use Cases
Running Jupyter Notebooks on non-persistient storage (still being worked out).
Storing user uploads for Django applications (e.g. the media folder).
Usage
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are expected to be set:
$ AWS_ACCESS_KEY_ID=xxxxxxxxxxx
$ AWS_SECRET_ACCESS_KEY=xxxxxxxxxxx
Basic usage:
from s3monkey import S3FS
with S3FS(bucket='media.kennethreitz.com', mount_point='/app/data') as fs:
# Create a 'test' key on S3, with the contents of 'hello'.
with open('/app/data/test', 'w') as f:
f.write('hello')
# List the keys in the S3 bucket.
print(os.listdir('/app/data'))
# ['file1.txt', 'file2.txt', 'file2.txt', 'test', …]
Installation
$ pipenv install s3monkey
This module only supports Python 3.
Release files for s3monkey 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| s3monkey-0.1.1.tar.gz | 61.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| s3monkey-0.1.1-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details |
Total release size: 130.3 kB
Release files / s3monkey-0.1.1.tar.gz
| Download URL | s3monkey-0.1.1.tar.gz |
|---|---|
| Size | 61.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
743f3bf8dde13f9894e7a5bf7811f112c8ff54954e065146f842706c332f8dcc
|
|
BLAKE2b-256 checksum How to use checksums |
785bee6a1667effb5c5a5eae30f1fef2bf90307d76aeafdf401ca4dc393e1c4e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.44.1 CPython/3.8.2
|
Release files / s3monkey-0.1.1-py2.py3-none-any.whl
| Download URL | s3monkey-0.1.1-py2.py3-none-any.whl |
|---|---|
| Size | 69.1 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
696bf06be995421f57fb16a501260249070e68cea7ff6894e65856158f9c99f2
|
|
BLAKE2b-256 checksum How to use checksums |
d2cea4f759026b6bb9e4c1b5aa4d5d65dc1ec1cdae319b96f6917db7238db690
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.44.1 CPython/3.8.2
|