Skip to main content

Platforms like Heroku don’t allow for FUSE filesystem usage, so I had to get a bit creative.

Introducing, s3monkey, a library that mocks out all standard Python library system file operations, allowing you to use already–written code to interface with Amazon S3.

All standard library file operation modules are patched when using the provided context manager, including the built–in open, os, io, & pathlib.

If you’re interested in financially supporting Kenneth Reitz open source, consider visiting this link. Your support helps tremendously with sustainability of motivation, as Open Source is no longer part of my day job.

Potential Use Cases

  • Running Jupyter Notebooks on non-persistient storage (still being worked out).

  • Storing user uploads for Django applications (e.g. the media folder).

Usage

AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are expected to be set:

$ AWS_ACCESS_KEY_ID=xxxxxxxxxxx
$ AWS_SECRET_ACCESS_KEY=xxxxxxxxxxx

Basic usage:

from s3monkey import S3FS

with S3FS(bucket='media.kennethreitz.com', mount_point='/app/data') as fs:

    # Create a 'test' key on S3, with the contents of 'hello'.
    with open('/app/data/test', 'w') as f:
        f.write('hello')

    # List the keys in the S3 bucket.
    print(os.listdir('/app/data'))
    # ['file1.txt', 'file2.txt', 'file2.txt', 'test', …]

Installation

$ pipenv install s3monkey

This module only supports Python 3.

Release files for s3monkey 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for s3monkey 0.1.1
File Size Uploaded
s3monkey-0.1.1.tar.gz 61.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for s3monkey 0.1.1
File Interpreter ABI Platform
s3monkey-0.1.1-py2.py3-none-any.whl Python 2, Python 3 none any Details

Total release size: 130.3 kB

Release files / s3monkey-0.1.1.tar.gz

Download URL s3monkey-0.1.1.tar.gz
Size 61.1 kB
Tags Source
SHA-256 checksum
How to use checksums
743f3bf8dde13f9894e7a5bf7811f112c8ff54954e065146f842706c332f8dcc
BLAKE2b-256 checksum
How to use checksums
785bee6a1667effb5c5a5eae30f1fef2bf90307d76aeafdf401ca4dc393e1c4e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.44.1 CPython/3.8.2

Release files / s3monkey-0.1.1-py2.py3-none-any.whl

Download URL s3monkey-0.1.1-py2.py3-none-any.whl
Size 69.1 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
696bf06be995421f57fb16a501260249070e68cea7ff6894e65856158f9c99f2
BLAKE2b-256 checksum
How to use checksums
d2cea4f759026b6bb9e4c1b5aa4d5d65dc1ec1cdae319b96f6917db7238db690
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.44.1 CPython/3.8.2

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page