Skip to main content

Smart dispatch between pathlib.Path and s3path.S3Path. It was borrowed from the project by EMTF:

Project description


Dispatch to pathlib.Path if a local file system path is given, or s3path.S3Path if the given path is prefixed with s3://



from smart_path import smart_path
from pathlib import Path
from s3path import S3Path

# A local path on file system
path_local = smart_path("/tmp/t")
assert isinstance(path_local, Path)

# A s3 path on oss
path_brainpp_oss = smart_path("s3://public-datasets-contrib/", endpoint_url='')
assert isinstance(path_local, S3Path)

Stub Mode

from smart_path import smart_path

# Stub mode works like the symbolic link in Unix. The source path can be either
# on local file system or brain++ oss, so do the target path. Stub Mode is
# excepted to be helpful when we need to store a large file on oss and pretend 
# it is on local file system.
# In stub mode, smart_path writes to two destinations:
# 1. The string "hello-world" is written to `s3://somewhere/on/oss`
# 2. The path "s3://somewhere/on/oss" is written to `corresponding/local/path`
path = smart_path('s3://somewhere/on/oss', stub='corresponding/local/path')
# smart_path knows it is a stub file
stub = smart_path('corresponding/local/path')
print(stub.read_text())  # prints "hello-world"


pip install -e git://
# Or from PyPI
pip install smart_path


The entire project is borrowed from EMTF: EMTF-at-megvii-dot-com.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

smart_path-1.0.1.tar.gz (3.4 kB view hashes)

Uploaded source

Built Distribution

smart_path-1.0.1-py3-none-any.whl (3.7 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page