Skip to main content

Pathlib API extended to use fsspec backends

Project description

Universal Pathlib

Universal Pathlib is a python library that aims to extend Python's built-in pathlib.Path api to use a variety of backend filesystems using fsspec

Installation

Pypi

pip install universal_pathlib

conda

conda install -c conda-forge universal_pathlib

Basic Usage

>>> from upath import UPath
>>> import s3fs

>>> s3path = UPath("s3://test_bucket") / "example.txt"
>>> s3path.name
example.txt
>>> s3path.stem
example
>>> s3path.suffix
.txt
>>> s3path.exists()
True
>>> s3path.read_text()
'Hello World'

For more examples, see the example notebook here

Currently supported filesystems (and schemes)

  • file: Local filessystem
  • memory: Ephemeral filesystem in RAM
  • az:, adl: and abfs: Azure Storage (requires adlfs to be installed)
  • http: and https: HTTP(S)-based filesystem
  • hdfs: Hadoop distributed filesystem
  • gs: and gcs: Google Cloud Storage (requires gcsfs to be installed)
  • s3: and s3a: AWS S3 (requires s3fs to be installed)
  • webdav+http: and webdav+https: WebDAV-based filesystem on top of HTTP(S) (requires webdav4[fsspec] to be installed)

Other fsspec-compatible filesystems may also work, but are not supported and tested. Contributions for new filesystems are welcome!

License

MIT License

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

universal_pathlib-0.0.21.tar.gz (21.9 kB view hashes)

Uploaded source

Built Distribution

universal_pathlib-0.0.21-py3-none-any.whl (25.7 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page