Skip to main content

No project description provided

Project description

cached-path

A file utility library that provides a unified, simple interface for accessing both local and remote files. This can be used behind other APIs that need to access files agnostic to where they are located.

CI PyPI Documentation Status License

Quick links

Installation

cached-path requires Python 3.7 or later.

Installing with pip

cached-path is available on PyPI. Just run

pip install cached-path

Installing from source

To install cached-path from source, first clone the repository:

git clone https://github.com/allenai/cached_path.git
cd cached_path

Then run

pip install -e .

Usage

from cached_path import cached_path

Given something that might be a URL or local path, cached_path() determines which. If it's a remote resource, it downloads the file and caches it to the cache directory, and then returns the path to the cached file. If it's already a local path, it makes sure the file exists and returns the path.

For URLs, http://, https://, s3:// (AWS S3), gs:// (Google Cloud Storage), and hf:// (HuggingFace Hub) are all supported out-of-the-box.

For example, to download the PyTorch weights for the model epwalsh/bert-xsmall-dummy on HuggingFace, you could do:

cached_path("hf://epwalsh/bert-xsmall-dummy/pytorch_model.bin")

For paths or URLs that point to a tarfile or zipfile, you can also add a path to a specific file to the url_or_filename preceeded by a "!", and the archive will be automatically extracted (provided you set extract_archive to True), returning the local path to the specific file. For example:

cached_path("model.tar.gz!weights.th", extract_archive=True)

Cache directory

By default the cache directory is ~/.cache/cached_path/, however there are several ways to override this setting:

  • set the environment variable CACHED_PATH_CACHE_ROOT,
  • call set_cache_dir(), or
  • set the cache_dir argument each time you call cached_path().

Team

cached-path is developed and maintained by the AllenNLP team, backed by the Allen Institute for Artificial Intelligence (AI2). AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering. To learn more about who specifically contributed to this codebase, see our contributors page.

License

cached-path is licensed under Apache 2.0. A full copy of the license can be found on GitHub.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cached_path-1.1.1.tar.gz (23.4 kB view details)

Uploaded Source

Built Distribution

cached_path-1.1.1-py3-none-any.whl (26.8 kB view details)

Uploaded Python 3

File details

Details for the file cached_path-1.1.1.tar.gz.

File metadata

  • Download URL: cached_path-1.1.1.tar.gz
  • Upload date:
  • Size: 23.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.12

File hashes

Hashes for cached_path-1.1.1.tar.gz
Algorithm Hash digest
SHA256 96a635290b71ae7e1b1bf17ae1b9005f6f107c123a8009bbaffa85ca3ec04516
MD5 0c90456c91e2623f8d4684e8a7461056
BLAKE2b-256 4374b00b41c2bb284061ffcd4f228775bfb6438444505b79e5ac6c4b7b2277d1

See more details on using hashes here.

File details

Details for the file cached_path-1.1.1-py3-none-any.whl.

File metadata

  • Download URL: cached_path-1.1.1-py3-none-any.whl
  • Upload date:
  • Size: 26.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.12

File hashes

Hashes for cached_path-1.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 b5c73d412d0351115f6d7cae462bb5a506678b7584d1ca96d093b8f7e426d242
MD5 a3a888bf86bf34aa943d012133eab364
BLAKE2b-256 296a488fe2307bb32ddb2e2d191d44c9859f3a5b0bfc30282691b0d0869eff66

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page