modeldownloaderutil 1.0.5

Download model files from local paths, HTTP, S3-compatible storage, GCS, and Git LFS.

model-downloader-util

Download or resolve model files from local paths, HTTP(S), S3-compatible storage (S3, MinIO, RustFS), Google Cloud Storage, and Git LFS.

Install

pip install modeldownloaderutil

From source:

uv sync

Usage

from modeldownloaderutil import download_model, cache_dir

path = download_model("s3://my-bucket/models/weights.onnx")
path = download_model("https://example.com/model.onnx", force_download=True)

Supported sources

Scheme	Example
Local	/path/to/model.onnx, ~/models/x.onnx
HTTP(S)	https://host/path/model.onnx
S3	s3://bucket/key
MinIO	minio://bucket/key (MINIO_ENDPOINT)
RustFS	rustfs://bucket/key (RUSTFS_ENDPOINT)
GCS	gs://bucket/object
Git LFS	git+https://github.com/org/repo.git#path/in/repo.onnx

Cache

Default: platformdirs cache for model_registry (override with MODEL_CACHE_DIR).

<cache>/
  s3/<bucket>/<key>
  gs/<bucket>/<object>
  url/<host>/<path>
  git/<repo_slug>/<file_path>
  git/<repo_slug>.lock   # serializes parallel Git / Git LFS pulls for the same repo

Parallel download_model calls for different files in the same Git repo are safe: they take a per-repo file lock, then skip work if another process already cached the file.

Environment

Copy .env.example to .env in the project root (loaded automatically on download_model).

Variable	Purpose
MODEL_CACHE_DIR	Download cache root (cache_dir())
RUSTFS_ENDPOINT	Required for rustfs:// URLs
MINIO_ENDPOINT	Required for minio:// URLs
MODEL_ACCESS_KEY / MODEL_SECRET_KEY	S3-compatible credentials

Development

make test
make build

Publish to PyPI

Create an API token at pypi.org (scope: project modeldownloaderutil or entire account).

export UV_PUBLISH_TOKEN=pypi-xxxxxxxx
make publish

Dry run on TestPyPI first:

export UV_PUBLISH_TOKEN=pypi-xxxxxxxx
make publish-test
pip install --index-url https://test.pypi.org/simple/ modeldownloaderutil