Skip to main content

ultralytics-disk-cache-hook

Keep Ultralytics disk caches off the dataset filesystem.

When training uses cache="disk", this package redirects image .npy files and dataset .cache files into a local cache directory on the training node. This is useful when datasets live on shared or network storage.

中文说明

Install

pip install ultralytics-disk-cache-hook

The hook auto-enables itself in new Python processes after installation. It supports Ultralytics 8.4.0 through 8.4.84.

Use

Use Ultralytics normally. Set cache="disk" when you want disk caching:

from ultralytics import YOLO

YOLO("yolov8n.pt").train(data="coco128.yaml", cache="disk")

The original dataset directory remains unchanged. Cache files are written under the plugin cache root instead.

Configuration

Setting Default Purpose
ULTRALYTICS_DISK_CACHE_TMPDIR System temporary directory Parent directory for the plugin cache
ULTRALYTICS_IMAGE_DISK_CACHE 1 Enable image .npy cache redirection
ULTRALYTICS_DATASET_META_CACHE 1 Enable dataset .cache redirection
ULTRALYTICS_FORCE_DISK_CACHE 0 Force BaseDataset cache arguments to disk

For example, to use local NVMe storage:

export ULTRALYTICS_DISK_CACHE_TMPDIR=/local_nvme/tmp

Parameter forcing is off by default so normal Ultralytics cache behavior is preserved. In a managed training environment, enable it before Python starts:

export ULTRALYTICS_FORCE_DISK_CACHE=1

This changes BaseDataset cache values such as false, true, and "ram" to "disk". It does not cover classification datasets.

Slurm job-local temporary storage

This plugin works well with Slurm job-lifecycle-managed local temporary storage, such as job_container/tmpfs or a site-specific Prolog/Epilog setup. Those mechanisms create private local scratch space for a job and remove its contents when the job ends.

If the job-local directory is mounted as /tmp, no extra setting is needed because that is the default cache parent. If your cluster exposes another job-local path, point the plugin at it:

export ULTRALYTICS_DISK_CACHE_TMPDIR=/path/to/job-local-scratch

Each job then keeps cache files off the shared dataset filesystem and they are reclaimed with the job-local temporary storage.

Cache layout

The default cache root is:

<system-temp-dir>/ultralytics-disk-cache

Each source path is hashed, then stored in two hash-bucket directories. Image caches use .npy and dataset metadata uses .cache.

Notes

  • The plugin patches Ultralytics internals and rejects unsupported versions.
  • Disk-space checks are skipped because the cache root may be outside the dataset filesystem. Ensure the selected cache disk has enough free space.
  • The upstream proposal for native configurable disk-cache paths is Add cache_dir support for disk image caching. Follow its progress if you need first-party support.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ultralytics_disk_cache_hook-0.3.0.tar.gz (8.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ultralytics_disk_cache_hook-0.3.0-py3-none-any.whl (9.3 kB view details)

Uploaded Python 3

File details

Details for the file ultralytics_disk_cache_hook-0.3.0.tar.gz.

File metadata

File hashes

Hashes for ultralytics_disk_cache_hook-0.3.0.tar.gz
Algorithm Hash digest
SHA256 15398d9d3c356cc63bba54a03ac132cc64ec0c6078faf2dad9f5b85301fb00c8
MD5 7269e22265b9231c1a7f3218af677567
BLAKE2b-256 2bc5adfc737a91af0c97ef92f3187d6193d6171d0b1dc24c5edf7373c9f7e04d

See more details on using hashes here.

Provenance

The following attestation bundles were made for ultralytics_disk_cache_hook-0.3.0.tar.gz:

Publisher: publish.yml on xx025/ultralytics-disk-cache-hook

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ultralytics_disk_cache_hook-0.3.0-py3-none-any.whl.

File metadata

File hashes

Hashes for ultralytics_disk_cache_hook-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 8c832d84fe87b61cc12b63de085c0712508a9c955c5be3da8be77b6bca6db6f6
MD5 56270e721f9359bb2580c8b1d734fd16
BLAKE2b-256 0060006a6e420f902b9c4d6e5b57444115947ed82332f873af289b67a115e4af

See more details on using hashes here.

Provenance

The following attestation bundles were made for ultralytics_disk_cache_hook-0.3.0-py3-none-any.whl:

Publisher: publish.yml on xx025/ultralytics-disk-cache-hook

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.3.0 This release

2 files

0.2.1

2 files

0.2.0

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page