cattensors

Faster safetensors loading for JuiceFS.

Note: This library was developed and benchmarked against JuiceFS, but in theory should work with any network backed distributed filesystem.

cattensors monkey-patches safetensors to pre-warm the OS page cache before tensor reads. On network backed filesystems, the first read of a large model file is slow. cattensors reads the file into page cache using parallel threads so subsequent mmap based access hits RAM instead of the network.

Compatibility

• POSIX platforms (Linux/macOS)
• Not supported on Windows (fcntl.flock is used for cross-process coordination)

Installation

pip install git+https://github.com/badayvedat/cattensors

Usage

Installing or importing cattensors does not patch anything. Call patch() explicitly in each process that loads models:

import cattensors

cattensors.patch()

from safetensors.torch import load_file

Call it before importing libraries such as diffusers or transformers that may cache references to safetensors functions. Importing torch first is fine; existing from safetensors import safe_open or from safetensors.torch import load_file references cannot be updated by patching later. Calling patch() before any torch/safetensors-related imports is the simplest safe convention.

Flashpack patching is separate and opt-in:

from cattensors.flashpack_patch import patch as patch_flashpack

patch_flashpack()

Call this before importing code that caches flashpack reader functions.

No startup .pth file is installed, and CATTENSORS_AUTO_PATCH is no longer supported. When upgrading an older installation, ensure its cattensors.pth has been removed from site-packages (a normal package upgrade should remove it).

Configuration

Environment variables:

Flags are read when used, so changes affect subsequent load calls.