Skip to main content

gpumutex

This library can be injected into existing CUDA programs with GPUs, such that no process also run under libgpumutex can interfere with another.

This is accomplished through relatively interesting means. See the design writeup: https://apaz.dev/blog/LD_PRELOAD.html

Linux only (glibc; uses dl_iterate_phdr/ELF parsing internally).

Installing with pip

pip install gpumutex
GPUMUTEX_VISIBLE_DEVICES=0,1,2,3 gpumutex-run -- python train.py
# or, equivalently:
gpumutex-run --visible-devices 0,1,2,3 -- python train.py
import gpumutex

gpumutex.run(["python", "train.py"], visible_devices="0,1,2,3", check=True)
gpumutex.system("python train.py", visible_devices=[0, 1, 2, 3])

Using Standalone

# Compile
cc -shared -fPIC -fvisibility=hidden -o libgpumutex.so gpumutex/libgpumutex.c -ldl -lpthread

Then to run a program under it:

LD_PRELOAD=./libgpumutex.so GPUMUTEX_VISIBLE_DEVICES=... program program_args

Configuration

Both usages below are configured the same way as the raw interposer, via environment variables. gpumutex-run and gpumutex.run()/gpumutex.system() also accept these directly as CLI flags / keyword arguments, which set the variable in the child's environment only, without touching your own:

Variable CLI flag Python kwarg Description Default
GPUMUTEX_VISIBLE_DEVICES --visible-devices visible_devices Comma-separated device indices to manage (e.g. "0,2,3"), or an iterable of indices in Python. Required; without it, the preload is inert. none (required)
GPUMUTEX_NUM_DEVICES --num-devices num_devices How many of those devices one process tree needs at once. 1
GPUMUTEX_LOCK_DIR --lock-dir lock_dir Where lockfiles and the wait queue live. /tmp/libgpumutex

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gpumutex-0.2.1.tar.gz (33.5 kB view details)

Uploaded Source

File details

Details for the file gpumutex-0.2.1.tar.gz.

File metadata

  • Download URL: gpumutex-0.2.1.tar.gz
  • Upload date:
  • Size: 33.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for gpumutex-0.2.1.tar.gz
Algorithm Hash digest
SHA256 22c0d1a54a58679056ed666ef650563465dd3f7be687a014cd00a8afac0e2f56
MD5 cca4cb1446ee5f1e62b89ebd7829040a
BLAKE2b-256 fe2f3997fcce4f5b8492edae080ec01e1175c0dfe7eb0559ff8f2073d148b093

See more details on using hashes here.

Provenance

The following attestation bundles were made for gpumutex-0.2.1.tar.gz:

Publisher: publish.yml on apaz-cli/libgpumutex

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.2.1 This release

1 file

0.1.1

1 file

0.1.0

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page