Skip to main content

A tool for pulling singularity images in HPC environments.

Project description

Application to populate the singularity cache. Useful for HPC clusters where multiple nodes may pull to the same cache at the same time if images are not yet present.


prepull-singularity is tested on python 3.5, 3.6, 3.7 and 3.8. It can be installed with:

pip install prepull-singularity

How to run

  1. Make a YAML file listing the images to be pulled.
alpine: "alpine:latest"
debian: "debian:stretch-slim"
  1. Run the program prepull-singularity images.yml


usage: prepull-singularity [-h] [-a MAX_ATTEMPTS] [-p PREFIX]
                           [--stop-on-failure] [--show-output-on-failure]
                           [--singularity-exe SINGULARITY_EXE] [--use-digest]

Pull images from listed in a YAML file, so they get cached and can be run
without pulling later.

positional arguments:
  input                 A YAML file listing the images to be pulled, either as
                        a map or list.

optional arguments:
  -h, --help            show this help message and exit
  -a MAX_ATTEMPTS, --max-attempts MAX_ATTEMPTS
                        Maximum number of times to attempt pulling each image;
                        defaults to 3.
  -p PREFIX, --prefix PREFIX
                        Prefix for the image url; defaults to 'docker://'.
  --stop-on-failure     Stop when pulling an image fails; by default all
                        images will be attempted to be pull even if one fails.
                        Print the stderr and stdout when pulling an image
                        Print the stderr and stdout when pulling an image
  --singularity-exe SINGULARITY_EXE
                        The command for running singularity; defaults to
  --use-digest          Retrieve the image digestes from dockerhub or
                        and use those (instead of the tags) to pull the
                        images. Only usable with docker images.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

prepull-singularity-0.2.0.tar.gz (5.7 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page