Jean Zay tailored Hydra submitit launcher.
Project description
JZ-hydra-submitit-launcher
A Jean Zay (JZ) tailored Hydra submitit launcher based on Hydra and its submitit-launcher plugin. Basically it extends the submitit-launcher plugin with defaults that make sense for JZ.
Install
This package can be installed from pypi (using --user
is required on JZ):
pip install --user jz-hydra-submitit-launcher
You can also install it from source:
git clone https://github.com/zaccharieramzi/jz-hydra-submitit-launcher.git
cd jz-hydra-submitit-launcher
pip install --user .
The main command is then installed in your local binaries.
You need to add them to your path in order for the command to be found (and avoid the error -bash: hydra-submitit-launch: command not found
):
echo "export PATH=$PATH:$HOME/.local/bin" >> $HOME/.bashrc
Don't forget to source $HOME/.bashrc
before using the command.
Use
The primary use is with the hydra-submitit-launch
command with your script name and the config type:
hydra-submitit-launch my_app.py dev
Available configs
6 different configs are available:
dev
: with 2 hours, 1 gpu, and qos_gpu-dev.t3
: with 20 hours, 1 gpu, and qos_gpu-t3.t4
: with 100 hours, 1 gpu, and qos_gpu-t4.4gpus_dev
: with 2 hours, 4 gpus, and qos_gpu-dev.4gpus_t3
: with 20 hours, 4 gpus, and qos_gpu-t3.4gpus_t4
: with 100 hours, 4 gpus, and qos_gpu-t4.
By default, all the configs select 32Gb GPUs, use a single node and use the gpu_p1 partition.
Advanced configs
You can override the SLURM config, the same way you would with any hydra configuration.
The parameters you can override are defined in the hydra-submitit-launcher
plugin doc.
For example, if you want to use the gpu_p2 partition, you would need to do:
hydra-submitit-launch my_app.py dev hydra.launcher.setup=null hydra.launcher.partition=gpu_p2 hydra.launcher.cpus_per_task=3
In order to change the timeout on the SLURM job to for example 10 hours, you would need to do:
hydra-submitit-launch my_app.py base +hours=10
This will automatically select the right qos for you (and if you want to force a certain qos, you can always add hydra.launcher.qos=<your-qos>
).
Logs
Your SLURM logs are stored in the following path: launch-dir/multirun/day-of-launch/time-of-launch/.submitit/job_array_task_id/
This is the default from hydra
and submitit
and this can be changed using both the submitit
plugin parameters (see here) and the hydra
job configurations (see here).
References
- Hydra: https://hydra.cc/docs/intro/
- submitit-launcher: https://hydra.cc/docs/plugins/submitit_launcher/
- submitit: https://github.com/facebookincubator/submitit
- JZ docs: http://www.idris.fr/ or https://jean-zay-doc.readthedocs.io/en/latest/
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for jz-hydra-submitit-launcher-0.1.5.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 45153115a5f3e6410c13908b95e2daabd27040beacde5e31dde2476036dea643 |
|
MD5 | cb0848180c25519b9feb542a10e50006 |
|
BLAKE2b-256 | 89e873ba3299ffb0f989a58e9d5b8ef29097dc572cf49c50ebd7828a13549789 |
Hashes for jz_hydra_submitit_launcher-0.1.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6d2a7fcc20e84177b361bff4e5a6806a29366cf36fe62a1bfd2b0552270570a |
|
MD5 | b0f49b742654639520394f077a4ba93a |
|
BLAKE2b-256 | 487143b8cdf6eefddaed5b9adbe955f190586d1f449a87065c18a1dc678c25a9 |