Jean Zay tailored Hydra submitit launcher.
Project description
JZ-hydra-submitit-launcher
A Jean Zay tailored Hydra submitit launcher based on Hydra and its submitit-launcher plugin. Basically it extends the submitit-launcher plugin with defaults that make sense for Jean Zay.
Install
This package can be installed from pypi:
pip install jz-hydra-submitit-launcher
You can also install it from source:
git clone https://github.com/zaccharieramzi/jz-hydra-submitit-launcher.git
cd jz-hydra-submitit-launcher
pip install .
Use
The primary use is with the hydra-submitit-launch
command with your script name and the config type:
hydra-submitit-launch my_app.py dev
Available configs
6 different configs are available:
dev
: with 2 hours, 1 gpu, and qos_gpu-dev.t3
: with 20 hours, 1 gpu, and qos_gpu-t3.t4
: with 100 hours, 1 gpu, and qos_gpu-t4.4gpus_dev
: with 2 hours, 4 gpus, and qos_gpu-dev.4gpus_t3
: with 20 hours, 4 gpus, and qos_gpu-t3.4gpus_t4
: with 100 hours, 4 gpus, and qos_gpu-t4.
By default, all the configs select 32Gb GPUs, use a single node and use the gpu_p1 partition.
Advanced configs
You can override the SLURM config, the same way you would with any hydra configuration.
The parameters you can override are defined in the hydra-submitit-launcher
plugin doc.
For example, if you want to use the gpu_p2 partition, you would need to do:
hydra-submitit-launch my_app.py dev hydra.launcher.setup=null hydra.launcher.partition=gpu_p2
References
- Hydra: https://hydra.cc/docs/intro/
- submitit-launcher: https://hydra.cc/docs/plugins/submitit_launcher/
- submitit: https://github.com/facebookincubator/submitit
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for jz-hydra-submitit-launcher-0.0.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1e8831b4022d15c031a5271bae319c60f4fe30f98ccf175aee51efcda76d37d |
|
MD5 | e761eb3664fff07c1468529924c47863 |
|
BLAKE2b-256 | 32c72bf84a091d5fc882310ace4b9eb7deda31f163f23a100fbc91d44d773f6b |
Hashes for jz_hydra_submitit_launcher-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1460e507e9f52ff8be4ad83b26ea3c80d34480c3613b42a214f935c233d7492c |
|
MD5 | 8ed8fe37c6ebaabe1fb7c61f784c55f2 |
|
BLAKE2b-256 | 2bacd188389923da8ef250fe04c46cb4d1500de1b9d1c7be9b8837b8e0b78f67 |