Python script to submit torque jobs in a very hackish way
Project description
This is as very simple hackish hack, that allows you to any serializable python callable as a torque job.
Features:
- Allows you to send enviorment on the other side
- Allows to execute any callable function as a torque task
- Allows you to launch a bunch of taska as a torque array job.
Dependencies:
- Working python 2.7/3.4 enviorment
- Some python dependencies (see REQUIREMENTS)
- Working qsub command (no need for other PBS/torque stuff).
Works by serializing the enviorment and callable function to the enviorment variables (or enviorment variables and files). For serialization we use dill if avilable or pickle.
Enviorment can be initialized using arbirtary bash script — this script will be sourced before running provided python callable.
Example
Example without enviorment:
callable = partial(print, "Hello World!") from torqsubmit import Submitter s = Submitter() s.tasks = [callable] s.submit()
Example with enviorment:
callable = partial(print, "Hello World!") from torqsubmit._submit import Submitter ENV = """ source ${HOME}/.bashrc workon torque-submit export MSG="Hello World!" """ def print_from_env(): import os print(os.environ["MSG"]) s = Submitter() s.tasks = [callable] s.enviorment = ENV s.submit()
Example submitting many tasks:
from __future__ import print_function from functools import partial callable = partial(print, "Hello World!") from torqsubmit._submit import Submitter ENV = """ source ${HOME}/.bashrc workon torque-submit export MSG="Hello World!" """ def print_from_env(): import os print(os.environ["MSG"]) s = Submitter() s.enviorment = ENV s.tasks = [print_from_env, print_from_env] s.submit()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size torque-submit-0.0.3.tar.gz (5.7 kB) | File type Source | Python version None | Upload date | Hashes View |