Skip to main content

Used for file transfer and sending predefined commands to an hpc cluster

Project description

hpc-interact

Used for scripting file transfer (sftp) and sending commands (ssh) to hpc - a wrapper for expect

Requires expect

If needed, install via:

$ sudo apt install expect

Install

From PyPI:

pip install hpc-interact

Or just clone the repo and use hpc_interact.py directly.

Usage

Instantiate a Scripter object:

>>> # Connect via sftp
>>> from hpc_interact import Scripter
>>> scripter = Scripter(config="./login_config", site='somewhere.uni.edu', mode='sftp')
>>> # NOTE: If "./login_config" doesn't exist, you'll be prompted to create it

Prepare any steps to undertake

>>> outdir = "/Users/me/somedir"
>>> scripter.cwd("/some/remote/directory")
>>> for file in ["file1.txt","file2.txt"]:
...     scripter.get(file,outdir)

To see what steps you've added, run:

>>> scripter.preview_steps()

Output:

Command preview:
(0, 'cd /some/remote/directory\n')
(1, 'mkdir /Users/me/somedir\n')
(2, 'cd /Users/me/somedir\n')
(3, 'put file1.txt \n\n')
(4, 'mkdir /Users/me/somedir\n')
(5, 'cd /Users/me/somedir\n')
(6, 'put file2.txt \n\n')

To run:

>>> scripter.run()

Forgot something but don't want to create a new Scripter object?

>>> scripter.clear()
>>> scripter.put("/Users/me/another-dir/cool_script.sh",outdir="/some/remote/directory",new_name="coolest_script_on_the_hpc.sh")
>>> scripter.run()

Run a script on the cluster

>>> # Connect via ssh
>>> from hpc_interact import cluster.Scripter
>>> scripter = Scripter(config="./login_config", site='somewhere.uni.edu', mode='ssh')
>>> 
>>> # run it
>>> scripter.basic_step("bash","/some/remote/directory/coolest_script_on_the_hpc.sh")
>>> scripter.run()

Started as ssh but want need to transfer files?

>>> scripter.reset_mode("sftp")
>>> add_some_steps(scripter)
>>> scripter.run()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hpc_interact-0.1.1.tar.gz (8.1 kB view hashes)

Uploaded Source

Built Distribution

hpc_interact-0.1.1-py3-none-any.whl (8.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page