Skip to main content

Used for file transfer and sending predefined commands to an hpc cluster

Project description

hpc-interact

Used for scripting file transfer (sftp) and sending commands (ssh) to hpc - a wrapper for expect

Requires expect

If needed, install via:

$ sudo apt install expect

Install

From PyPI:

pip install hpc-interact

Or just clone the repo and use hpc_interact.py directly.

Usage

Instantiate a Scripter object:

>>> # Connect via sftp
>>> from hpc_interact import Scripter
>>> scripter = Scripter(config="./login_config", site='somewhere.uni.edu', mode='sftp')
>>> # NOTE: If "./login_config" doesn't exist, you'll be prompted to create it

Prepare any steps to undertake

>>> outdir = "/Users/me/somedir"
>>> scripter.cwd("/some/remote/directory")
>>> for file in ["file1.txt","file2.txt"]:
...     scripter.get(file,outdir)

To see what steps you've added, run:

>>> scripter.preview_steps()

Output:

Command preview:
(0, 'cd /some/remote/directory\n')
(1, 'mkdir /Users/me/somedir\n')
(2, 'cd /Users/me/somedir\n')
(3, 'put file1.txt \n\n')
(4, 'mkdir /Users/me/somedir\n')
(5, 'cd /Users/me/somedir\n')
(6, 'put file2.txt \n\n')

To run:

>>> scripter.run()

Forgot something but don't want to create a new Scripter object?

>>> scripter.clear()
>>> scripter.put("/Users/me/another-dir/cool_script.sh",outdir="/some/remote/directory",new_name="coolest_script_on_the_hpc.sh")
>>> scripter.run()

Run a script on the cluster

>>> # Connect via ssh
>>> from hpc_interact import cluster.Scripter
>>> scripter = Scripter(config="./login_config", site='somewhere.uni.edu', mode='ssh')
>>> 
>>> # run it
>>> scripter.basic_step("bash","/some/remote/directory/coolest_script_on_the_hpc.sh")
>>> scripter.run()

Started as ssh but want need to transfer files?

>>> scripter.reset_mode("sftp")
>>> add_some_steps(scripter)
>>> scripter.run()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hpc_interact-0.1.1.tar.gz (8.1 kB view details)

Uploaded Source

Built Distribution

hpc_interact-0.1.1-py3-none-any.whl (8.8 kB view details)

Uploaded Python 3

File details

Details for the file hpc_interact-0.1.1.tar.gz.

File metadata

  • Download URL: hpc_interact-0.1.1.tar.gz
  • Upload date:
  • Size: 8.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.0 CPython/3.8.16 Linux/4.4.0-22621-Microsoft

File hashes

Hashes for hpc_interact-0.1.1.tar.gz
Algorithm Hash digest
SHA256 c9f20161d826badb0f59b8517412d851cc0432b687f03066eab88f75bf6e9dab
MD5 24734e4704a08c5e1deca3d9552990f2
BLAKE2b-256 91099c224ae40f5f541b2ecf2bddc9337f96957e4ab4aecf49ef92071035684d

See more details on using hashes here.

File details

Details for the file hpc_interact-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: hpc_interact-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 8.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.0 CPython/3.8.16 Linux/4.4.0-22621-Microsoft

File hashes

Hashes for hpc_interact-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 87143a371871fb31259527474e6c6eb3b2669b86df2882733f29aeacaa927522
MD5 847087239e19f350054ffaa45ba67c0c
BLAKE2b-256 aa095c483d4accc32cccddf521ae05a8a08fda4329909ab612e85211dd95b355

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page