secbatch 1.2.0

A tool to securely run jobs on Slurm clusters.

SecBatch

A CLI tool to simplify running jobs with encrypted data on a Slurm cluster. This tool automates key generation, data encryption, uploading, Slurm job submission, monitoring, and result retrieval.

Installation

Prerequisites

Before installing, you must have the following command-line tool available in your system's PATH:

• GnuPG (gpg): For encryption and decryption.

You can typically install this using your system's package manager (e.g., apt-get, brew, yum).

Note: Installation under Windows requires different steps described below.

Installing with uv

Once the prerequisites are met, you can install the tool from PyPI using uv:

uv pip install secbatch

This will install the tool and make the secbatch command available in your environment.

Installing with pip

Once the prerequisites are met, you can install secbatch from PyPI using pip in Command Prompt or PowerShell:

pip install secbatch

This will install the tool and its dependencies, making the secbatch command available in your environment.

Usage

After installation, you can use the secbatch command directly, or run it as a module: python -m secbatch.

Full Command Reference

secbatch --host <your-host> <command> [options]

Global Arguments:

• --host: The SSH hostname or an alias from your ~/.ssh/config.
• -u, --user: (Optional) Your SSH username. Overrides the user in ~/.ssh/config.
• --ssh-key-file: (Optional) Path to your SSH private key. Overrides the IdentityFile in ~/.ssh/config.

Commands:

• run: Execute the full secure workflow.

  • -i, --input-dir: The local directory containing your input data.
  • -e, --exec: The command to execute on the compute node.
  • -s, --script-file: Path to a local script file to execute.
  • -o, --output-dir: The local directory for results.
  • -n, --job-name: (Optional) A name for the Slurm job.
  • --keep-local-temp: Do not delete the local temporary directory after the workflow completes.
  • -p, --param: (Optional) Additional parameters to pass to sbatch. Can be used multiple times.
  • ... (and other Slurm parameters like --nodes, --time, etc.)

• test-connection: Test the SSH connection to the host.

• test-job: Submit a predefined example job to test the full workflow.

Example: Running a Job

secbatch --host my-cluster run \
    -i ./my-data \
    -e 'grep "sensitive" input/patient.txt > processed.txt' \
    -o ./results

::include{file=TUTORIAL_WINDOWS.md}