Skip to main content

parallel-ssh

Asynchronous parallel SSH client library.

Run SSH commands over many - hundreds/hundreds of thousands - number of servers asynchronously and with minimal system load on the client host.

Native code based clients with extremely high performance, making use of C libraries.

License Latest Version https://circleci.com/gh/ParallelSSH/parallel-ssh/tree/master.svg?style=svg https://codecov.io/gh/ParallelSSH/parallel-ssh/branch/master/graph/badge.svg https://img.shields.io/pypi/wheel/parallel-ssh.svg Latest documentation

Installation

pip install parallel-ssh

An update to pip may be needed to be able to install binary wheels.

pip install -U pip
pip install parallel-ssh

Usage Example

See documentation on read the docs for more complete examples.

Run uname on two hosts in parallel.

from pssh.clients import ParallelSSHClient

hosts = ['localhost', 'localhost']
client = ParallelSSHClient(hosts)

output = client.run_command('uname')
for host_output in output:
    for line in host_output.stdout:
        print(line)
    exit_code = host_output.exit_code
Output:
Linux
Linux

Single Host Client

Single host client with similar API can be used if parallel functionality is not needed.

from pssh.clients import SSHClient

host = 'localhost'
cmd = 'uname'
client = SSHClient(host)

host_out = client.run_command(cmd)
for line in host_out.stdout:
    print(line)
exit_code = host_out.exit_code

Questions And Discussion

Github discussions can be used to discuss, ask questions and share ideas regarding the use of parallel-ssh.

Native clients

The default client in parallel-ssh is a native client based on ssh2-python - libssh2 C library - which offers much greater performance and reduced overhead compared to other Python SSH libraries.

See this post for a performance comparison of different Python SSH libraries.

Alternative clients based on ssh-python (libssh) are also available under pssh.clients.ssh. See client documentation for a feature comparison of the available clients in the library.

parallel-ssh makes use of clients and an event loop solely based on C libraries providing native code levels of performance and stability with an easy to use Python API.

Native Code Client Features

  • Highest performance and least overhead of any Python SSH library

  • Thread safe - makes use of native threads for CPU bound calls like authentication

  • Natively asynchronous utilising C libraries implementing the SSH protocol

  • Significantly reduced overhead in CPU and memory usage

Why This Library

Because other options are either immature, unstable, lacking in performance or all of the aforementioned.

Certain other self-proclaimed leading Python SSH libraries leave a lot to be desired from a performance and stability point of view, as well as suffering from a lack of maintenance with hundreds of open issues, unresolved pull requests and inherent design flaws.

The SSH libraries parallel-ssh uses are, on the other hand, long standing mature C libraries in libssh2 and libssh that have been in production use for decades and are part of some of the most widely distributed software available today - Git itself, OpenSSH, Curl and many others.

These low level libraries are far better placed to provide the maturity, stability and performance needed from an SSH client for production use.

parallel-ssh provides easy to use SSH clients that hide the complexity, while offering stability and native code levels of performance and as well as the ability to scale to hundreds or more concurrent hosts.

See alternatives for a more complete comparison of alternative SSH libraries, as well as performance comparisons mentioned previously.

Waiting for Completion and Exit Codes

The client’s join function can be used to wait for all commands in output to finish.

After join returns, commands have finished and all output can be read without blocking.

Once either standard output is iterated on to completion, or client.join() is called, exit codes become available in host output.

Iteration ends only when remote command has completed, though it may be interrupted and resumed at any point - see join and output timeouts documentation.

HostOutput.exit_code is a dynamic property and will return None when exit code is not ready, meaning command has not finished, or unavailable due to error.

Once all output has been gathered exit codes become available even without calling join as per previous examples.

output = client.run_command('uname')

client.join()

for host_out in output:
    for line in host_out.stdout:
        print(line)
    print(host_out.exit_code)
Output:
Linux
0
Linux
0

Similarly, exit codes are available after client.join() without reading output.

output = client.run_command('uname')

client.join()

for host_output in output:
    print(host_out.exit_code)
Output:
0
0

Built in Host Output Logger

There is also a built in host logger that can be enabled to log output from remote hosts for both stdout and stderr. The helper function pssh.utils.enable_host_logger will enable host logging to stdout.

To log output without having to iterate over output generators, the consume_output flag must be enabled - for example:

from pssh.utils import enable_host_logger

enable_host_logger()
client.run_command('uname')
client.join(consume_output=True)
Output:
[localhost]       Linux

SCP

SCP is supported - native client only - and provides the best performance for file copying.

Unlike with the SFTP functionality, remote files that already exist are not overwritten and an exception is raised instead.

Note that enabling recursion with SCP requires server SFTP support for creating remote directories.

To copy a local file to remote hosts in parallel with SCP:

from pssh.clients import ParallelSSHClient
from gevent import joinall

hosts = ['myhost1', 'myhost2']
client = ParallelSSHClient(hosts)
cmds = client.scp_send('../test', 'test_dir/test')
joinall(cmds, raise_error=True)

See SFTP and SCP documentation for more examples.

SFTP

SFTP is supported in the native client.

To copy a local file to remote hosts in parallel:

from pssh.clients import ParallelSSHClient
from pssh.utils import enable_logger, logger
from gevent import joinall

enable_logger(logger)
hosts = ['myhost1', 'myhost2']
client = ParallelSSHClient(hosts)
cmds = client.copy_file('../test', 'test_dir/test')
joinall(cmds, raise_error=True)
Output:
Copied local file ../test to remote destination myhost1:test_dir/test
Copied local file ../test to remote destination myhost2:test_dir/test

There is similar capability to copy remote files to local ones with configurable file names via the copy_remote_file function.

In addition, per-host configurable file name functionality is provided for both SFTP and SCP - see documentation.

Directory recursion is supported in both cases via the recurse parameter - defaults to off.

See SFTP and SCP documentation for more examples.

https://ga-beacon.appspot.com/UA-9132694-7/parallel-ssh/README.rst?pixel

Metadata

Release files for parallel-ssh 2.8.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for parallel-ssh 2.8.0
File Size Uploaded Provenance Actions
parallel-ssh-2.8.0.tar.gz 66.4 kB Without provenance Details for parallel-ssh-2.8.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for parallel-ssh 2.8.0
File Interpreter ABI Platform Actions
parallel_ssh-2.8.0-py3-none-any.whl Python 3 none any Details for parallel_ssh-2.8.0-py3-none-any.whl

Total release size: 149.3 kB

Release files / parallel-ssh-2.8.0.tar.gz

Download URL parallel-ssh-2.8.0.tar.gz
Size 66.4 kB
Tags Source
SHA-256 checksum
How to use checksums
b6492599d278a51f9808e63ddbc484246f1db407e8ccea19165fe12a99c817ab
BLAKE2b-256 checksum
How to use checksums
60ad13f9cf1a952c127f96256c5e49d43a674626d2d55e62d695257acd535168
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

Release files / parallel_ssh-2.8.0-py3-none-any.whl

Download URL parallel_ssh-2.8.0-py3-none-any.whl
Size 82.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
056664a7cec91a494004ba54c6e6d8c7522c81fc43f2daf4b32d2fc5cb3ebbf0
BLAKE2b-256 checksum
How to use checksums
14a9363f99827db642855c55cd7e1d5baf1412c914e8036bf89614edab9ba676
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

Security for parallel-ssh 2.8.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

2.8.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page