shpyx 0.1.0

shpyx is a simple, lightweight and typed library for running shell commands in Python.

Use shpyx.run to run a shell command in a subprocess:

>>> import shpyx
>>> shpyx.run("echo 1")
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)
>>> shpyx.run("echo 1").return_code
0
>>> shpyx.run("echo 1").stdout
'1\n'
>>> shpyx.run("echo 1").stderr
''

Installation

Install with pip:

pip install shpyx

How Tos

Run a command

In string format:

>>> shpyx.run("echo 1")
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)

In list format:

>>> shpyx.run(["echo", "1"])
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)

Run a command and print live output

>>> shpyx.run("echo 1", log_output=True)
1
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)

Run a command with shell specific logic

When the argument to run is a string, an actual shell is created in the subprocess and shell logic can be used. For example, the pipe operator can be used in bash/sh:

>>> shpyx.run("seq 1 5 | grep '2'")
ShellCmdResult(cmd="seq 1 5 | grep '2'", stdout='2\n', stderr='', all_output='2\n', return_code=0)

Create a custom runner

Use a custom runner to override the execution defaults, and not have to pass them to every call.

For example, we can change the default value of log_cmd, so that all commands are logged:

>>> shpyx.run("echo 1")
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)

>>> shpyx.run("echo 1", log_cmd=True)
Running: echo 1
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)

>>> runner = shpyx.Runner(log_cmd=True)
>>> runner.run("echo 1")
Running: echo 1
ShellCmdResult(cmd='echo 1', stdout='1\n', stderr='', all_output='1\n', return_code=0)

Propagating terminal control sequences

Note: this is only supported on Linux and macOS. On any other system (including Windows), unix_raw=True raises ShpyxOSNotSupportedError.

Some commands, like psql, might output special characters used for terminal management like cursor movement and colors. For example, the psql command is used to start an interactive shell against a Postgres DB:

shpyx.run(f"psql -h {host} -p {port} -U {user} -d {database}", log_output=True)

However, the above call will not work as well as running psql directly, due to terminal control sequences not being properly propagated. To make it work well, we need to use the script utility which will properly propagate all control sequences:

# Linux:
shpyx.run(f"script -q -c 'psql -h {host} -p {port} -U {user} -d {database}'", log_output=True)
# MacOS:
shpyx.run(f"script -q /dev/null psql -h {host} -p {port} -U {user} -d {database}", log_output=True)

shpyx provides a keyword argument that does this wrapping automatically, unix_raw:

shpyx.run(f"psql -h {host} -p {port} -U {user} -d {database}", log_output=True, unix_raw=True)

It also works with list arguments, which are shell-quoted and always run through a shell:

shpyx.run(["psql", "-h", host, "-p", port, "-U", user, "-d", database], log_output=True, unix_raw=True)

The flag is disabled by default, and should only be used for interactive commands like psql.

API Reference

The following arguments are supported by Runner:

The following arguments are supported by run:

Implementation details

shpyx is a wrapper around the excellent subprocess module, aiming to concentrate all the different API functions (Popen/communicate/poll/wait) into a single function - shpyx.run.

The output streams (stdout and stderr) are each read by a dedicated thread, and decoded incrementally (UTF-8 by default, see decoder_factory, typed as shpyx.DecoderFactory) as the output arrives. This allows logging the live output, keeps the order of the combined output (all_output), and avoids deadlocks when a command fills one pipe while shpyx waits on the other. A KeyboardInterrupt (CTRL-C), or any other exception raised during the run, stops it promptly and kills the subprocess along with all of its descendants (e.g. every command of a shell pipeline).

shpyx supports Linux, macOS and Windows. Other systems (e.g. FreeBSD) are treated as generic POSIX systems: run works as usual, but unix_raw raises ShpyxOSNotSupportedError.

Security

The call to subprocess.Popen uses shell=True when the input to run is a string (to support shell logic like bash piping). This means that an actual system shell is being created, and the subprocess has the permissions of the main Python process.

It is therefore recommended not pass any untrusted input to shpyx.run.

For more info, see security considerations.

Useful links

Relevant Python libraries:

• subprocess
• shlex

Other 3rd-party libraries for running shell commands in Python:

• sarge
• sh
• invoke

Contributing

Contributions are welcome!

See CONTRIBUTING.md for details.

License

shpyx is distributed under the terms of the MIT license.