Skip to main content

Run multiple subprocesses asynchronous/in parallel with streamed output/non-blocking reading

Project description

A simple way to manage several parallel subprocesses. This provides for asynchronous processes and non-blocking reading of their output.

Using the Job system is the quickest approach to just run processes and log their output (by default in files named ‘/tmp/job_ID.log’)

from shelljob import job

jm = job.FileMonitor()
jm.run([
    [ 'ls', '-alR', '/usr/local' ],
    [ 'my_prog' ],
    [ 'build', 'output', 'input' ],
])

Note the command items are lists passed directly to subprocess.Popen.

The lower level Group class provides a simple container for more manual job management.

from shelljob import proc

g = proc.Group()
p1 = g.run( [ 'ls', '-al', '/usr/local' ] )
p2 = g.run( [ 'program', 'arg1', 'arg2' ] )

while g.is_pending():
    lines = g.readlines()
    for proc, line in lines:
        sys.stdout.write( "{}:{}".format( proc.pid, line ) )

You can use my Launchpad project to submit issues.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shelljob-0.3.0.tar.gz (7.1 kB view details)

Uploaded Source

File details

Details for the file shelljob-0.3.0.tar.gz.

File metadata

  • Download URL: shelljob-0.3.0.tar.gz
  • Upload date:
  • Size: 7.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for shelljob-0.3.0.tar.gz
Algorithm Hash digest
SHA256 164f7cc2d399f14f000bb098710583eae5a7af5f4a1ba13dba8b09d5374d1df4
MD5 0d93e1ab82e1997076339d8d598735d5
BLAKE2b-256 25304704cb34dfa1acdd5bdeb161b33130eb8ec3866c0579973fe8e42e135ac9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page