vorlauf 1.0.0

What?

Vorlauf is a very minimal tool that helps you to create process pipelines (in the shell sense). It also helps separate out the definition of a Process from the running of a that process with a given stdin, stdout, and stderr.

Why?

Because the subprocess api for chaining processes together is cumbersome and not very well documented.

How?

This library does basically nothing - it’s implemented in fewer than 100 lines. There are two classes available, Process and Pipeline.

critical_grepper = Process('grep', 'CRITICAL')

syslog = open('/var/log/syslog', 'r')
apachelog = open('/var/log/httpd/error.log', 'r')

filtered = open('critical.log', 'w')

for logfile in (syslog, apachelog):
  critical_grepper.run(stdin=logfile, stdout=filtered)

Example

from vorlauf import Pipeline, Process

pipeline = Pipeline()
pipeline.add(Process('cat', 'foo.txt'))
pipeline.add(Process('grep', 'something'))
pipeline.add(Process('uniq'))

with open('new.txt', 'wb') as fd:
    pipeline.run(stdout=fd)

And because of operator overloading built into the Process and Pipeline classes, this can be simplified as:

from vorlauf import Process as P

pipeline = P('cat', 'foo.txt') | P('grep', 'something') | P('uniq')
with open('new.txt', 'wb') as fd:
    pipeline.run(stdout=fd)

Finally, you could use the Process class to create reusable components:

from vorlauf import Process

class GPG(Process):

    def __init__(self, passphrase):
        super(GPG, self).__init__('gpg', '-c', '--passphrase', passphrase, '-')


class MySQLDump(Process):

    def __init__(self, password, dbname, **kwargs):
        super(MySQLDump, self).__init__(
            'mysqldump', '-u', 'root', '-p{}'.format(password), dbname
        )


with open('mysql.dump', 'wb') as fd:
    pipeline = MySQLDump('loldongs', 'foo') | GPG('supersekrit')
    pipeline.run(stdout=fd)

Tests

Run:

virtualenv venv
venv/bin/pip install -e .
venv/bin/python test.py