gipc 0.2.0

gevent-cooperative child processes and inter-process communication.

What can gipc do for you?

Naive usage of multiprocessing in the context of a gevent-powered application may raise various problems and most likely breaks the application in some way. That is where gipc comes into play: it is developed with the motivation to solve these issues transparently and make using gevent in combination with multiprocessing a no-brainer again.

With gipc (pronunciation “gipsy”) multiprocessing.Process-based child processes can safely be created anywhere within your gevent-powered application. Furthermore, gipc provides gevent-cooperative inter-process communication and useful helper constructs.

gipc is lightweight and very simple to integrate. In the following code snippet, a Python object is sent from a greenlet in the main process to a child process:

import gevent
import gipc

obj = 0

def child(reader):
    assert reader.get() == obj

if __name__ == "__main__":
    with gipc.pipe() as (reader, writer):
        writelet = gevent.spawn(lambda w: w.put(obj), writer)
        readchild = gipc.start_process(child, args=(reader,))
        writelet.join()
        readchild.join()

Can’t I do this with just gevent+multiprocessing?

It requires care: child process creation via multiprocessing in the context of gevent yields an undesired event loop state in the child. Greenlets spawned before forking are duplicated in the child. Furthermore, blocking method calls such as join() on a multiprocessing.Process or the send()/recv() methods on a multiprocessing.Connection are not gevent-cooperative. gipc overcomes these challenges for you transparently and in a straight-forward fashion. It allows for simple integration of child processes in your application – on POSIX-compliant systems as well as on Windows.

Documentation

The documentation with technical notes, API details, installation instructions, requirements, and code examples can be found at http://gehrcke.de/gipc.

Availability

Releases are available at PyPI. The development version can be received from the mercurial repository at bitbucket.

Author & license

gipc is written and maintained by Jan-Philip Gehrcke and is licensed under the Apache License 2.0.

Contact

Your feedback is highly appreciated. You can contact me at [e-mail] or use the Bitbucket issue tracker.