Skip to main content

gevent-cooperative child processes and inter-process communication.

Project description

What can gipc do for you?

Naive usage of multiprocessing in the context of a gevent-powered application may raise various problems and most likely breaks the application in some way. That is where gipc comes into play: it is developed with the motivation to solve these issues transparently and make using gevent in combination with multiprocessing a no-brainer again.

With gipc (pronunciation “gipsy”) multiprocessing.Process-based child processes can safely be created anywhere within your gevent-powered application. Furthermore, gipc provides gevent-cooperative inter-process communication and useful helper constructs.

gipc is lightweight and very simple to integrate. In the following code snippet, a Python object is sent from a greenlet in the main process to a child process:

import gevent
import gipc

obj = 0

def child(reader):
    assert reader.get() == obj

if __name__ == "__main__":
    with gipc.pipe() as (reader, writer):
        writelet = gevent.spawn(lambda w: w.put(obj), writer)
        readchild = gipc.start_process(child, args=(reader,))
        writelet.join()
        readchild.join()

Can’t I do this with just gevent+multiprocessing?

It requires care: child process creation via multiprocessing in the context of gevent yields an undesired event loop state in the child. Greenlets spawned before forking are duplicated in the child. Furthermore, blocking method calls such as join() on a multiprocessing.Process or the send()/recv() methods on a multiprocessing.Connection are not gevent-cooperative. gipc overcomes these challenges for you transparently and in a straight-forward fashion. It allows for simple integration of child processes in your application – on POSIX-compliant systems as well as on Windows.

Documentation

The documentation with technical notes, API details, installation instructions, requirements, and code examples can be found at http://gehrcke.de/gipc.

Availability

Releases are available at PyPI. The development version can be received from the mercurial repository at bitbucket.

Author & license

gipc is written and maintained by Jan-Philip Gehrcke and is licensed under the Apache License 2.0.

Contact

Your feedback is highly appreciated. You can contact me at jgehrcke@googlemail.com or use the Bitbucket issue tracker.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
gipc-0.2.0.tar.gz (21.4 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page