zproc 1.0.0a2

ZProc - Process on steroids

ZProc lets you do shared-state multitasking without the perils of having shared-memory.

Behold, the power of ZProc:

# Some initialization
import zproc


ctx = zproc.Context(wait=True)  # wait for processes in this context
ctx.state["cookies"] = 0


# Define "atomic" operations

@zproc.atomic
def eat_cookie(state):
    """Eat a cookie."""

    state["cookies"] -= 1
    print("nom nom nom")


@zproc.atomic
def bake_cookie(state):
    """Bake a cookie."""
    state["cookies"] += 1
    print("Here's a cookie!")


# Fire up processes

@ctx.call_when_change('cookies')
def cookie_eater(_, state):
    """Eat cookies as they're baked."""

    eat_cookie(state)


for i in range(5):
    bake_cookie(ctx.state)

Result:

Here's a cookie!
Here's a cookie!
nom nom nom
Here's a cookie!
nom nom nom
Here's a cookie!
nom nom nom
Here's a cookie!
nom nom nom
nom nom nom

(baker and eater run in different processes)

The core idea

ZProc tries to breathe new life into the archaic idea of shared-state multitasking by protecting application state with logic and reason.

Shared state is frowned upon by almost everyone, (mostly) due to the fact that memory is inherently dumb.

Like memory doesn't really care who's writing to it.

ZProc's state tries to keep a track of who's doing what.

The Goal

ZProc aims to make building multi-taking applications easier and faster for everyone, in a pythonic way.

It started out from the core idea of having a smart state -- eventually wanting to turn into a full-fledged framework for all things multitasking.

Install

$ pip install zproc

Or, from the development branch

$ pip install git+https://github.com/pycampers/zproc.git@next#egg=zproc

License: MIT License (MIT)
Requires: Python >=3.5

Documentation ( )

Read the docs

Examples

Features

• 🌠   Process management

  • Process Factory
  • Remembers to kill processes when exiting, for general peace. (even when they're nested)
  • Keeps a record of processes created using ZProc.
  • 🔖

• 🌠   Process Maps

  • Automatically manages worker processes, and delegates tasks to them.
  • 🔖

• 🌠   Asynchronous paradigms without async def

  • Build a combination of synchronous and asynchronous systems, with ease.
  • By watching for changes in state, without Busy Waiting.
  • 🔖

• 🌠   Atomic Operations

  • Perform an arbitrary number of operations on state as a single, atomic operation.
  • 🔖

Caveats

• The state only gets updated if you do it directly.
  This means that if you mutate objects inside the state, they wont get reflected in the global state.

• The state should be pickle-able.

• It runs an extra Process for managing the state.
  Its fairly lightweight though, and shouldn't add too much weight to your application.

FAQ

• Fast?

  • Above all, ZProc is written for safety and the ease of use.
  • However, since its written using ZMQ, it's plenty fast for most stuff.
  • Run -> 🔖 for a taste.

• Stable?

  • Mostly. However, since it's still in the 0.x.x stage, you can expect some API changes.

• Production ready?

  • Please don't use it in production right now.

• Windows compatible?

  • Probably?

Local development

git clone https://github.com/pycampers/zproc.git

cd zproc
pipenv install
pipenv install -d
 
pipenv shell

pip install -e .
pytest 

Build documentation

Assuming you have sphinx installed (Linux)

cd docs
./build.sh 

./build.sh loop  # starts a build loop.

ZProc in the wild

• Oscilloscope

• Muro

Thanks

• Thanks to open logos for providing the wonderful ZProc logo.
• Thanks to pieter hintjens, for his work on the ZeroMQ library and for his amazing book.
• Thanks to tblib, ZProc can raise First-class Exceptions from the zproc server!
• Thanks to psutil, ZProc can handle nested procesess!
• Thanks to Kennith Rietz. His setup.py was used to host this project on pypi. Plus a lot of documentation structure is blatantly copied from his documentation on requests

---

ZProc is short for Zero Process.

---

🐍🏕️