Skip to main content

Motiv, Simple and Efficient processing pipelining.

Project description


Simple and Efficient processing pipelining.

Build Status Total alerts codecov License Gitter chat PyPI

Motiv is a framework to ease and enables building minimal pipelines and pipeline-like applications. It abstracts away communication, messaging patterns and execution patterns.

When to use Motiv?

Motiv is for minimal pipelining and pipeline-like applications, you can build a tightly-coupled or a loosely-coupled pipeline using it. However, if you're pipeline's purpose is data processing, motiv is a good start. If the purpose is big data processing, then you're better off using a big data processing engine (e.g. Spark).

Why Motiv?

Motiv is simple, it provides many messaging-patterns, it manages them for you and it provides you direct control over them. Motiv is stateless, and so should the applications that use it. Motiv uses efficient and very fast communication/message-passing (zmq) for the underlying communication.


$ pip install motiv

or (recommended)

$ git clone
$ cd motiv
$ python3 install


Motiv has two type of patterns that are glued together to form a component in a pipeline. A stream (messaging-pattern) and an actor (execution-pattern). Streams can be a Emitter, Subscriber, Ventilator, Worker, Sink. an actor can be Ticker or Proxy.


Streams are boilerplate messaging-patterns that either send out messages or receive messages. for example an Emitter has a typical publisher behavior. Unlike the Emitter A Ventilator distributes messages between listening components (workers).

Execution patterns?

An execution pattern defines how a component should execute, for example, a component can simply be a Proxy between two streams that forwards messages received over subscriber stream to a ventilator, or from a worker to a publisher, or from a worker to a sink. It can also be a Ticker which is execute-till-halt event-loop with each cycle calling Ticker.tick method function, this type of behavior is convenient for processing incoming messages and sending out results to different actors or do an action depending on these messages.

Streams + Execution Pattern

An execution pattern or an actor expects to have an input stream or output stream or both for it to be runnable.


See examples: motiv/examples/

Run examples,

$ git clone
$ cd motiv
$ python3 motiv/examples/
$ python3 motiv/examples/


Motiv is still in development, master might not be stable, features will be rolled out fast. I encourage you to fix any code you see wrong, raise any issues that concern you and actively participate in developing Motiv.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

motiv-0.1.0.tar.gz (12.6 kB view hashes)

Uploaded source

Built Distribution

motiv-0.1.0-py3-none-any.whl (19.4 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page