pipelayer 0.7.0

A lightweight pipeline framework

PipeLayer

PipeLayer is a event-driven pipeline framework. Define a series of steps, and chain them together to create modular applications.

Table of Contents

• Installation
• Getting Started
• The Framework
  Complete documentation can be found here: greaterthan.solutions/pipelayer
  
  

Installation

From the command line:

pip install pipelayer

Getting Started

Step 1: Create The Filters

hello_world_filters.py

from pipelayer import Filter


class HelloFilter(Filter):
    def run(self, data, context):
        return "Hello"


class WorldFilter(Filter):
    def run(self, data, context):
        return f"{data}, World!"

functions.py

def create_message(data, context):
    return {"message": data}

Step 2: Create The Pipeline

Create a module to run the pipeline:

app.py

import json
from pipelayer import Pipeline
from pipelayer.util import render_manifest

from functions import create_message
from hello_world_filters import HelloFilter, WorldFilter


if __name__ == "__main__":
    hello_world_pipeline = Pipeline(
        [
            HelloFilter,
            WorldFilter,
            create_message,
            lambda data, context: json.dumps(data),
        ]
    )

    output = hello_world_pipeline.run(None)

    # output = '{"message": "Hello, World!"}'

    print("\nPipeline Output:")
    print(output)
    print("\nManifest:")
    print(render_manifest(hello_world_pipeline.manifest))

Step 3: Run the Pipeline

from the command line:

run app.py