A lightweight pipeline framework
Project description
PipeLayer
PipeLayer is a lightweight Python pipeline framework. Define a series of steps, and chain them together to create modular applications.
)
Table of Contents
- Installation
- Getting Started
- The Framework
Documentation has been moved to greaterthan.solutions
Installation
From the command line:
pip install pipelayer
Getting Started
Step 1: Create Pipeline Filters
hello_world_filters.py
from pipelayer import Filter
class HelloFilter(Filter):
def run(self, data, context):
return "Hello"
class WorldFilter(Filter):
def run(self, data, context):
return f"{data}, World!"
functions.py
def create_message_dict(data, context):
return {"message": data}
Step 2: Create a Pipeline
Create a module to run the pipeline:
app.py
from pipelayer import Pipeline
from functions import create_message
from hello_world_filters import HelloFilter, WorldFilter
if __name__ = "__main__":
hello_world_pipeline = Pipeline([
HelloFilter, # pipeline.Filter type
WorldFilter, # pipeline.Filter instance
create_message_dict # function type
lambda data, context: json.dumps(data) # anonymous function
])
output = hello_world_pipeline.run()
# output = '{"message": "Hello, World!"}'
print(f"Pipeline Output: {output}")
print(hello_world_pipeline.manifest.__dict__)
Step 3: Run the Pipeline
from the command line:
run app.py
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pipelayer-0.5.1.tar.gz
(10.5 kB
view hashes)
Built Distribution
pipelayer-0.5.1-py3-none-any.whl
(12.2 kB
view hashes)
Close
Hashes for pipelayer-0.5.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b5782c3d92b3c9b0e96530692e370a4b59a1e9a8f1e7f35d6a23f8b23ae09b38 |
|
MD5 | bff6a324b13dab687dc92ad00378e9cd |
|
BLAKE2b-256 | e8b9a382df7b39f0205808976a0db726dee708c798433d62b45b46d2df679369 |