A tiny mlops library for building machine learning pipelines on your local machine.
Project description
tinypipeline
Overview
tinypipeline
is a tiny mlops library that provides a simple framework for organizing your machine learning pipeline code into a series of steps. It does not handle networking, I/O, or compute resources. You do the rest in your pipeline steps.
Installation
$ pip install tinypipeline
Usage
tinypipeline
exposes two main objects:
pipeline
: a decorator for defining your pipeline. Returns aPipeline
instance.step
: a decorator that is used to define individual pipeline steps. Returns aStep
instance.
Each object requires you provide a name
, version
, and description
to explicitly define what pipeline you're creating.
The Pipeline
object that is returned from the decorator has a single method: run()
.
API
If you'd like to use this package, you can follow the example.py
below:
from tinypipeline import pipeline, step
@step(name='step_one', version='0.0.1', description='first step')
def step_one():
print("Step function one")
@step(name='step_two', version='0.0.1', description='second step')
def step_two():
print("Step function two")
@pipeline(
name='test-pipeline',
version='0.0.1',
description='a test tinypipeline',
)
def pipe():
return [step_one, step_two]
pipe = pipe()
pipe.run()
Output:
You can run the example.py
like so:
$ python example.py
+-------------------------------------------------------------------+
| Running pipeline: Pipeline(name='test-pipeline', version='0.0.1') |
+-------------------------------------------------------------------+
Running step [step_one]...
Step function one
Step [step_one] completed in 0.000356 seconds
Running step [step_two]...
Step function two
Step [step_two] completed in 0.000317 seconds
Running tests
Tests are run using pytest. To run the tests you can do:
$ pip install pytest
$ pytest
Limitations
- Currently
tinypipeline
only supports linear pipelines, and doesn't support full dependency graphs
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tinypipeline-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | efdea828d6fb81d483a5b45012f50d2c27d9768e80a2d85d98aa1475f253c318 |
|
MD5 | de6faee15e6566dc430150d2cac9eb0a |
|
BLAKE2b-256 | 71d81b952ceaa5f1edea9d64a45a9681b25fcf69b50b7acc8bf60a2f4378d748 |