pyPhases 0.7.0

A Framework for creating a boilerplate template for ai projects that are ready for MLOps

pyPhases

Foundation library for config-driven pipeline frameworks. Provides Phase, Config, Project, and Exporter abstractions.

→ Documentation

Install

pip install pyPhases phases

Quick start

from pyPhases import Phase

class MyPhase(Phase):
    def main(self):
        value = self.getConfig("myKey")
        self.registerData("result", value)

Install

Python >= 3.6 is required.

pip install phases pyPhases

Hello World Example

The project is using two phases, one to generate the sentence to be said and the other one to actually say it.

First we need a data directory where where the data will be stored (default data): mkdir data

poject.yaml

name: HelloWorld

exporter:
   # exporter that can handle primitive types
 - PickleExporter

phases:
   - name: GenerateIt
     exports:
       - sentence
   - name: SayIt

data:
 - name: sentence
   dependsOn:
     - who

config:
 who: world

HelloWorld/phases/GenerateIt.py

from pyPhases import Phase


class GenerateIt(Phase):
    """This phase generates the sentence to be said."""

    def main(self):
        self.log("Generating the sentence")
        greetWho = self.getConfig("who")
        self.registerData("sentence", "Hello %s!" % greetWho)

HelloWorld/phases/SayIt.py

from pyPhases import Phase


class SayIt(Phase):
   """This phase says the sentence."""

   def main(self):
       sentence = self.getData("sentence", str)
       self.logSuccess(sentence)

Run the Example: python -m phases run SayIt

First Output will generate the sentence in phase GenerateIt:

gitpod /workspace/pyphases-hello-world (main) $ phases run SayIt
[phases] Phases v1.0.2 with pyPhases v1.1.5 (Log level: LogLevel.INFO)
[SayIt] RUN phase SayIt: SayIt
[Project] Data sentenceworld--current was not found, try to find phase to generate it
[GenerateIt] RUN phase GenerateIt: GenerateIt
[GenerateIt] Generating the sentence
[SayIt] Hello world!

The data is now stored in the file: data/sentence--current

If your run it again (python -m phases run SayIt) the data is loaded using the Pickleexporter:

gitpod /workspace/pyphases-hello-world (main) $ phases run SayIt
[phases] Phases v1.0.2 with pyPhases v1.1.5 (Log level: LogLevel.INFO)
[SayIt] RUN phase SayIt: SayIt
[SayIt] Hello world!

If you change the config value who in the project to whole world. The data is generated again:

gitpod /workspace/pyphases-hello-world (main) $ phases run --set who="whole world" SayIt
[phases] Phases v1.0.2 with pyPhases v1.1.5 (Log level: LogLevel.INFO)
[SayIt] RUN phase SayIt: SayIt
[Project] Data sentencewholeworld--current was not found, try to find phase to generate it
[GenerateIt] RUN phase GenerateIt: GenerateIt
[GenerateIt] Generating the sentence
[SayIt] Hello whole world!

if you want to explicit generate the data you can just run the first phase: python -m phases run GenerateIt

Example projects

• https://gitlab.com/tud.ibmt.public/pyphases/pyphasesml-example-bumpdetector
• https://gitlab.com/tud.ibmt.public/pyphases/arousaldetector

Data with parameter

For getting Data with parameter all data handler need to support the additional parameters.

project.getData('metadata', pd.DataFrame, myid=1)

requires the Exporter:

• to have the attribute dataHandler = True
• the method getDataHandler(self, data) that returns a datahandler for a specific data instance.
• the get method get(self, **kwargs) that returns the data or tailored data depending on the kwargs, e.g. get(self, myid=1)

the phase that generates the data needs a compatible generateData method such as generateData(self, dataName, myId=None):

Development

build

python setup.py sdist bdist_wheel

publish

twine upload dist/*

documentation

sphinx-apidoc -o docs/source pyPhases sphinx-build -b html docs/source docs/build

test

python -m unittest discover -v