pinexq-procon 2.1.0.dev3

Framework to create containers for DataCybernetic's PineXQ computing platform

PineXQ ProCon Framework

Computations in DC-Cloud are done by Workers running inside a ProcessingContainer, or short "ProCon", which is also the name of the framework. ProCon provides an unobtrusive wrapper around function definitions without introducing new semantics, allowing for a clean definition of the computational task, while handling all cloud-related communication and data-management transparently in the background. This removes the code and configuration required from the function implementation.

Installation

To install the package use the pip command, to either install from a package feed:

pip install pinexq-procon

Creating a container

To publish a function in a container it has to be a method of a class inheriting from the Step class.

from pinexq.procon.step import Step # import package

class MyStepCollection(Step):  # define the container class
    def calculate_square(self, x: float) -> float:  # define a step function
        """Calculate the square of x

        :param x: a float number
        :returns: the square of x
        """
        return x ** 2

    # More step functions can go in the same class    

if __name__ == '__main__':  # add script guard
    MyStepCollection()  # run the container - this will spawn the cli

It is mandatory to annotate the types of parameters and return value. Docstrings are optional, but highly recommended.

The documentation has a detailed section about implementing processing-steps.

Running a Step-function locally

The Python file with the container is itself a cli-tool. You get a list of all available commands with the --help parameter.

python ./my_step_file.py --help

With the run option you can call a function in the container directly and the result is written to the console.

python ./my_step_file.py run --function calculate_square --parameters "{'x': 5}"
25

You can find full list of available commands in the cli documentation. All possible parameters and environment variables are listed here.