preemo-worker-sdk 0.2.1

No project description provided

Preemo Worker SDK

This subrepo contains the python implementation of the Preemo Worker SDK.

Installation

pip install preemo-worker-sdk

Usage

Register Function

In order to register a function with Preemo workers, you can use register to decorate your functions.

from preemo.worker import register

@register(name="some_name", namespace="dev")
def do_something(params: str):
    ...

Both parameters, name and namespace, are optional. If the name isn't specified, it will default to the name of the function. If the namespace isn't specified, it will default to a global namespace.

@register
def do_something(params: str):
    # registers with name do_something in the global namespace
    ...

At the moment, only functions that take 0 or 1 string arguments will work. These functions should also either return None or a string.

Execute Function

In order to execute a function that you have previously registered with Preemo workers, you can use get_function.

from preemo.worker import get_function

do_something = get_function(name="some_name", namespace="dev")
result = do_something("params")
...

The second parameter, namespace, is optional. If the namespace isn't specified, it will default to a global namespace.

# gets the function named do_something in the global namespace
do_something = get_function("do_something")
result = do_something("params")
...

Parallelize Function Execution

In order to execute a function with multiple parameters in parallel, you can use parallelize.

from preemo.worker import parallelize

do_something = get_function(name="some_name", namespace="dev")
results = parallelize(
    do_something,
    params=[
        "params1",
        "params2",
        ...
    ]
)
...

If your function doesn't take a parameter and you'd like to run multiple instances of it in parallel, you can use the count parameter.

do_something = get_function(name="some_name", namespace="dev")
results = parallelize(
    do_something,
    count=10
)
...

Contributing

Contribution guidelines for this project