Skip to main content

A light-weight framework for defining and running dependent tasks.

Project description

codep

A light-weight framework for defining and running dependent tasks. By building a dependency graph between tasks, the runner makes sure that tasks that are required by multiple dependants are only run once and reuses the results.

Usage

In the example below the calculate_circumference and calculate_volume tasks both depend on the acquire_radius and calculate_pi tasks. The runner will infer that from the dependency graph and run them before.

import codep
import immutables


@codep.make_partial()
def acquire_radius(state: immutables.Map) -> immutables.Map:
    return state.set("radius", 6_371)


@codep.make_partial()
def calculate_pi(state: immutables.Map) -> immutables.Map:
    return state.set("pi", 3.14)


@codep.make_partial(depends=(calculate_pi, acquire_radius))
def calculate_circumference(state: immutables.Map) -> immutables.Map:
    return state.set("circumference", 2 * state["pi"] * state["radius"])


@codep.make_partial(depends=(acquire_radius, calculate_pi))
def calculate_volume(state: immutables.Map) -> immutables.Map:
    return state.set("volume", (4/3) * state["pi"] * state["radius"] ** 3)


if __name__ == '__main__':
    r1, r2 = codep.run(calculate_circumference, calculate_volume)
    circumference = r1.state["circumference"]
    volume = r2.state["volume"]
    print(
        f"The circumference of earth is {circumference} km and its volume is " 
        f"{volume} km^3"
    )

Todo

  • [ ] Make Partial a Generic, and make tasks only able to return a value. Partial.apply should instead assign the result to the state which becomes a mapping Type[Partial] -> Any.
  • [ ] Remove print(), use logging
  • [ ] Concurrent task execution

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for codep, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size codep-0.0.1-py2.py3-none-any.whl (2.6 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size codep-0.0.1.tar.gz (2.6 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page