A light-weight framework for defining and running dependent tasks.
Project description
codep
A light-weight framework for defining and running dependent tasks. By building a dependency graph between tasks, the runner makes sure that tasks that are required by multiple dependants are only run once and reuses the results.
Usage
In the example below the calculate_circumference
and calculate_volume
tasks
both depend on the acquire_radius
and calculate_pi
tasks. The runner will
infer that from the dependency graph and run them before.
import codep
import immutables
@codep.make_partial()
def acquire_radius(state: immutables.Map) -> immutables.Map:
return state.set("radius", 6_371)
@codep.make_partial()
def calculate_pi(state: immutables.Map) -> immutables.Map:
return state.set("pi", 3.14)
@codep.make_partial(depends=(calculate_pi, acquire_radius))
def calculate_circumference(state: immutables.Map) -> immutables.Map:
return state.set("circumference", 2 * state["pi"] * state["radius"])
@codep.make_partial(depends=(acquire_radius, calculate_pi))
def calculate_volume(state: immutables.Map) -> immutables.Map:
return state.set("volume", (4/3) * state["pi"] * state["radius"] ** 3)
if __name__ == '__main__':
r1, r2 = codep.run(calculate_circumference, calculate_volume)
circumference = r1.state["circumference"]
volume = r2.state["volume"]
print(
f"The circumference of earth is {circumference} km and its volume is "
f"{volume} km^3"
)
Todo
- Make Partial a Generic, and make tasks only able to return a value.
Partial.apply
should instead assign the result to the state which becomes a mappingType[Partial] -> Any
. - Remove
print()
, use logging - Concurrent task execution
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
codep-0.0.1.tar.gz
(2.6 kB
view hashes)
Built Distribution
Close
Hashes for codep-0.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ee21d163962c3c769746e2749b2e3a83e54063b2eef0c8bc6cd7483de7481e0 |
|
MD5 | 097459e78cc90972b823a39b7b227c41 |
|
BLAKE2b-256 | 3de2be0644e1e0461708940989f801ef3617c56aa5cdbb0a2e5d32c7b6aeb045 |