Skip to main content

Tools to create executable dags from interdependent functions.

Project description

dags

PyPI PyPI - Python Version Conda Version Conda Platform PyPI - License Documentation Status GitHub Workflow Status codecov pre-commit.ci status

About

dags provides tools to combine several interrelated functions into one function. The order in which the functions are called is determined by a topological sort on a dag that is constructed from the function signatures. You can specify which of the function results will be returned in the combined function.

dags is a tiny library, all the hard work is done by the great NetworkX.

Example

To understand what dags does, let's look at a very simple example of a few functions that do simple calculations.

def f(x, y):
    return x**2 + y**2


def g(y, z):
    return 0.5 * y * z


def h(f, g):
    return g / f

Assume that we are interested in a function that calculates h, given x, y and z.

We could hardcode this function as:

def hardcoded_combined(x, y, z):
    _f = f(x, y)
    _g = g(y, z)
    return h(_f, _g)


hardcoded_combined(x=1, y=2, z=3)
0.6

Instead, we can use dags to construct the same function:

from dags import concatenate_functions

combined = concatenate_functions([h, f, g], targets="h")

combined(x=1, y=2, z=3)
0.6

More examples can be found in the documentation

Notable features

  • The dag is constructed while the combined function is created and does not cause too much overhead when the function is called.
  • If all individual functions are jax compatible, the combined function is jax compatible.
  • When jitted or vmapped with jax, we have not seen any performance loss compared to hard coding the combined function.
  • When there is more than one target, you can determine whether the result is returned as tuple, list or dict or pass in an aggregator to combine the multiple outputs.
  • Since the relationships are discoverd from function signatures, dags provides decorators to rename arguments in order to make it easy to wrap functions you do not control yourself.

Installation

dags is available on PyPI and conda-forge. Install it with

$ pip install dags

# or

$ pixi add dags

# or

$ conda install -c conda-forge dags

Documentation

The documentation is hosted on Read the Docs.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dags-0.3.0.tar.gz (16.4 kB view details)

Uploaded Source

Built Distribution

dags-0.3.0-py3-none-any.whl (18.0 kB view details)

Uploaded Python 3

File details

Details for the file dags-0.3.0.tar.gz.

File metadata

  • Download URL: dags-0.3.0.tar.gz
  • Upload date:
  • Size: 16.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for dags-0.3.0.tar.gz
Algorithm Hash digest
SHA256 8c3ec87b51e8db7f69787ae5c5a57499c8ab0242026d5c9a26c27999c8c7bf06
MD5 d37d312a662fec60d9acc8f7b1742b73
BLAKE2b-256 bfb673b9fa046f4ec022761137cdcd160603fd4fae928b10502208256fb55310

See more details on using hashes here.

File details

Details for the file dags-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: dags-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 18.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for dags-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d47025019bf72d4261c9df6f73d3402568feaafba2d44a17206b74dfc0e7e8bb
MD5 7fd12991d90314081fdba522a9a3876d
BLAKE2b-256 50087acb9d35c4a193a14fafdf4dbf0d1240dc3f74a8961fa46f3544a80c1f6b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page