Skip to main content

Lightweight, asynchronous Directed Acyclic Graph (DAG) orchestrator for Python.

Project description

TidyDAG

Lightweight, asynchronous Directed Acyclic Graph (DAG) orchestrator for Python.

Python Pytest Ruff Taskfile AWS CodeBuild coverage


TidyDAG allows you to define complex task dependencies and execute them concurrently while maintaining a shared state. Built on asyncio, it provides a simple API to resolve execution order automatically via topological sorting.


📦 Features

  • Asynchronous Execution: Built on asyncio for non-blocking task orchestration.
  • 🔗 Topological Sorting: Automatically resolves execution order based on dependencies.
  • 🧠 Shared Context: Pass and mutate state across your graph nodes seamlessly.
  • 🛡️ Type Safety: Built with modern Python 3.13+ features and comprehensive type hints.
  • 🧩 Simple API: Define nodes and dependencies with minimal boilerplate.
  • 🤖 Task Automation: Managed with uv, ruff, and Taskfile.yml.

📁 Project Layout

.
├── src/                          # Core library source code
│   └── tidydag/
│       ├── node/                 # Base Node and state definitions
│       └── orchestrator/         # DAG execution engine
├── test/                         # Pytest-based test suite
│   └── unit/
├── Taskfile.yml                  # Task automation
├── pyproject.toml                # Python project config
└── README.md

🔧 Setup

  1. Install dependencies (requires Python ≥ 3.13 and uv):

    task deps:install
    

💡 Core Concepts

1. Node

The basic building block of a DAG. Every task must inherit from the Node class and implement the execute method.

2. Orchestrator

The engine that manages the execution of the graph. It ensures nodes are executed only after their parents have successfully completed.

3. OrchestratorContext

A container for shared state that is passed to every node during execution.


🚀 Quick Start

Defining Custom Nodes

from dataclasses import dataclass
from tidydag.node.base import Node, NodeState, SuccessState, OrchestratorContext

class HelloNode(Node):
    async def execute(self, ctx: OrchestratorContext) -> NodeState:
        print(f"Hello from {self.name}!")
        return SuccessState()

class ProcessNode(Node):
    async def execute(self, ctx: OrchestratorContext) -> NodeState:
        ctx.state.count += 1
        return SuccessState()

Running the DAG

from tidydag.orchestrator import Orchestrator, OrchestratorContext

@dataclass
class HelloContext(OrchestratorContext):
    count: int = 0

# Initialize the orchestrator
orchestrator = Orchestrator()

# Define nodes and dependencies
node_a = HelloNode(name="A")
node_b = ProcessNode(name="B", parents=node_a)
node_c = HelloNode(name="C", parents=node_a)
node_d = HelloNode(name="D", parents=[node_b, node_c])

# Add nodes to the orchestrator
for node in [node_a, node_b, node_c, node_d]:
    orchestrator.add_node(node)

initial_state = HelloContext()
# Run the graph with an initial state
orchestrator.run_sync(state=initial_state)

print(f"Final state: {initial_state}")

🧪 Testing

Run all tests with coverage:

task test:run

To generate a coverage badge (docs/coverage.svg), run:

task test:run BADGE=true

✨ Automation (Taskfile)

Common commands available via task:

task                    # Show available tasks
task deps:install       # Create venv and sync Python dependencies
task test:run           # Run all tests with pytest
task lint:check         # Run linters and formatters in check mode
task lint:format        # Format code with ruff

📄 License

Proprietary — ©Taidy. All rights reserved.

This project is intended for internal use only and may not be copied, distributed, or modified outside of authorized teams without written permission from Taidy.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tidydag-0.1.10.tar.gz (5.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tidydag-0.1.10-py3-none-any.whl (7.0 kB view details)

Uploaded Python 3

File details

Details for the file tidydag-0.1.10.tar.gz.

File metadata

  • Download URL: tidydag-0.1.10.tar.gz
  • Upload date:
  • Size: 5.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.11.8 {"installer":{"name":"uv","version":"0.11.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for tidydag-0.1.10.tar.gz
Algorithm Hash digest
SHA256 c3ec5fc3add30367da569c39ad1dab09604def6f59c4cd16e1d9d332a6318489
MD5 2eac272f69a3cc4ef1280c8f97f6669d
BLAKE2b-256 2486cc73b1b6b7a11021ee8e6c7e8c44fee618c19b2e1baee37c29ba7b3a83ec

See more details on using hashes here.

File details

Details for the file tidydag-0.1.10-py3-none-any.whl.

File metadata

  • Download URL: tidydag-0.1.10-py3-none-any.whl
  • Upload date:
  • Size: 7.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.11.8 {"installer":{"name":"uv","version":"0.11.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for tidydag-0.1.10-py3-none-any.whl
Algorithm Hash digest
SHA256 18f7d0a0a3f66eeff84a4ee31e7479705ded8cd46d69e418caba6739e91daad4
MD5 016c7864f65108e8bbacc958a750e528
BLAKE2b-256 66e7c099508c01e6894c7452e2bc97eb4fe19e5bbd8f7ae076604f2ccffc438a

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page