Skip to main content

Lightweight, asynchronous Directed Acyclic Graph (DAG) orchestrator for Python.

Project description

TidyDAG

Lightweight, asynchronous Directed Acyclic Graph (DAG) orchestrator for Python.

Python Pytest Ruff Taskfile AWS CodeBuild coverage


TidyDAG allows you to define complex task dependencies and execute them concurrently while maintaining a shared state. Built on asyncio, it provides a simple API to resolve execution order automatically via topological sorting.


📦 Features

  • Asynchronous Execution: Built on asyncio for non-blocking task orchestration.
  • 🔗 Topological Sorting: Automatically resolves execution order based on dependencies.
  • 🧠 Shared Context: Pass and mutate state across your graph nodes seamlessly.
  • 🛡️ Type Safety: Built with modern Python 3.13+ features and comprehensive type hints.
  • 🧩 Simple API: Define nodes and dependencies with minimal boilerplate.
  • 🤖 Task Automation: Managed with uv, ruff, and Taskfile.yml.

📁 Project Layout

.
├── src/                          # Core library source code
│   └── tidydag/
│       ├── node/                 # Base Node and state definitions
│       └── orchestrator/         # DAG execution engine
├── test/                         # Pytest-based test suite
│   └── unit/
├── Taskfile.yml                  # Task automation
├── pyproject.toml                # Python project config
└── README.md

🔧 Setup

  1. Install dependencies (requires Python ≥ 3.13 and uv):

    task deps:install
    

💡 Core Concepts

1. Node

The basic building block of a DAG. Every task must inherit from the Node class and implement the execute method.

2. Orchestrator

The engine that manages the execution of the graph. It ensures nodes are executed only after their parents have successfully completed.

3. OrchestratorContext

A container for shared state that is passed to every node during execution.


🚀 Quick Start

Defining Custom Nodes

from dataclasses import dataclass
from tidydag.node.base import Node, NodeState, SuccessState, OrchestratorContext

class HelloNode(Node):
    async def execute(self, ctx: OrchestratorContext) -> NodeState:
        print(f"Hello from {self.name}!")
        return SuccessState()

class ProcessNode(Node):
    async def execute(self, ctx: OrchestratorContext) -> NodeState:
        ctx.state.count += 1
        return SuccessState()

Running the DAG

from tidydag.orchestrator import Orchestrator, OrchestratorContext

@dataclass
class HelloContext(OrchestratorContext):
    count: int = 0

# Initialize the orchestrator
orchestrator = Orchestrator()

# Define nodes and dependencies
node_a = HelloNode(name="A")
node_b = ProcessNode(name="B", parents=node_a)
node_c = HelloNode(name="C", parents=node_a)
node_d = HelloNode(name="D", parents=[node_b, node_c])

# Add nodes to the orchestrator
for node in [node_a, node_b, node_c, node_d]:
    orchestrator.add_node(node)

initial_state = HelloContext()
# Run the graph with an initial state
orchestrator.run_sync(state=initial_state)

print(f"Final state: {initial_state}")

🧪 Testing

Run all tests with coverage:

task test:run

To generate a coverage badge (docs/coverage.svg), run:

task test:run BADGE=true

✨ Automation (Taskfile)

Common commands available via task:

task                    # Show available tasks
task deps:install       # Create venv and sync Python dependencies
task test:run           # Run all tests with pytest
task lint:check         # Run linters and formatters in check mode
task lint:format        # Format code with ruff

📄 License

Proprietary — ©Taidy. All rights reserved.

This project is intended for internal use only and may not be copied, distributed, or modified outside of authorized teams without written permission from Taidy.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tidydag-0.1.9.tar.gz (5.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tidydag-0.1.9-py3-none-any.whl (7.0 kB view details)

Uploaded Python 3

File details

Details for the file tidydag-0.1.9.tar.gz.

File metadata

  • Download URL: tidydag-0.1.9.tar.gz
  • Upload date:
  • Size: 5.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.8.17

File hashes

Hashes for tidydag-0.1.9.tar.gz
Algorithm Hash digest
SHA256 1dbfa7458acef5eb2af80551be4ad69a38feb6bf9ce8862f4dd9e4b9cfbd2307
MD5 8d98f22274385ee181a9e902f56426b7
BLAKE2b-256 b32bed37442671e4a09554722bd19e9ccd6fa0d78be7f293bf9991fbda0040c4

See more details on using hashes here.

File details

Details for the file tidydag-0.1.9-py3-none-any.whl.

File metadata

  • Download URL: tidydag-0.1.9-py3-none-any.whl
  • Upload date:
  • Size: 7.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.8.17

File hashes

Hashes for tidydag-0.1.9-py3-none-any.whl
Algorithm Hash digest
SHA256 db3301bd51100e305e6d666f48de3c48a021a780cdf9426c732453fe0fb0d0b3
MD5 c339019006a378c3d049b56c2c65ba6f
BLAKE2b-256 79fdf59eba751164fb681fb3edaa39d6ffe5fb2ff534aa67529819e14ce2bb19

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page