hexdag 0.5.0.dev6

Lightweight DAG orchestration framework with enterprise pipeline capabilities

🤖 hexDAG - AI Agent Orchestration Framework

Enterprise-ready AI agent orchestration with low-code declarative workflows and powerful macro system

hexDAG revolutionizes AI development by making agent orchestration and data science workflows accessible through declarative YAML configurations, reusable macro templates, and advanced conversation patterns, while maintaining the power and flexibility needed for enterprise deployments.

✨ Why hexDAG?

Traditional AI frameworks force you to choose between simplicity and power. hexDAG delivers both through:

• 🤖 Agent-First Design: Build complex multi-agent systems with simple YAML
• 📊 Data Processing Ready: Mix AI agents with traditional data processing seamlessly
• 🌊 Real-Time Streaming: See agent thoughts and memory operations as they happen
• 🔧 Low-Code Development: Non-technical users can create sophisticated workflows
• 🏢 Enterprise Grade: Production-ready with comprehensive monitoring and control
• 🎭 Macro System: Reusable pipeline templates that expand into full workflows
• 💬 Conversation Patterns: Built-in support for multi-turn conversations with memory

🎯 The Six Pillars

1. Async-First Architecture - Non-blocking execution for maximum performance
2. Event-Driven Observability - Real-time monitoring of agent actions
3. Pydantic Validation Everywhere - Type safety and runtime validation
4. Hexagonal Architecture - Clean separation of business logic and infrastructure
5. Composable Declarative Files - Build complex workflows from simple components
6. DAG-Based Orchestration - Intelligent dependency management and parallelization

🚀 Quick Start

Installation

# Install from PyPI
pip install hexdag

# Or with uv (recommended)
uv pip install hexdag

# With optional dependencies
pip install hexdag[openai]      # OpenAI LLM support
pip install hexdag[anthropic]   # Anthropic Claude support
pip install hexdag[all]         # All optional dependencies

Development Installation

# Clone and install for development
git clone https://github.com/omniviser/hexdag.git
cd hexdag
uv sync

MCP Server for LLM Editors

hexDAG includes a built-in MCP (Model Context Protocol) server that exposes pipeline building capabilities to Claude Code, Cursor, and other LLM-powered editors:

# Development: Install MCP dependencies
uv sync --extra mcp

# Production: Install from PyPI with MCP support
uv pip install "hexdag[mcp]"

# Configure in Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json)
{
  "mcpServers": {
    "hexdag": {
      "command": "uv",
      "args": ["run", "python", "-m", "hexdag", "--mcp"]
    }
  }
}

The MCP server provides LLMs with tools to:

• List available nodes, adapters, tools, and macros from your registry
• Build and validate YAML pipelines interactively
• Get component schemas and documentation
• Auto-discover custom plugins from your pyproject.toml

Custom Adapters and Plugins

hexDAG supports three levels of component discovery:

1. Builtin - Core adapters and nodes from hexdag.builtin
2. Plugins - Community plugins from the hexdag_plugins namespace
3. User-authored - Your custom adapters and nodes

To make your custom components discoverable by MCP server, hexdag-studio, and the Python API, use the HEXDAG_PLUGIN_PATHS environment variable:

# Set custom plugin paths (colon-separated on Unix, semicolon on Windows)
export HEXDAG_PLUGIN_PATHS="./my_adapters:./my_nodes"

# Now MCP server, Studio, and API all discover your components
uv run python -m hexdag --mcp

Claude Desktop with custom plugins:

{
  "mcpServers": {
    "hexdag": {
      "command": "uv",
      "args": ["run", "python", "-m", "hexdag", "--mcp"],
      "env": {
        "HEXDAG_PLUGIN_PATHS": "/path/to/my_adapters:/path/to/my_nodes"
      }
    }
  }
}

Programmatic configuration:

from hexdag.core.discovery import set_user_plugin_paths
from pathlib import Path

# Configure custom plugin paths
set_user_plugin_paths([Path("./my_adapters"), Path("./my_nodes")])

# Now list_adapters() and list_nodes() include your components
from hexdag.api.components import list_adapters, list_nodes
adapters = list_adapters()  # Includes your custom adapters
nodes = list_nodes()        # Includes your custom nodes

See examples/mcp/ for detailed configuration guides.

Your First Agent Workflow

Create a simple AI agent workflow with YAML:

# research_agent.yaml
name: research_workflow
description: AI-powered research assistant

nodes:
  - type: agent
    id: researcher
    params:
      initial_prompt_template: "Research the topic: {{topic}}"
      max_steps: 5
      available_tools: ["web_search", "summarize"]
    depends_on: []

  - type: agent
    id: analyst
    params:
      initial_prompt_template: |
        Analyze the research findings: {{researcher.results}}
        Provide actionable insights.
      max_steps: 3
    depends_on: [researcher]

  - type: function
    id: formatter
    params:
      fn: format_report
      input_mapping:
        title: "researcher.topic"
        findings: "researcher.results"
        insights: "analyst.insights"
    depends_on: [researcher, analyst]

Run it with Python:

from hexdag import Orchestrator, YamlPipelineBuilder

# Load and execute the workflow
builder = YamlPipelineBuilder()
graph, metadata = builder.build_from_yaml_file("research_agent.yaml")

orchestrator = Orchestrator()
result = await orchestrator.run(graph, {"topic": "AI trends 2024"})

📚 Documentation & Learning

📓 Interactive Notebooks (Recommended Start)

Learn hexDAG through comprehensive, working Jupyter notebooks:

Core Concepts:

• 01. Introduction - Your first pipeline (15 min)
• 02. YAML Pipelines - Declarative workflows (25 min)
• 03. Practical Workflow - Real-world patterns (30 min)

Advanced Features:

• 06. Dynamic Reasoning Agent - Advanced agent patterns
• Advanced Few-shot & Retry - Error handling and examples
• YAML Includes & Composition - Modular pipeline composition

All notebooks execute successfully: ✅ All notebook(s) validated successfully!

📚 Complete Documentation

• 📖 Documentation Hub - Complete navigation with learning paths
• 🤔 Philosophy & Design - Six pillars and design principles
• 🔧 Implementation Guide - Production-ready workflows
• ⌨️ CLI Reference - Complete CLI documentation
• 🔌 Plugin System - Custom component development
• 🗺️ Roadmap - Future vision and features

📝 Additional Resources

• Demo Directory - Live demonstration scripts
• Integration Tests - Production test scenarios

🎪 Interactive Notebooks

Explore comprehensive Jupyter notebooks for hands-on learning:

# Start Jupyter to explore notebooks
jupyter notebook notebooks/

# Or run specific notebooks
jupyter notebook notebooks/01_introduction.ipynb           # Getting started
jupyter notebook notebooks/02_yaml_pipelines.ipynb         # YAML workflows
jupyter notebook notebooks/03_practical_workflow.ipynb     # Real-world patterns
jupyter notebook notebooks/06_dynamic_reasoning_agent.ipynb # Advanced agents

Running the Demo

# Run the startup pitch demo
uv run python examples/demo/run_demo_pitch.py

# Or explore the YAML configuration
cat examples/demo/demo_startup_pitch.yaml

🛠️ Development

# Setup development environment
uv run pre-commit install

# Run tests
uv run pytest

# Code quality checks
uv run pre-commit run --all-files

# Build documentation (via MkDocs)
uv run hexdag docs build            # Build HTML documentation
uv run hexdag docs build --clean    # Clean and rebuild
uv run hexdag docs build --strict   # Build with warnings as errors
uv run hexdag docs serve            # Live-reload dev server

🌟 Key Features

🤖 Multi-Agent Orchestration

• Sequential agent chains for complex reasoning
• Parallel specialist agents for diverse perspectives
• Hierarchical agent networks with supervisor patterns

📊 Data Processing Integration

• Mix AI agents with traditional data processing
• Real-time streaming for Jupyter notebooks
• Extensible adapter system for custom integrations

🌊 Real-Time Streaming

• Async event-driven streaming of agent actions
• Memory operation visualization
• Interactive debugging and control

🔧 Low-Code Development

• YAML-based workflow definitions
• Template system for reusable patterns
• Automatic field mapping between nodes
• Interactive Studio UI for building and testing workflows (hexdag studio)

🔄 Smart Data Mapping

• Automatic Input Mapping: Define how data flows between nodes with simple mappings
• Nested Field Extraction: Access deeply nested data with dot notation
• Type Inference: Automatic type detection from Pydantic models
• Flexible Patterns: Support for passthrough, rename, and prefixed mappings

🎭 Powerful Macro System

• Reusable Templates: Define pipeline patterns once, use everywhere
• Built-in Macros: ConversationMacro, LLMMacro, ToolMacro, ReasoningAgentMacro
• YAML Integration: Seamlessly use macros in declarative pipelines
• Dynamic Expansion: Macros expand at runtime into full DAG subgraphs
• Configuration Inheritance: Override macro defaults per invocation

🔒 Production Security

Docker Build Command

The hexdag build command generates containerized deployments from YAML pipelines.

⚠️ IMPORTANT: This command is designed for development and trusted pipelines only.

Production Safety:

# Disable build command in production environments
export HEXDAG_DISABLE_BUILD=1

For detailed documentation, including security threat model, hardening checklist, and Docker Compose patterns, see the CLI Reference.

🤝 Community

• Contributing: See CONTRIBUTING.md

📄 License

Apache License 2.0 - see LICENSE for details.

---

Built with ❤️ for the AI community by the hexDAG team