prefect-nomad 0.2.0

Prefect integration for running workflows on HashiCorp Nomad

prefect-nomad

Prefect integration for running workflows on HashiCorp Nomad.

Overview

prefect-nomad is a Prefect collection that enables you to run Prefect flows as batch jobs on HashiCorp Nomad clusters. It provides a NomadWorker that submits flow runs as Nomad jobs using the Docker task driver.

Features

• Native Nomad Integration: Submit Prefect flow runs as Nomad batch jobs
• Docker Support: Run flows in Docker containers with configurable images
• Resource Management: Configure CPU, memory, and other resources for job execution
• TLS & Authentication: Secure connections with ACL tokens and mutual TLS
• Job Monitoring: Real-time log streaming and status tracking
• Flexible Configuration: Customize Nomad job specifications, task configurations, and scheduling

Installation

pip install prefect-nomad

Quick Start

1. Set up Nomad Credentials

Create a Prefect block to store your Nomad cluster credentials:

from prefect_nomad.credentials import NomadCredentials

# Create and save credentials
credentials = NomadCredentials(
    address="http://nomad.example.com:4646",
    token="your-nomad-acl-token",  # Optional
    namespace="default",            # Optional
    region="global"                 # Optional
)
credentials.save("my-nomad-cluster")

2. Start a Nomad Worker

Start a worker that will poll for flow runs and submit them to Nomad:

prefect worker start --pool nomad-pool --type nomad

3. Create a Deployment

Deploy your flow to run on the Nomad worker pool:

from prefect import flow

@flow
def my_flow(name: str = "world"):
    print(f"Hello, {name}!")

if __name__ == "__main__":
    my_flow.deploy(
        name="my-nomad-deployment",
        work_pool_name="nomad-pool",
        image="prefecthq/prefect:3-python3.13",
    )

4. Run Your Flow

prefect deployment run my-flow/my-nomad-deployment

Docker Usage

Pre-built Docker images with the prefect-nomad integration are available on GitHub Container Registry (GHCR). These images are built on top of the official Prefect base image and include the integration pre-installed.

Using Pre-built Images (Recommended)

Pull the latest image:

docker pull ghcr.io/ubiquitousbyte/prefect-nomad:latest

Available image variants:

# Latest version with Prefect 3 and Python 3.13 (recommended)
ghcr.io/ubiquitousbyte/prefect-nomad:latest

# Specific version with default Prefect
ghcr.io/ubiquitousbyte/prefect-nomad:0.1.0

# Specific version with explicit Python and Prefect versions
ghcr.io/ubiquitousbyte/prefect-nomad:0.1.0-py3.13-prefect3
ghcr.io/ubiquitousbyte/prefect-nomad:0.1.0-py3.13-prefect3.6.20

# Just Python version specified (uses default Prefect)
ghcr.io/ubiquitousbyte/prefect-nomad:0.1.0-py3.13

Run a Nomad worker:

docker run -it --rm \
  -e PREFECT_API_URL=https://your-prefect-server.com/api \
  ghcr.io/ubiquitousbyte/prefect-nomad:latest \
  prefect worker start --pool nomad-pool --type nomad

Run Prefect server with the integration:

docker run -d -p 4200:4200 \
  ghcr.io/ubiquitousbyte/prefect-nomad:latest \
  prefect server start

Use in deployments:

my_flow.deploy(
    name="my-nomad-deployment",
    work_pool_name="nomad-pool",
    image="ghcr.io/ubiquitousbyte/prefect-nomad:0.1.0-py3.13-prefect3",
)

Building from Source

If you need to build a custom image or test local changes:

Build the image with the default Prefect version (3-python3.13):

docker build -t prefect-nomad:latest .

Or specify a custom Prefect version:

docker build --build-arg PREFECT_VERSION=3.1-python3.13 -t prefect-nomad:3.1 .

Image Details

• Base: Official Prefect image (prefecthq/prefect)
• Integration: prefect-nomad installed from source
• Verification: Runs verify_integration.py during build to ensure proper installation
• Entrypoint: Inherits from base Prefect image (supports all Prefect CLI commands)
• Registry: GitHub Container Registry

Configuration

Worker Configuration

The Nomad worker can be configured with various options through the work pool:

from prefect.deployments import DeployConfig

my_flow.deploy(
    name="my-nomad-deployment",
    work_pool_name="nomad-pool",
    job_variables={
        "image": "my-registry/my-image:latest",
        "cpu": 500,                    # MHz
        "memory": 512,                 # MB
        "namespace": "production",
        "region": "us-east-1",
        "job_priority": 50,
        "job_timeout": 3600,          # seconds
        "env": {
            "MY_VAR": "value"
        }
    }
)

TLS Configuration

For secure connections to Nomad:

from prefect_nomad.credentials import NomadCredentials

credentials = NomadCredentials(
    address="https://nomad.example.com:4646",
    token="your-acl-token",
    tls_ca_cert="/path/to/ca.crt",
    tls_client_cert="/path/to/client.crt",
    tls_client_key="/path/to/client.key",
    tls_skip_verify=False
)
credentials.save("secure-nomad-cluster")

Architecture

The Nomad worker uses a two-phase monitoring approach:

1. Evaluation Phase: Monitors job registration and scheduling decisions

   • Validates that Nomad can schedule the job
   • Detects resource constraints or placement failures
   • Raises errors for failed evaluations

2. Allocation Phase: Monitors job execution

   • Tracks task allocation and execution
   • Streams logs in real-time
   • Reports task exit codes

Requirements

• Python 3.13+
• Prefect 3.6.21+
• HashiCorp Nomad cluster with Docker task driver enabled
• Docker images available to Nomad clients

Resources

• Prefect Documentation
• HashiCorp Nomad Documentation
• python-nomad Client

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Disclaimer

This is an independent project and is not officially associated with or endorsed by HashiCorp or Prefect Technologies, Inc.