zoo-runner-common

A shared utility library for ZOO-Project CWL runners – centralizing reusable components across runners like Calrissian, Argo Workflows, and WES.

Overview

The zoo-runner-common repository provides core shared components used across multiple ZOO CWL runners. It eliminates code duplication by hosting:

• Common base class (BaseRunner) with 8+ shared methods
• Zoo-specific configuration handlers (ZooConf, ZooInputs, ZooOutputs)
• CWL workflow parsing and resource evaluation (CWLWorkflow)
• Execution handler interface (ExecutionHandler) for hooks
• Service stubs (ZooStub) to communicate with ZOO kernel

Key Benefits

• ✅ ~437 lines of code duplication eliminated across 3 runners
• ✅ Single source of truth for common functionality
• ✅ Easier maintenance - fix once, benefit everywhere
• ✅ Consistent behavior across all runners

Directory Structure

zoo-runner-common/
├── base_runner.py      # Abstract BaseRunner with common methods
├── zoo_conf.py         # ZooConf, ZooInputs, ZooOutputs, CWLWorkflow
├── handlers.py         # ExecutionHandler abstract base class
├── zoostub.py          # ZooStub for ZOO kernel communication
└── __init__.py         # Package initialization

Installation

As a dependency (recommended)

Add to your runner's pyproject.toml:

[project]
dependencies = [
    "zoo-runner-common @ git+https://github.com/ZOO-Project/zoo-runner-common.git@main",
]

Local development

export PYTHONPATH="$PYTHONPATH:/path/to/zoo-runner-common"

Or use relative imports:

import sys
sys.path.insert(0, os.path.abspath('../zoo-runner-common'))
from base_runner import BaseRunner
from zoo_conf import ZooConf, ZooInputs, ZooOutputs, CWLWorkflow
from handlers import ExecutionHandler

Components

BaseRunner (base_runner.py)

Abstract base class providing common functionality for all CWL runners:

Methods provided:

• get_workflow_id() - Get workflow identifier
• get_workflow_inputs(mandatory=False) - Get workflow input parameters
• get_max_cores() - Get maximum CPU cores from CWL
• get_max_ram() - Get maximum RAM from CWL
• get_volume_size(unit='Gi') - Calculate volume size (supports Mi/Gi)
• assert_parameters(mandatory=True) - Validate required inputs
• get_processing_parameters() - Get execution parameters
• get_namespace_name() - Generate unique namespace name
• update_status(progress, message) - Update execution status
• prepare() - Pre-execution preparation with hooks
• finalize(log, output, usage_report, tool_logs) - Post-execution finalization

Abstract methods (to implement):

• wrap() - Wrap CWL with stage-in/stage-out
• execute() - Execute the workflow

Zoo Configuration Classes (zoo_conf.py)

Key Features:

• Advanced type conversion (int, float, bool, arrays)
• File handling with format metadata
• OGC bounding box support
• NULL value handling
• Array inputs with isArray

ExecutionHandler (handlers.py)

Abstract base class for execution customization:

class ExecutionHandler(ABC):
    @abstractmethod
    def pre_execution_hook(self): pass
    
    @abstractmethod
    def post_execution_hook(self, log, output, usage_report, tool_logs): pass
    
    @abstractmethod
    def get_secrets(self): pass
    
    @abstractmethod
    def get_pod_env_vars(self): pass
    
    @abstractmethod
    def get_pod_node_selector(self): pass
    
    @abstractmethod
    def handle_outputs(self, log, output, usage_report, tool_logs): pass
    
    @abstractmethod
    def get_additional_parameters(self): pass

Usage Example

from zoo_runner_common.base_runner import BaseRunner
from zoo_runner_common.handlers import ExecutionHandler

class MyCustomRunner(BaseRunner):
    def wrap(self):
        # Implement CWL wrapping logic
        return wrapped_cwl
    
    def execute(self):
        # Prepare execution
        cwljob = self.prepare()
        
        # Execute workflow (custom logic)
        result = my_executor.run(cwljob.cwl, cwljob.params)
        
        # Finalize
        self.finalize(log, output, usage_report, tool_logs)
        return result

Runners Using zoo-runner-common

Module Reference

Used By

• zoo-wes-runner
• zoo-argowf-runner
• zoo-calrissian-runner