CAPIO-CL — Cross-Application Programmable I/O Coordination Language

Platform support

OS / Arch
	YES	YES	YES
	No CI/CD support	YES	N.A.

Documentation

CAPIO-CL is a novel I/O coordination language that enables users to annotate file-based workflow data dependencies with synchronization semantics for files and directories. Designed to facilitate transparent overlap between computation and I/O operations, CAPIO-CL allows multiple producer–consumer application modules to coordinate efficiently using a JSON-based syntax.

For detailed documentation and examples, please visit:

Overview

The CAPIO Coordination Language (CAPIO-CL) allows applications to declare:

• Data objects, I/O dependencies, and access modes
• Synchronization semantics across different processes
• Commit policies for I/O objects

At runtime, CAPIO-CL’s parser and engine components analyze, track, and manage these declared relationships, enabling * transparent data sharing* and cross-application optimizations.

Building

Requirements & dependencies

• C++17 or greater
• Cmake 3.15 or newer
• Python3 to bundle CAPIO-CL json schemas into target binaries
• danielaparker/jsoncons to parse, serialize and validate CAPIO-CL JSON config files
• GoogleTest for automated testing
• pybind11 when building python wheels
• CALF for both logs and CLI messages

jsoncons, GoogleTest and pybind11 are fetched automatically by CMake — no manual setup required.

Steps

Clone
git clone https://github.com/High-Performance-IO/CAPIO-CL.git


mkdir -p CAPIO-CL/build && cd CAPIO-CL/build
cmake ..
make 

By default, this will:

• Build the "libcapio_cl" static library
• Build the "CAPIO_CL_tests" executable (GoogleTest-based)
• Build the "py_capio_cl" python bindings (pybind11)

Integration as a Subproject

CAPIO-CL can be included directly into another CMake project using:

include(FetchContent)

#####################################
# External projects
#####################################
FetchContent_Declare(
        capio_cl
        GIT_REPOSITORY https://github.com/High-Performance-IO/CAPIO-CL.git
        GIT_TAG main
)
FetchContent_MakeAvailable(capio_cl)

#####################################
# Include files and directories
#####################################
target_include_directories(${TARGET_NAME} PRIVATE
        ${capio_cl_SOURCE_DIR}
)

#####################################
# Link libraries
#####################################
target_link_libraries(${PROJECT_NAME} PRIVATE
        libcapio_cl
)

When included this way, unit tests and python bindings are not built, keeping integration clean for external projects.

Python Bindings

CAPIO-CL now provides native Python bindings built using pybind11.
These bindings expose the core C++ APIs (Engine, Parser and Serializer), directly to Python, allowing the CAPIO-CL logic to be used within python projects.

Install from PyPI

CAPIO-CL is available on PyPI! Simply run

pip install py_capio_cl

Building the Bindings

You can build and install the Python bindings directly from the CAPIO-CL source tree using:

pip install --upgrade pip
pip install -r build-requirements.txt
python -m build
pip install dst/*.whl

This will build the Python wheel and install it into your current environment using an ad-hoc build environment, which is downloaded, installed, and configured in isolation. A faster way to build and install CAPIO-CL is to use native system packages and then run from within the CAPIO-CL root directory:

pip install .

This assumes that all build dependencies not fetched by cmake are available.

API Snapshot

A simplified example of CAPIO-CL usage in C++:

#include "capiocl.hpp"

int main() {
    capiocl::Engine engine;
    engine.newFile("Hello_World.txt")
    engine.print();
    
    // Dump engine to configuration file
    capiocl::Serializer::dump(engine, "my_workflow", "my_workflow.json")
    return 0;
}

The py_capio_cl module provides access to CAPIO-CL’s core functionality through a high-level Python interface.

from py_capio_cl import Engine, Serializer

engine = Engine()
engine.newFile("Hello_World.txt")
engine.print()

# Dump engine to configuration file
Serializer.dump(engine, "my_workflow", "my_workflow.json")

Notes

• All GET endpoints expect a JSON body containing the targeted file path.
• The API is intended for local control and orchestration, not public exposure.

Developing team

Former Members