deliapy 0.1.5

Python wrapper for the DeLIA C++ library (MPI-enabled)

DeLIApy: Python Wrapper for the DeLIA C++ Library

This document provides a complete technical guide for the use, development, and architectural understanding of DeLIApy, a high-performance wrapper developed in Cython for the DeLIA fault tolerance library.

Table of Contents

1. Overview and Documentation
2. Prerequisites
3. Installation and Quick Start
4. Implementation Examples
5. Project Structure
6. Development and Build Guide
7. Test Suite
8. Architecture and Design Decisions
9. Known Issues and Workarounds
10. Final Considerations and Next Steps

Overview and Documentation

DeLIApy is the official Python interface for DeLIA (Dependability Library for Iterative Applications), a C++ library designed to provide checkpointing and fault tolerance mechanisms in High-Performance Computing (HPC) environments.

• Important Links:
  • Original DeLIA Repository (C++)
  • DeLIA Technical Documentation

Prerequisites

To use or compile the project, the environment must meet the following specifications:

• Python: Version >= 3.11 and < 3.12.
• OpenMPI: Version 5.0.8 (expected at /opt/openmpi-5.0.8).
• Compiler: GCC/G++ with C++17 support.
• System Dependencies: Python and MPI development headers.

Installation and Quick Start

Installation via PyPI

For users who only wish to use the library in their Poetry or Pip projects:

# Via Pip
pip install deliapy

# Via Poetry
poetry add deliapy

Installing pre-releases from TestPyPI (Optional)

If you want to install from TestPyPI instead:

# Via Pip
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ deliapy

# Via Poetry
# 1. Add TestPyPI as a supplemental repository
poetry config repositories.testpypi https://test.pypi.org/legacy/

# 2. Configure the source in the project
poetry source add --priority=supplemental testpypi https://test.pypi.org/simple/

# 3. Install deliapy specifying the source
poetry add --source testpypi deliapy

Implementation Examples

Below is a practical example integrating mpi4py with DeLIApy's checkpointing and monitoring features.

import deliapy
from mpi4py import MPI
import array

# 1. Define data structure with persistence logic
class MyAppStats(deliapy.IDataLocal):
    def __init__(self, name, rank):
        self.name = name
        self.rank = rank
        self.data = array.array('d', [rank * 1.5])
        super().__init__()

    def SerializeLocal(self, ft_path: str):
        # Concatenate ft_path and the filename since ft_path is a path/filename prefix
        with open(f"{ft_path}{self.name}_{self.rank}.bin", "wb") as f:
            f.write(self.data.tobytes())

    def DeserializeLocal(self, ft_path: str) -> bool:
        # Logic to restore data after a failure
        return True

    def SerializeTasks(self, ft_path: str):
        pass

    def DeserializeAllTaks(self, ft_path: str) -> bool:
        return True

    def getName(self):
        return self.name

# 2. MPI Communicator Setup
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

# 3. DeLIA Initialization
# Arguments: rank, size, path_params, path_config, [data_global], [data_local]
stats = MyAppStats("local_stats", rank)
deliapy.DeLIA_Init(rank, size, "params.json", "config.txt", data_local=[stats])

if deliapy.DeLIA_CanWork():
    # 4. Start Heartbeat Monitoring (C++ Thread)
    deliapy.DeLIA_HeartbeatMonitoring_Init()
    
    # 5. Save settings and initial state
    deliapy.DeLIA_SaveSettings()
    deliapy.DeLIA_SaveLocalData()
    
    print(f"Rank {rank} processing...")
    
    # 6. Finalization
    deliapy.DeLIA_HeartbeatMonitoring_Finalize()
    deliapy.DeLIA_Finalize()

Project Structure

The file organization reflects the separation between Python logic, the Cython bridge, and the C++ core:

• deliapy/:
  • core.pyx: Main wrapper implementation and type conversion logic.
  • core.pxd: C++ interface declarations for Cython.
  • core.pyi: Typing stubs for IDE support.
• libs/:
  • IDataWrappers.cpp/h: C++ bridge implementation that allows the native core to call Python object methods.
• deliacpp/: Git submodule containing the original DeLIA library source code.
• tests/: Unit and MPI integration test suite.
• build.py: Custom extension compilation script.

Development and Build Guide

The project uses Poetry in conjunction with PoeThePoet to automate the C++ extension build cycle.

Environment Configuration

The project depends on the correct location of MPI shared libraries. Use the .env file to define execution paths:

cp .env.example .env
# Ensure LD_LIBRARY_PATH points to your OpenMPI installation

Local Compilation Workflow

1. Install development dependencies:

   poetry install
   

2. Compile the extension (Inplace):

   poetry run poe build-ext
   

   • This command runs build.py, which uses cythonize to generate .cpp files from .pyx and compiles the binary .so object.

CI/CD Pipeline

The project uses GitLab CI for automated linting, testing, and publishing (via a custom OpenMPI Docker environment). For complete details on the pipeline architecture, trigger matrix, and how to publish new releases, please read the CI/CD Pipeline Documentation.

Test Suite

Project validation is divided into two domains:

Serial Tests (OOP and Bindings)

Validates whether Python classes can be instantiated and if polymorphism between C++ and Python functions correctly.

poetry run poe test-serial

MPI Tests (Distributed)

Tests library behavior in a real parallel environment with 3 processes.

• test_mpi_reads.py: Verifies if global/local checkpointing restores in-memory data after intentional corruption.
• test_mpi_signals.py: Validates the interception of the SIGTSTP signal (kill -20) for automatic local state saving.
• test_mpi_hm.py: Tests Heartbeat Monitoring, simulating a "hang" in one of the processes to verify failure detection.

poetry run poe test-mpi

Architecture and Design Decisions

The primary architectural challenge of this wrapper is allowing the C++ core (which is unaware of Python) to execute methods like SerializeGlobal defined in a Python class.

IDataWrappers

To solve this, we created C++ "Wrapper" classes (IDataGlobalWrapper, etc.) that inherit from the original DeLIA abstract classes.

1. These classes store a pointer to the Python object (PyObject *py_obj).
2. When the C++ core calls SerializeGlobal, the wrapper uses the Python C API to locate the method on the Python object and execute it.
3. Memory Management: The wrapper increments the reference count (Py_XINCREF) of the Python object upon creation, ensuring it is not reclaimed by the Garbage Collector while the C++ library is using it.

Linking and RPATH

Unlike standard installations, build.py uses the runtime_library_dirs directive.

• Why?: This embeds the OpenMPI path (RPATH) directly into the .so file.
• Benefit: Users do not need to manually configure LD_LIBRARY_PATH every time they run a script, provided MPI is in the default build location.

Known Issues and Workarounds

1. Termination with Segmentation Fault (Signal 11) in MPI Tests

• Symptom: Upon finishing the distributed test suite (test-mpi), mpiexec may report a segmentation fault, even after Pytest confirms all tests passed.
• Technical Cause: This is an Exit-time Crash. It occurs due to a race condition between the Python Garbage Collector and the DeLIA (C++) background monitoring threads. During shutdown, if Heartbeat threads or C++ Signal Handlers attempt to access memory pointers that the MPI or Python runtime has already invalidated, a Signal 11 occurs.
• Impact: None. The error occurs exclusively during the final teardown phase. All checkpoint data and local states are successfully saved and validated before this fault happens.

2. ImportError: libmpi.so.12 not found

• Symptom: Failure to import deliapy.core or mpi4py stating the MPI dynamic library was not located.
• Solution: Use the .env file to correctly inject LD_LIBRARY_PATH. The build script also attempts to mitigate this by injecting RPATH flags during compilation.

Final Considerations and Next Steps

DeLIApy bridges the gap between Python's convenience and the robustness required for large-scale MPI systems.

Next Steps:

• Expansion of automated documentation (Sphinx/Doxygen).
• Support for complex multi-dimensional arrays via the buffer interface.
• Improved Python exception handling within C++ calls to prevent abrupt runtime aborts.