fdq 0.0.66

ML runtime (https://pypi.org/project/fdq/)

FDQ | Fonduecaquelon

A fonduecaquelon is the heavy pot that keeps cheeses (e.g. 50% Gruyère and 50% Vacherin) melting smoothly into a perfectly blended whole — and FDQ does the same for deep learning. It keeps models, data loaders, training loops, and tools at a steady “temperature” so everything works seamlessly together, streamlining PyTorch workflows by automating repetitive tasks and providing a flexible, extensible framework for experiment management. Built for ML engineers who want to focus on experiments rather than boilerplate, FDQ lets you spend more time innovating and less time setting up.

• GitHub Repository
• PyPI Package

🚀 Features

• Minimal Boilerplate: Define only what matters — FDQ handles the rest.
• Flexible Experiment Configuration: Use JSON config files with inheritance support for easy experiment management.
• Multi-Model Support: Seamlessly manage multiple models, losses, and data loaders.
• Cluster Ready: Submit jobs to SLURM clusters with ease using built-in utilities such as automatic job resubmission.
• Extensible: Easily integrate custom models, data loaders, and training/testing loops.
• Automatic Dependency Management: Install additional pip packages per experiment.
• Distributed Training: Out-of-the-box support for PyTorch DDP.
• Model Export & Optimization: Export trained models to ONNX with optimization options.
• High-Performance Inference: TensorRT integration for GPU-accelerated inference with up to 10x speedup.
• Model Compilation: JIT tracing/scripting and torch.compile support for optimized execution.
• Interactive Model Dumping: Intuitive interface for exporting and optimizing trained models.
• Monitoring Tools: Built-in support for Weights & Biases and TensorBoard.

🛠️ Installation

Install the latest release from PyPI:

pip install fdq

If you have an NVIDIA GPU and want to run inference, install GPU dependencies:

pip install fdq[gpu]

For development and the latest features, clone the repository:

git clone https://github.com/mstadelmann/fonduecaquelon.git
cd fonduecaquelon
pip install -e .[dev,gpu]

📖 Usage

Local Experiments

All experiment parameters are defined in a config file. Config files can inherit from a parent file for easy reuse and organization.

Run an experiment locally:

fdq <path_to_config_file.json>

SLURM Cluster Execution

To run experiments on a SLURM cluster, add a slurm_cluster section to your config. See this example.

Submit your experiment:

python <path_to>/fdq_submit.py <path_to_config_file.json>

Model Export and Optimization

After training, export and optimize models for deployment:

# Interactive model dumping with export options
fdq <path_to_config_file.json> -nt -d

This launches an interactive interface where you can:

• Export to ONNX: Convert PyTorch models to ONNX format using Dynamo or TorchScript
• JIT Compilation: Trace or script models with PyTorch JIT
• TensorRT Optimization: Compile models for GPU inference with FP32, FP16, or INT8 precision
• Performance Benchmarking: Compare optimized vs. original model performance

Additional CLI Options

FDQ provides multiple command-line options:

# Run training (default)
fdq <config_file.json>

# Skip training
fdq <config_file.json> -nt

# Train and test automatically
fdq <config_file.json> -ta

# Interactive testing
fdq <config_file.json> -nt -ti

# Export and optimize models
fdq <config_file.json> -nt -d

# Run inference tests
fdq <config_file.json> -nt -i

# Print model architecture before training
fdq <config_file.json> -p

# Resume from checkpoint
fdq <config_file.json> -rp /path/to/checkpoint

🚄 Model Export & Deployment

FDQ offers full model export and optimization support for deployment:

Export Options

• ONNX Export: Convert models to ONNX for cross-platform use

  • Dynamo-based export for the latest PyTorch features
  • TorchScript export for broad compatibility
  • Automatic optimization and file size reporting

• JIT Compilation: PyTorch JIT tracing and scripting

  • Trace models for static graphs
  • Script models to preserve control flow
  • Automatic performance comparison with original models

• TensorRT Integration: GPU-accelerated inference with NVIDIA TensorRT

  • FP32, FP16, and INT8 precision
  • Automatic engine building and caching

Performance Features

• Automatic Benchmarking: Built-in performance testing with statistics
• Memory Optimization: Dynamic batch sizing and memory-efficient engines
• Cross-Platform: Compatible with various GPU architectures and CUDA versions

⚙️ Configuration Overview

FDQ uses JSON config files to define experiments. These specify models, data loaders, training/testing scripts, and cluster settings.

Models

Models are defined as dictionaries. You can use pre-installed ones (e.g. Chuchichaestli) or your own. Example:

"models": {
    "ccUNET": {
        "class_name": "chuchichaestli.models.unet.unet.UNet"
    }
}

Access models in training via experiment.models["ccUNET"]. The same structure applies to losses and data loaders.

Data Loaders

Your data loader class must implement create_datasets(experiment, args), returning:

return {
    "train_data_loader": train_loader,
    "val_data_loader": val_loader,
    "test_data_loader": test_loader,
    "n_train_samples": n_train,
    "n_val_samples": n_val,
    "n_test_samples": n_test,
    "n_train_batches": len(train_loader),
    "n_val_batches": len(val_loader) if val_loader is not None else 0,
    "n_test_batches": len(test_loader),
}

These values are available as experiment.data["<name>"].<key>.

Training Loop

Define a function in your training script:

def fdq_train(experiment: fdqExperiment):

Within it, you can access components:

nb_epochs = experiment.exp_def.train.args.epochs
data_loader = experiment.data["OXPET"].train_data_loader
model = experiment.models["ccUNET"]

See train_oxpets.py for an example.

Testing Loop

Testing is similar. Define:

def fdq_test(experiment: fdqExperiment):

See oxpets_test.py for reference.

💾 Dataset Caching

FDQ includes a dataset caching system to speed up training by caching preprocessed data to disk and loading it into RAM. See segment_pets_05_cached.json for an example.

How It Works

1. Deterministic Preprocessing & Caching: Expensive transformations (resizing, normalization, data loading) are applied once and cached as HDF5 files.
2. On-the-fly Augmentation: Fast, random augmentations (e.g. flips, rotations) are applied during training.

Configuration

Enable caching in your config:

"data": {
    "OXPET": {
        "class_name": "experiment_templates.segment_pets.oxpets_data.OxPetsData",
        "args": {
            "data_path": "/path/to/data",
            "batch_size": 8
        },
        "caching": {
            "cache_dir": "/path/to/cache",
            "shuffle_train": true,
            "shuffle_val": false,
            "shuffle_test": false
        }
    }
}

Custom Augmentations

Define augmentations:

# oxpets_augmentation.py
def augment(sample, transformers=None):
    """Apply custom augmentations to cached dataset samples."""
    sample["image"], sample["mask"] = transformers["random_vflip_sync"](
        sample["image"], sample["mask"]
    )
    return sample

Reference in your config:

"data": {
    "OXPET": {
        "caching": {
            "augmentation_script": "experiment_templates.segment_pets.oxpets_augmentation"
        }
    }
}

🖧 Distributed Training

To run with PyTorch DDP, add:

"slurm_cluster": {
    "world_size": 2,
    "cpus_per_task": 16,
    "gres": "gpu:h200sxm:2",
}

See segment_pets_03_distributed_w2.json.

Use the same number of GPUs as your world size. DDP requires more CPU cores and memory, since multiple data loaders run in parallel. It’s most beneficial for large models, as overhead is significant.

Example speedup on H200SXM GPUs:

Experiment	Time per epoch [s]
segment pets default	170
DDP with 2 GPUs	100
DDP with 4 GPUs	60

📦 Installing Additional Python Packages in SLURM

If your experiment requires extra packages, specify them in additional_pip_packages. FDQ installs them before execution.

Example:

"slurm_cluster": {
    "fdq_version": "0.0.66",
    "...": "...",
    "additional_pip_packages": [
        "monai==1.4.0",
        "prettytable"
    ]
}

🐛 Debugging

For debugging, install FDQ in development mode:

git clone https://github.com/mstadelmann/fonduecaquelon.git
cd fonduecaquelon
pip install -e .

VS Code Setup

1. Open your project in VS Code.
2. Add or update .vscode/launch.json to run run_experiment.py:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "FDQ Experiment Debug",
            "type": "debugpy",
            "request": "launch",
            "debugJustMyCode": false,
            "program": "${workspaceFolder}/src/fdq/run_experiment.py",
            "console": "integratedTerminal",
            "args": ["PATH_TO/experiment.json"],
            "cwd": "${workspaceFolder}"
        }
    ]
}

3. Debug/test your code.

📝 Tips

• Config Inheritance: Use the parent key to inherit from another config and reduce duplication.
• Multiple Models/Losses: Add multiple models and losses to config dictionaries as needed.
• Cluster Submission: fdq_submit.py handles SLURM job script generation, submission, environment setup, and result copying.
• Model Export: Use -d or --dump for interactive model export and optimization.

📚 Resources

• Experiment Templates
• Chuchichaestli Models

🤝 Contributing

Contributions are welcome! Please open issues or pull requests on GitHub.

🧀 Enjoy your Fondue!