calibrate-suite 0.1.1

Comprehensive camera-lidar calibration suite with GUI and fleet management

Calibrate-Suite: Multi-Modal Sensor Calibration

A comprehensive calibration suite for LiDAR-Camera extrinsic calibration using AprilTag fiducials(for 2d-3d method) and a plane board(for 3d-3d method). Provides three calibration frameworks with CLI tools, REST API, and PyQt5 GUI for desktop and fleet-scale deployments(dummy test).

---

📋 Table of Contents

1. Project Overview
2. Features
3. Installation
4. Dependencies & Conflicts
5. Quick Start
6. Calibration Frameworks
7. Command-Line Tools
8. Data Formats
9. Cautions & Best Practices
10. Troubleshooting
11. FAQ
12. Version

---

🎯 Project Overview

Calibrate-Suite provides production-ready tools for calibrating the extrinsic transformation between LiDAR and camera sensors in robotic platforms. The package includes:

• Three calibration frameworks targeting different sensor configurations (LiDAR-only, 3D-3D, 2D-3D (AprilTag-based))
• Desktop GUI (PyQt5) for interactive calibration workflows
• REST API (Flask) for distributed fleet management dummy illustration
• Command-line tools for automated batch processing
• Multiple installation methods supporting both pip and ROS2 environments

Supports: Ubuntu 20.04+, Python 3.8+, ROS2(humble)(optional)

Hardware Setup

Pattern Generator: i used the Calib io Pattern Generator for the AprilGrid

---

✨ Features

Core Calibration

• ✅ LiDAR-Camera extrinsic calibration using AprilTag fiducials for 2D-3D and plane board for 3D-3D
• ✅ Three frameworks for different sensor setups and use cases
• ✅ Automatic corner normalization in single-pass calibration(2D-3D)
• ✅ Robust quality filtering with per-pose metrics
• ✅ Comprehensive error reporting (reprojection, translational, rotational)
• ✅ Multi-pose support (20+ poses used)

Desktop GUI

• ✅ Tab-based interface for all frameworks and tools
• ✅ Real-time data visualization with 3D point cloud overlay
• ✅ Interactive parameter tuning with live metrics
• ✅ Web-based 3D viewer for browser access(the dummy server)
• ✅ PCD file browser and upload functionality

REST API (dummy Fleet Server)

• ✅ Calibration management endpoints
• ✅ Result persistence with YAML/JSON export
• ✅ Metrics dashboard for fleet monitoring
• ✅ File upload/download for distributed workflows
• ✅ Browser-based interface for web access

Data Processing

• ✅ Automatic data extraction from ROS2 bags
• ✅ Synchronized sensor capture (camera + LiDAR)
• ✅ Point cloud filtering and preprocessing
• ✅ AprilTag detection and corner localization
• ✅ Board segmentation and quality assessment

Deployment

• ✅ pip installation for standard Python environments
• ✅ ROS2 colcon build for local deployments on ROS2
• ✅ System package compatibility (no Qt/RViz conflicts)
• ✅ Docker support ready
• ✅ Minimal dependencies with optional extras

---

📦 Installation

Prerequisites

# Python 3.8 or higher
python3 --version

# System packages (Ubuntu 22.04)
sudo apt update

Installation Options

Option 1: ROS2 + System Packages (Recommended for ROS users in development mode)

# Install system dependencies
sudo apt install python3-opencv python3-open3d python3-pyqt5 python3-yaml

# Install in development mode
pip install -e .

# Or with GUI dependencies
pip install -e .[gui]

# Or with all features
pip install -e .[full]

# Build with ROS2
cd ~/calib_ws
colcon build --packages-select calibrate-suite
source install/setup.bash

Option 2: pip - Core Only (Minimal)

pip install calibrate-suite==0.1.0

Option 3: pip - with full features

pip install calibrate-suite==0.1.0 [full]

Option 4: pip - Full Installation (Isolated venv recommended)

python3 -m venv calib_env
source calib_env/bin/activate
pip install calibrate-suite==0.1.0

# Launch commands after installation
calibrate-suite  # GUI
fleet-server      # REST API (optional parameters: --host 0.0.0.0 --port 8080 --debug)

Verification

# Check installation and verify commands are available
which calibrate-suite
which fleet-server

# Quick test of GUI
calibrate-suite --version

# Quick test of server
fleet-server --version

---

🔧 Dependencies & Conflicts

Core Dependencies (Always Required)

Package	Version	Purpose
numpy	≥1.20.0	Numerical computing
scipy	≥1.7.0	Scientific functions
scikit-learn	≥1.0.0	Machine learning algorithms
Flask	2.3.3	REST API framework
requests	2.31.0	HTTP client
python-dotenv	1.0.0	Environment configuration
pyyaml	≥6.0	YAML parsing

Optional Dependencies (Feature-Specific)

Extra	Packages	When to Use
[cv]	opencv-python ≥4.5.0	Computer vision, AprilTag detection
[3d]	open3d ≥0.13.0	3D point cloud processing
[visualization]	matplotlib ≥3.3.0	Result visualization
[gui]	PyQt5 ≥5.15.0	Desktop GUI application
[full]	All packages	Complete feature set
[dev]	pytest, black, flake8	Development & testing

Known Conflicts & Solutions

Issue: RViz snap glibc conflicts

Problem: RViz snap conflicts with system glibc (__libc_pthread_init error)
Solution: Use system RViz instead of snap
  sudo apt install ros-humble-rviz2
  # OR continue without RViz (extractor still works)

Issue: NumPy version conflicts

Problem: Mixed system (1.21.5) and pip NumPy versions causing sklearn conflicts
Solution: Use consistent package sources
  # Remove pip scientific packages
  pip uninstall numpy scipy scikit-learn open3d opencv-python
  
  # Use system packages
  sudo apt install python3-numpy python3-scipy python3-sklearn python3-opencv python3-open3d
  
  # Reinstall calibrate-suite without dependencies
  pip install -e . --no-deps
  or 
  # Remove pip-installed numpy and opencv
pip uninstall numpy opencv-python

# Use system packages
sudo apt install python3-numpy python3-opencv

Issue: OpenCV conflicts with system Qt/RViz

Problem: pip opencv-python can conflict with system OpenCV
Solution: Use system package instead
  sudo apt install python3-opencv
  pip install calibrate-suite  # without [cv]

Issue: PyQt5 conflicts in ROS2 environment

Problem: pip PyQt5 may conflict with ROS2 system packages
Solution: Use system PyQt5
  sudo apt install python3-pyqt5
  pip install calibrate-suite  # without [gui]

Issue: NumPy 2.x compatibility

Problem: Some packages compiled against NumPy 1.x
Solution: Pin NumPy version if needed
  pip install 'numpy<2' calibrate-suite[full]

---

🚀 Quick Start

GUI Application (Recommended)

# Launch desktop application
calibrate-suite

# Tabs available:
# - Home: Quick navigation
# - Recorder: Capture ROS2 bag data
# - Extractor: Extract frames and point clouds
# - Single LiDAR: LiDAR-only calibration
# - 3D-3D: Dense 3D point cloud calibration
# - 2D-3D: AprilTag-based calibration (recommended)
# - Calibrator: Advanced settings and batch processing

GUI Workflow Tabs:

Home/Record	Overlay

Command-Line (Quick Calibration)

cd /home/jamih/calib_ws/src/calibrate-suite/src

#launching calibrator suite from terminal
# Method 1: Direct Python execution
python3 -m gui.main

# Method 2: Direct script execution
python3 gui/main.py

# 2D-3D Calibration (AprilTag-based, recommended)
python3 2d_3d_calibrator.py --data_root ../calib_data/multimodal_captures

# View results
cat ../calib_data/multimodal_captures/final_extrinsic.yaml

Data Processing Pipeline

# 1. Extract synchronized data from ROS2 bag
python3 2d_3d_data_extractor.py \
  --data_root ../calib_data/multimodal_captures\
  --output ../calib_data/multimodal_captures \
  --num-poses 20

# 2. Run calibration
python3 2d_3d_calibrator.py --data_root ../calib_data/multimodal_captures

# 3. Visualize results
python3 2d_3d_overlay.py --data_root ../calib_data/multimodal_captures

---

📐 Calibration Frameworks

Framework 1: Single LiDAR Calibration

Purpose: Point cloud self-registration and intrinsic validation

Features:

• Point cloud pair registration using Point-to-Plane ICP
• FPFH-RANSAC global registration for difficult alignments
• Hybrid initialization (centroid multi-start + FPFH-RANSAC)
• Multi-scale ICP pyramid (coarse → medium → fine)
• GICP final refinement for sub-millimeter accuracy
• Statistical outlier removal
• Adaptive Voxel-based downsampling
• Fitness score and RMSE metrics

Input:

• Source and target PCD files (.pcd)

Output:

• Transformation matrix (4×4)
• Fitness score (0-1)
• RMSE error (meters)
• Aligned point cloud visualization

When to Use:

• Sensor self-calibration and intrinsic validation
• Point cloud registration debugging
• Temporal drift assessment
• LiDAR-to-LiDAR pose estimation

Note:

• when you run single Lidar capturing under bag mode, Rviz might seem unstable, just click the pause/play button, it will stabilize after that

GUI Usage:

GUI → Single LiDAR Tab → Upload PCD files → Select method → Run → View metrics

Record	Calibrate

---

Framework 2: 3D-3D Camera-LiDAR Calibration

Purpose: Dense point cloud correspondence calibration

Features:

• 3D point cloud extraction from both camera and LiDAR
• DBSCAN clustering for board segmentation
• Centroid-based initial estimation
• Least-squares iterative refinement
• Per-pose quality metrics (centroid error, normal error, point-to-plane RMSE)
• Cloud overlay visualization

Input:

• Camera point clouds (depth+image-to-3D conversion)
• LiDAR point clouds (PCD files)
• Synchronized multi-pose captures

Output:

final_extrinsic.yaml:
  rotation_matrix: 3×3
  translation_vector: [x, y, z]
  metrics:
    centroid_error_m: float
    normal_error_deg: float
    rmse_error_m: float
  per_pose_quality: [...]

When to Use:

• Both sensors provide dense 3D data
• High-precision applications required
• Full point cloud validation needed
• Stereo camera + LiDAR setups

GUI Usage:

GUI → 3D-3D Tab → Select data directory → Configure parameters → Run → View metrics

---

Framework 3: 2D-3D Camera-LiDAR Calibration (Recommended) ⭐

Purpose: AprilTag fiducial-based corner correspondence calibration (RECOMMENDED)

Features:

• AprilTag detection in camera images (6×6 grid)
• 2D corner localization in pixel coordinates
• LiDAR corner extraction via PCA and DBSCAN
• 8-permutation search for optimal corner matching
• Automatic corner index normalization in single pass
• Quality filtering (best N poses by board quality)
• Huber loss optimization with outlier rejection
• Comprehensive error metrics per pose
• Reprojection and translational error reporting
• Convergence guarantees with verification pass

Input:

• 20+ synchronized multi-pose captures
• Camera images (arbitrary resolution, e.g., 1920×1440)
• Point cloud scans (E.g., Velodyne, RoarIndar, etc.)
• AprilTag board (8×11 grid, any tag size or grid)

final_extrinsic.yaml:
  method: camera_pose_iterative_quality_filtered
  rotation_matrix: 3×3
  translation_vector: [x, y, z]
  euler_angles_deg: [roll, pitch, yaw]
  reprojection_error_metrics: {...}
  fitness_scores: {...}
  quality_distribution: {...}
  calibration_info: {...}

Output:

When to Use:

• AprilTag board is available
• Corner-based correspondences required
• Multi-pose dataset available (20+ poses)
• Need comprehensive error metrics
• Want automatic normalization and single-pass completion
• Standard camera+LiDAR robotics applications

GUI Usage:

GUI → 2D-3D Tab → Select data directory → Configure quality filter → Run → View metrics

GUI Interface	Alignment Visualization

---

💻 Command-Line Tools(without GUI)

Data Extraction Tools

# Synchronized extraction from ROS2 bag for 3d-3d mode
cd src
python3 3d_3d_data_extractor.py \ 
  --bag path-to-bag \ 
  --output ../calib_data/captures \ 
  --num-poses 20 \ 
  --sync-tolerance 100

# Extract camera images only for 2D-3D mode
python3 2d_3d_data_extractor.py \ 
  --bag path-to-bag \ 
  --type camera \ 
  --output ../calib_data/camera_images

# Extract LiDAR point clouds only for 2D-3D mode
python3 2d_3d_data_extractor.py \ 
  --bag path-to-bag \ 
  --type lidar \ 
  --output ../calib_data/lidar_clouds

# single lidar data extraction
python3 single_data_extractor.py \ 
  --bag path-to-bag \

Calibration Tools

# 2D-3D calibration (AprilTag-based, recommended)
python3 2d_3d_calibrator.py \ 
  --data_root ../calib_data/multimodal_captures \  
  --quality-filter 0.05 \ 
  --output final_extrinsic.yaml

# 3D-3D calibration (Dense point cloud)
python3 3d_3d_calibrator.py \ 
  --camera_pcds ../calib_data/camera_pcd \ 
  --lidar_pcds ../calib_data/lidar_pcd \ 
  --output final_extrinsic.yaml

# Single LiDAR registration
python3 single_lidar_calib.py \ 
  --source source.pcd \ 
  --target target.pcd \ 
  --output registration.yaml

Visualization Tools

# Generate 3d_3d mode alignment overlays
python3 3d_3d_overlay.py \ 
  --data_root ../calib_data/multimodal_captures \ 
  --output alignment_viz/

# Overlay AprilTag detection for 2d_3d
python3 2d_3d_overlay.py \ 
  --data_root ../calib_data/multimodal_captures \ 
  --pose 1

# generate/Verify corner detection in lidar and camera
python3 camtag_det.py \ 
  --data_root ../calib_data/multimodal_captures

python3 lidtag_det.py \ 
--data_root ../calib_data/multimodal_captures

To use the package camera calibrator(if you're using AprilTag Grid)

# After pip install calibrate-suite
calibrate-camera --data_root /path/to/data

#For PyPI Users (after adding entry point)
pip install calibrate-suite[cv]  # Need OpenCV
calibrate-camera --data_root ./my_data

#if you already have your camera intrinsics, make sure to select them in the GUI before running any tab execution as it has hardcoded intrinsics params(though if you capture with the GUI, the system saves camera_intrinsics.yaml once, but verify path is right in GUI)

---

📊 Data Formats

Directory Structure (Package-Created)

When using calibrate-suite, the following directory structure is created(note: they can be changed as per your wish in the GUI or through terminal by passing --data-root flag with folder path):

calib_data/
├── multimodal_captures/              # Main calibration dataset
│   ├── camera_intrinsics.yaml        # Camera K, D matrices
│   ├── final_extrinsic.yaml          # Output extrinsic transformation
│   ├── calibration_metrics.json      # Per-pose metrics
│   ├── pose_1/
│   │   ├── frame.jpg                 # Original camera image
│   │   ├── anot_image.jpg            # Annotated with corner detections
│   │   ├── board_plane.pcd           # Filtered/segmented LiDAR cloud
│   │   ├── detections.yaml           # Camera and LiDAR corner detections
│   │   └── metrics.yaml              # Per-pose quality metrics
│   ├── pose_2/
│   │   └── [same structure]
│   └── alignment_viz/                # Visualization outputs
│   |   ├── pose_1_overlay.jpg        # Corner reprojection overlay
│   |   ├── pose_1_cloud.jpg          # Point cloud projection
│   |   └── ...
|----

Camera Intrinsics Format

All calibrate-suite scripts use a standardized camera_intrinsics.yaml format. If you have your own camera intrinsics, use this exact format:

# Standard format supported across all calibrate-suite scripts
camera_matrix: [[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]]  # 3x3 intrinsic matrix
dist_coeffs: [k1, k2, p1, p2, k3]                                # Distortion coefficients
image_size: [width, height]                                      # Image dimensions in pixels
distortion_model: "plumb_bob"                                     # OpenCV distortion model
camera_name: "camera_frame"                                       # Camera frame ID (optional)

# Example with real values:
camera_matrix: [[1364.0, 0.0, 944.66], [0.0, 1363.42, 570.16], [0.0, 0.0, 1.0]]
dist_coeffs: [0.1, -0.2, 0.001, 0.002, 0.05]
image_size: [1920, 1080]
distortion_model: "plumb_bob"
camera_name: "camera_optical_frame"

Important Notes:

• File must be named exactly camera_intrinsics.yaml
• Place in your data root directory (e.g., calib_data/multimodal_captures/camera_intrinsics.yaml)
• camera_matrix must be a 3×3 nested list (not a flat array)
• dist_coeffs must be a flat list of 5 coefficients [k1, k2, p1, p2, k3]
• All extractors, calibrators, and visualization scripts use this format

Board Geometry Configuration

board_type: "AprilTag"
grid_dimensions: [8, 11]              # 8×11 tag grid, it can be as per your board
tag_size_m: 0.04                     # Individual tag width
board_width_m: 0.609                 # Total board width
board_height_m: 0.457                # Total board height
tag_spacing_m: 0.005                 # Gap between tags (if any)

---

⚠️ Cautions & Best Practices

Data Collection Best Practices

✅ DO:

• Capture 20+ poses with board at varying angles (0°-60° from camera normal)
• Ensure consistent, even lighting throughout capture sequence
• Keep AprilTag board planar and undamaged
• Verify sensor time synchronization (<100ms offset)
• Calibrate camera intrinsics separately beforehand(if not sure of its intrinsics)
• Document environmental conditions (lighting, reflectance, temperature)
• Perform test calibration with subset before full dataset

✗ DON'T:

• Use fewer than 15 poses (insufficient statistical sampling)(might be less though)
• Allow board to contact sensor brackets or cables (occlusion/deformation)
• Capture images with severe motion blur
• Mix board orientations or rotate frame mid-capture
• Have LiDAR points sparse or with extreme reflectance outliers
• Assume intrinsic parameters are accurate without verification
• Recalibrate unnecessarily (extrinsics typically remain stable)

Quality Assessment

Warning Signs:

• High standard deviation in error distribution (outliers present)
• Asymmetric error distribution (systematic bias exists)
• Rotational error > 60° (check mechanical alignment)
• All 20 poses accepted without filtering (quality filter too lenient)
• Non-identity corner permutations after normalization (check board orientation)

AprilTag Board Setup

The calibrate-suite works with any AprilTag grid configuration, not limited to 8×11:

• Grid size: Configurable (e.g., 4×4, 5×5, 8×8)
• Tag size: Any size (adjust config/board_geometry.yaml)
• Board material: Any flat surface with AprilTags
• Spacing: Regular grid required for PCA-based board frame

Board must be:

• Planar (flat, no warping)
• Rigidly mounted (no vibration)
• Visible to both camera and LiDAR simultaneously
• Not occluded or partially obscured

---

🔧 Troubleshooting

Symptom	Probable Cause	Solution
AprilTag not detected	Poor lighting or high exposure time	Improve illumination, reduce exposure, check camera focus
Few LiDAR corners detected (<2)	Low point cloud density or reflectance	Move board closer, ensure LiDAR line-of-sight, check surface reflectance
High reprojection error (>100px)	Time sync offset or incorrect intrinsics	Verify camera sync (<100ms), recalibrate camera intrinsics
Rotation error >90°	Multiple outlier poses not filtered	Increase quality_filter parameter (0.1m), manually review poses
Calibration fails to converge	Dominant outliers or bad permutation search	Reduce huber_delta (1.5px), check board orientation consistency
Corner indices not normalized	Permutation search failed to find match	Verify board geometry in config, check corner detection quality
All poses marked "poor"	Board completely out of focus or occluded	Check camera focus, ensure board visibility, verify synchronization
Memory error during processing	Large point clouds not downsampled	Reduce voxel size or enable downsampling in config

Common Command Errors

# Error: "ModuleNotFoundError: No module named 'cv2'"
# Solution: Install OpenCV
sudo apt install python3-opencv
# OR
pip install calibrate-suite[cv]

# Error: "No module named 'open3d'"
# Solution: Install Open3D
sudo apt install python3-open3d
# OR
pip install calibrate-suite[3d]

# Error: "PyQt5 import failed" (GUI)
# Solution: Install PyQt5
sudo apt install python3-pyqt5
# OR
pip install calibrate-suite[gui]

# Error: "YAML parsing failed"
# Solution: Verify YAML format and reinstall
sudo apt install python3-yaml
pip install 'pyyaml>=6.0'

REST API Server (Fleet Management)

Note: The fleet server is a dummy reference implementation for demonstration purposes. It provides basic REST endpoints for uploading calibration results from multiple robots. You can test the upload functionality from the GUI by pointing it to your own server URL.

#launching through terminal
cd /home/jamih/calib_ws/src/calibrate-suite/src
# Method 1: Using entry point (if installed)
fleet-server

# Method 2: With custom host/port
fleet-server --host 0.0.0.0 --port 8080 --debug

# Method 3: Direct Python execution
python3 -m fleet_server.app

# Method 4: Direct script execution
python3 fleet_server/app.py


# Optional parameters:
# fleet-server --host 0.0.0.0 --port 8080 --debug

# Access dashboard: http://localhost:5000
# API endpoints: http://localhost:5000/api/...

Testing from GUI:

1. In the calibrate-suite GUI, go to the Fleet tab
2. Enter your server URL (e.g., http://localhost:5000 or your custom server address)
3. Upload calibration results to your own fleet management backend

---

Performance Optimization

If processing is slow:

# Reduce point cloud density
python3 2d_3d_calibrator.py \
  --data_root ../calib_data/multimodal_captures \
  --voxel-size 0.01              # Increase for faster processing (default 0.005)
  --max-correspondence-dist 0.2  # Increase search radius

# Reduce number of poses
python3 2d_3d_calib.py \
  --data_root ../calib_data/multimodal_captures \
  --max-poses 15                 # Use best 15 poses instead of 20

---

❓ FAQ

Q: Which calibration framework should I use?
A: Framework 3 (2D-3D AprilTag-based) is recommended for standard camera+LiDAR setups. Use Framework 2 for stereo+LiDAR or dense depth+LiDAR. Use Framework 1 for LiDAR self-calibration only.

Q: Why do I need 20 poses minimum?
A: Statistical robustness and noise reduction. Fewer poses provide insufficient sampling of the transformation space and make results sensitive to outliers.

Q: Can I use a different board configuration?
A: Yes. Modify config/board_geometry.yaml with your grid size and tag dimensions. The corner detection adapts to any regular AprilTag grid.

Q: How do I integrate the calibration into my ROS2 pipeline?
A: Load final_extrinsic.yaml into your tf2 broadcaster or calibration node. See ROS2 documentation for tf2 static transforms.

Q: What if I have a different LiDAR type?
A: The point cloud processing is format-agnostic. As long as you have synchronized PCD files, the calibration works with any LiDAR (Velodyne, RoarIndar, OS1, etc.).

Q: Can I run this in Docker?
A: Yes. The package is Docker-ready. Dockerfile and compose configuration are available in the repository.

Q: How often should I recalibrate?
A: Extrinsics typically remain stable for months. Recalibrate only if: (1) sensors were physically moved, (2) mechanical stress/vibration occurred, or (3) recalibration validation fails.

Q: Does the package support real-time calibration?
A: Current implementation is batch-based. Real-time streaming calibration is planned for future releases.

Q: Can I export calibration for other tools?
A: Yes. The output is standard YAML with rotation matrix and translation vector. Compatible with ROS2 tf2, OpenCV, and most robotics frameworks.

Q: What is the typical calibration accuracy?
A: Mean reprojection error typically 30-50 px depending on sensor quality and data collection. Rotational error < 5° for rigid platforms.

Q: How do I handle multiple sensors on the same platform?
A: Run separate calibrations for each camera+LiDAR pair. Combine results using platform geometry (e.g., manual mounting offsets).

---

📄 Version

Package Version: 0.1.0
License: MIT