pythermal 0.2.2

A lightweight Python library for thermal sensing and analytics on Linux platforms

🔥 PyThermal

A lightweight Python library for thermal sensing and analytics on Linux platforms (x86_64 and ARM). It provides unified APIs for recording, visualization, and intelligent analysis of thermal data from Hikvision or compatible infrared sensors.

---

🌡️ Features

• Raw Frame Recording Capture and store radiometric thermal frames (e.g., 96×96, 16-bit raw) with timestamps.

• Colored Visualization Generate pseudo-color thermal images (e.g., 240×240 RGB) with adjustable color maps.

• Live Stream Interface Stream frames in real time, perform temperature conversion and display dynamically.

• Shared Memory Architecture Efficient zero-copy access to thermal data via shared memory interface.

• Thermal Object Detection Detect objects based on temperature ranges with clustering support:

  • Temperature-based object detection (default: 31-39°C for human body)
  • Object center detection and clustering
  • Temperature statistics per detected object (min / max / avg)

• YOLO v11 Detection (Optional) Advanced object and pose detection using YOLO v11:

  • Object detection with YOLO v11 (supports default and custom thermal models)
  • Pose/keypoint detection with 17 COCO keypoints
  • Support for custom thermal-specific models
  • Real-time inference on thermal images

• Offline Replay and Analysis (Future Development) Replay recorded sessions for algorithm benchmarking or dataset generation.

---

🚀 Installation

Quick Install (Recommended)

Install from source with automatic USB setup:

git clone https://github.com/AIoT-Infrastructure/pythermal.git
cd pythermal
pip install -e .

✨ Automatic USB Setup: When you install with pip install -e ., the package automatically:

• Sets up USB device permissions (copies udev rules to /etc/udev/rules.d/)
• Adds your user to the plugdev group
• Reloads udev rules

You may be prompted for your password during installation to complete the USB setup. After installation:

• Disconnect and reconnect your thermal camera
• Log out and log back in (or restart) for group changes to take effect

Alternative: Manual USB Setup

If you prefer to set up USB permissions manually, or if automatic setup didn't work:

# After installing the package
pythermal-setup-usb

This command will prompt for your password and set up USB permissions manually.

Full Setup (Optional)

For a complete setup including system dependencies (FFmpeg libraries) and native compilation:

cd pythermal
./setup.sh

This script will:

1. Detect your system architecture (x86_64 or ARM) and install required system dependencies (FFmpeg libraries)
2. Set up USB device permissions for the thermal camera
3. Compile the native thermal recorder (pythermal-recorder) for your architecture

Note: The native binaries are already included in the package, so compilation is only needed if you want to rebuild them.

Install from PyPI

Install directly on a Linux device (x86_64 or ARM, e.g., x86_64 desktop, Jetson, OrangePi, Raspberry Pi):

uv pip install pythermal

After installation, run pythermal-setup-usb to set up USB permissions.

Optional: YOLO Detection Support

To enable YOLO v11 object and pose detection, install with the yolo extra:

uv pip install pythermal[yolo]

Or install ultralytics separately:

pip install ultralytics>=8.0.0

✅ Bundled Native Runtime The package ships with the native thermal recorder (pythermal-recorder) and required shared libraries (.so files) for both x86_64 (pythermal/_native/linux64/) and ARM (pythermal/_native/armLinux/). The library automatically detects your system architecture and uses the appropriate binaries.

---

🧠 Quick Start

1. Unified Capture Interface

The ThermalCapture class provides a unified interface for both live camera feeds and recorded sequences. It's similar to cv2.VideoCapture and automatically handles device initialization.

from pythermal import ThermalCapture

# For live camera (default)
capture = ThermalCapture()  # or ThermalCapture(0) or ThermalCapture(None)

# For recorded sequence
capture = ThermalCapture("recordings/thermal_20240101.tseq")

# Check for new frame
if capture.has_new_frame():
    # Get metadata
    metadata = capture.get_metadata()
    print(f"Frame {metadata.seq}: {metadata.min_temp:.1f}°C - {metadata.max_temp:.1f}°C")
    
    # Get YUYV frame (240x240)
    yuyv_frame = capture.get_yuyv_frame()
    
    # Get temperature array (96x96, uint16)
    temp_array = capture.get_temperature_array()
    
    # Mark frame as read
    capture.mark_frame_read()

# Cleanup
capture.release()

Or use as a context manager:

with ThermalCapture() as capture:
    if capture.has_new_frame():
        metadata = capture.get_metadata()
        yuyv_frame = capture.get_yuyv_frame()
        temp_array = capture.get_temperature_array()
        capture.mark_frame_read()
    # Automatically releases on exit

---

2. Live View

Display real-time thermal imaging feed:

from pythermal import ThermalLiveView

# For live camera (default)
viewer = ThermalLiveView()
viewer.run()  # Opens OpenCV window with live thermal feed

# For recorded sequence
viewer = ThermalLiveView("recordings/thermal_20240101.tseq")
viewer.run()  # Replays recorded sequence

Controls:

• Press q to quit
• Press t to toggle between YUYV view and temperature view
• Move mouse over image to see temperature at cursor position
• Press SPACE to pause/resume (for recorded sequences)

---

3. Record Thermal Frames

from pythermal import ThermalRecorder
import time

rec = ThermalRecorder(output_dir="recordings", color=True)
rec.start()              # Starts device and begins recording
rec.record_loop(duration=10)  # Record for 10 seconds
rec.stop()               # Stop recording

This records both:

• Raw temperature frames (96×96, uint16)
• YUYV visual frames (240×240)
• Colored RGB frames (240×240, uint8 RGB) if color=True

---

4. Access Thermal Data Directly

Using the unified ThermalCapture interface (recommended):

from pythermal import ThermalCapture

capture = ThermalCapture()  # Live camera, or pass file path for recorded data

# Check for new frame
if capture.has_new_frame():
    # Get metadata
    metadata = capture.get_metadata()
    print(f"Frame {metadata.seq}: {metadata.min_temp:.1f}°C - {metadata.max_temp:.1f}°C")
    
    # Get YUYV frame (240x240)
    yuyv_frame = capture.get_yuyv_frame()
    
    # Get temperature array (96x96, uint16)
    temp_array = capture.get_temperature_array()
    
    # Mark frame as read
    capture.mark_frame_read()

capture.release()

Advanced: Direct Shared Memory Access

For advanced use cases, you can access the shared memory interface directly:

from pythermal import ThermalDevice, ThermalSharedMemory

device = ThermalDevice()
device.start()
shm = device.get_shared_memory()

if shm.has_new_frame():
    metadata = shm.get_metadata()
    yuyv_frame = shm.get_yuyv_frame()
    temp_array = shm.get_temperature_array()
    
    # Get temperature map in Celsius (96x96, float32)
    temp_celsius = shm.get_temperature_map_celsius()
    
    shm.mark_frame_read()

device.stop()

---

5. Detect Objects in Thermal Images

Detect objects based on temperature ranges and visualize them with clustering:

from pythermal import ThermalCapture, detect_object_centers, cluster_objects

capture = ThermalCapture()  # Live camera, or pass file path for recorded data

if capture.has_new_frame():
    metadata = capture.get_metadata()
    temp_array = capture.get_temperature_array()
    
    # Detect objects in temperature range (default: 31-39°C for human body)
    objects = detect_object_centers(
        temp_array=temp_array,
        min_temp=metadata.min_temp,
        max_temp=metadata.max_temp,
        temp_min=31.0,  # Minimum temperature threshold
        temp_max=39.0,  # Maximum temperature threshold
        min_area=50     # Minimum area in pixels
    )
    
    # Cluster nearby objects together
    clusters = cluster_objects(objects, max_distance=30.0)
    
    # Each object contains: center_x, center_y, width, height, area,
    # avg_temperature, max_temperature, min_temperature
    for obj in objects:
        print(f"Object at ({obj.center_x:.1f}, {obj.center_y:.1f}): "
              f"{obj.avg_temperature:.1f}°C")
    
    capture.mark_frame_read()

capture.release()

See examples/detect_objects.py for a complete visualization example. All examples support both live camera feeds and recorded sequences using the ThermalCapture interface.

---

6. YOLO v11 Object and Pose Detection

Detect objects and human poses using YOLO v11 models:

from pythermal import ThermalCapture
from pythermal.detections.yolo import YOLOObjectDetector, YOLOPoseDetector
import cv2

capture = ThermalCapture()  # Live camera, or pass file path for recorded data

# Initialize YOLO detectors (default models auto-download on first use)
object_detector = YOLOObjectDetector(model_size="nano")  # Options: nano, small, medium, large, xlarge
pose_detector = YOLOPoseDetector(model_size="nano")

if capture.has_new_frame():
    yuyv_frame = capture.get_yuyv_frame()
    bgr_frame = cv2.cvtColor(yuyv_frame, cv2.COLOR_YUV2BGR_YUYV)
    
    # Object detection
    objects = object_detector.detect(bgr_frame)
    for obj in objects:
        print(f"Detected {obj['class_name']} with confidence {obj['confidence']:.2f}")
    
    # Pose detection
    poses = pose_detector.detect(bgr_frame)
    for pose in poses:
        print(f"Detected person with {len(pose['keypoints'])} keypoints")
    
    # Visualize
    vis_image = object_detector.visualize(bgr_frame, objects)
    # or
    vis_image = pose_detector.visualize(bgr_frame, poses)
    
    capture.mark_frame_read()

capture.release()

Using Custom Thermal Models

Place your custom YOLO v11 models (.pt files) in:

pythermal/pythermal/detections/yolo/models/

Quick usage:

# Use custom model from models directory
detector = YOLOObjectDetector(model_path="custom_thermal_object.pt")

📖 For detailed instructions on finding the models directory, uploading custom models, training, and troubleshooting, see the YOLO Detection Guide.

See examples/yolo_object_detection.py and examples/yolo_pose_detection.py for complete examples.

---

🧩 Command Line Interface

Command	Description
pythermal-preview	Live preview with temperature overlay
pythermal-setup-usb	Set up USB device permissions for thermal camera

Examples:

# Live preview
pythermal-preview

# Set up USB permissions (if not done during install)
pythermal-setup-usb

The pythermal-setup-usb command will:

• Copy udev rules to /etc/udev/rules.d/
• Add your user to the plugdev group
• Reload udev rules

After running, disconnect and reconnect your thermal camera, and log out/in for changes to take effect.

---

🧰 API Overview

Core Classes

Class	Purpose
ThermalCapture	Unified interface for live camera feeds and recorded sequences (recommended)
ThermalDevice	Manages thermal camera initialization via subprocess and shared memory access (advanced)
ThermalSharedMemory	Reads thermal data from shared memory (YUYV frames, temperature arrays, metadata) (advanced)
ThermalSequenceReader	Reads pre-recorded thermal sequences from .tseq files
ThermalRecorder	Records raw and colored frames to files
ThermalLiveView	Displays live thermal imaging feed with OpenCV
FrameMetadata	Named tuple containing frame metadata (seq, flag, dimensions, temperatures)

Detection Classes

Class	Purpose
DetectedObject	Dataclass representing a detected object with center, size, and temperature stats
BackgroundSubtractor	Background subtraction for motion detection using running average
ROI	Region of Interest definition with optional temperature thresholds
ROIManager	Manages multiple ROIs for zone monitoring and filtering
YOLOObjectDetector	YOLO v11 object detector (requires ultralytics package)
YOLOPoseDetector	YOLO v11 pose/keypoint detector (requires ultralytics package)

Detection Functions

Function	Purpose
detect_object_centers	Detect object centers from temperature map based on temperature range
detect_moving_objects	Detect moving objects using background subtraction
cluster_objects	Cluster detected objects that are close to each other

Shape Analysis Functions

Function	Purpose
calculate_aspect_ratio	Calculate aspect ratio (width/height) of detected object
calculate_compactness	Calculate compactness (circularity approximation)
calculate_circularity	Calculate true circularity from contour
calculate_convexity_ratio	Calculate convexity ratio from contour
filter_by_aspect_ratio	Filter objects by aspect ratio
filter_by_compactness	Filter objects by compactness
filter_by_area	Filter objects by area (min/max)
filter_by_shape	Filter objects by multiple shape criteria

YOLO Detection Methods

Method	Purpose
YOLOObjectDetector.detect()	Detect objects in image, returns list of detections with bbox, class, confidence
YOLOObjectDetector.visualize()	Draw bounding boxes and labels on image
YOLOPoseDetector.detect()	Detect poses/keypoints in image, returns list of poses with 17 keypoints
YOLOPoseDetector.visualize()	Draw skeleton, keypoints, and bounding boxes on image

---

🧪 Requirements

• Python ≥ 3.9
• Linux environment (x86_64 or ARM, e.g., x86_64 desktop, Jetson, OrangePi, Raspberry Pi)
• NumPy, OpenCV (auto-installed via pip)
• Thermal camera connected via USB
• Proper USB permissions (automatically set up during pip install -e ., or manually via pythermal-setup-usb)

Optional Dependencies

• ultralytics ≥ 8.0.0: Required for YOLO v11 detection features

  pip install ultralytics
  # or
  pip install pythermal[yolo]
  

---

⚙️ Architecture

Native Runtime

The library uses a native binary (pythermal-recorder) that runs as a separate process and writes thermal data to shared memory (/dev/shm/yuyv240_shm). The Python library communicates with this process via shared memory for efficient zero-copy data access.

Bundled Files

The package includes native files for both x86_64 and ARM architectures:

x86_64 (pythermal/_native/linux64/):

pythermal/_native/linux64/
├── pythermal-recorder      # Main thermal recorder executable
├── libHCUSBSDK.so            # Hikvision USB SDK library
├── libhpr.so                 # Hikvision processing library
├── libusb-1.0.so*            # USB library dependencies
└── libuvc.so                  # UVC library

ARM (pythermal/_native/armLinux/):

pythermal/_native/armLinux/
├── pythermal-recorder      # Main thermal recorder executable
├── libHCUSBSDK.so            # Hikvision USB SDK library
├── libhpr.so                 # Hikvision processing library
├── libusb-1.0.so*            # USB library dependencies
└── libuvc.so                  # UVC library

The library automatically detects your system architecture and loads the appropriate binaries.

Shared Memory Layout

The shared memory (/dev/shm/yuyv240_shm) contains:

Offset          Size            Content
0               FRAME_SZ        YUYV frame data (240×240×2 bytes)
FRAME_SZ        TEMP_DATA_SIZE  Temperature array (96×96×2 bytes, uint16)
FRAME_SZ+TEMP   ...             Metadata:
                                - seq (4 bytes, uint32)
                                - flag (4 bytes, uint32, 1=new, 0=consumed)
                                - width (4 bytes, uint32)
                                - height (4 bytes, uint32)
                                - min_temp (4 bytes, float)
                                - max_temp (4 bytes, float)
                                - avg_temp (4 bytes, float)
                                - reserved (4 bytes)

Process Management

The ThermalDevice class:

1. Starts pythermal-recorder as a subprocess
2. Waits for shared memory to become available
3. Provides access to thermal data via ThermalSharedMemory
4. Automatically cleans up the process on exit

Troubleshooting

• FileNotFoundError: pythermal-recorder not found Make sure you've run setup.sh to compile the native binaries, and that the package was installed correctly.

• PermissionError: pythermal-recorder is not executable Run chmod +x on the executable, or reinstall the package.

• TimeoutError: Shared memory did not become available

  • Check that the thermal camera is connected via USB
  • Verify USB permissions are set up correctly (run pythermal-setup-usb or setup.sh)
  • Try disconnecting and reconnecting the camera
  • Log out and log back in (or restart) after setting up USB permissions
  • Check that no other process is using the thermal camera

• RuntimeError: Thermal recorder process exited unexpectedly Check the process output for error messages. Common issues:

  • Camera not detected
  • Missing USB permissions (run pythermal-setup-usb to fix)
  • Missing shared libraries (check LD_LIBRARY_PATH)

• USB Permission Issues

  • If you get permission errors accessing the thermal camera:
    1. Run pythermal-setup-usb (or sudo pythermal-setup-usb)
    2. Disconnect and reconnect your thermal camera
    3. Log out and log back in (or restart your system)
    4. Verify with lsusb that your camera is detected

• ImportError: ultralytics package is required for YOLO detection Install the ultralytics package:

  pip install ultralytics
  # or
  pip install pythermal[yolo]
  

• FileNotFoundError: Model file not found

  • For custom models, ensure the .pt file is in pythermal/pythermal/detections/yolo/models/
  • Or provide the absolute path to the model file
  • Default models are automatically downloaded on first use (check internet connection)
  • See YOLO Detection Guide for detailed troubleshooting

---

📦 Directory Structure

pythermal/
├── pythermal/
│   ├── __init__.py
│   ├── core/                   # Core thermal camera components
│   │   ├── __init__.py
│   │   ├── device.py              # ThermalDevice class (manages subprocess)
│   │   ├── thermal_shared_memory.py  # Shared memory reader
│   │   ├── sequence_reader.py    # ThermalSequenceReader for recorded files
│   │   └── capture.py            # ThermalCapture unified interface
│   ├── record.py              # ThermalRecorder class
│   ├── live_view.py           # ThermalLiveView class
│   ├── detections/            # Object detection module
│   │   ├── __init__.py
│   │   ├── utils.py           # Shared utilities and shape analysis
│   │   ├── temperature_detection.py  # Temperature-based detection
│   │   ├── motion_detection.py       # Background subtraction and motion detection
│   │   ├── roi.py             # ROI management and zone monitoring
│   │   └── yolo/              # YOLO v11 detection module
│   │       ├── __init__.py
│   │       ├── object_detection.py   # YOLO object detection
│   │       ├── pose_detection.py    # YOLO pose detection
│   │       └── models/              # Custom thermal models directory
│   │           └── README.md         # Instructions for custom models
│   ├── usb_setup/             # USB setup scripts (included in package)
│   │   ├── setup.sh           # USB permissions setup script
│   │   ├── setup-thermal-permissions.sh
│   │   └── 99-thermal-camera.rules  # udev rules file
│   └── _native/
│       ├── linux64/           # x86_64 binaries
│       │   ├── pythermal-recorder
│       │   └── *.so            # Native libraries
│       └── armLinux/          # ARM binaries
│           ├── pythermal-recorder
│           └── *.so            # Native libraries
├── examples/                  # Example scripts
│   ├── live_view.py
│   ├── record_thermal.py
│   ├── detect_objects.py      # Object detection visualization example
│   ├── detect_motion.py       # Motion detection example
│   ├── detect_roi.py          # ROI zone monitoring example
│   ├── yolo_object_detection.py  # YOLO object detection example
│   └── yolo_pose_detection.py   # YOLO pose detection example
├── setup.sh                   # Full setup script (permissions, dependencies, compilation)
├── setup.py                   # Python package setup (includes automatic USB setup)
└── README.md

---

📚 References

@inproceedings{zeng2025thermikit,
  title={ThermiKit: Edge-Optimized LWIR Analytics with Agent-Driven Interactions},
  author={Zeng, Lan and Huang, Chunhao and Xie, Ruihan and Huang, Zhuohan and Guo, Yunqi and He, Lixing and Xie, Zhiyuan and Xing, Guoliang},
  booktitle={Proceedings of the 2025 ACM International Workshop on Thermal Sensing and Computing},
  pages={40--46},
  year={2025}
}

---

📄 License

This library is released under the Apache 2.0 License for research and non-commercial use. Only the compiled native library (.so) is shipped; no vendor source or headers are distributed.

---

💡 Acknowledgements

🏫 Developed by AIoT Lab, CUHK
📧 Device Access: [e-mail]