pythermal 0.2.1

A lightweight Python library for thermal sensing and analytics on Linux platforms

🔥 PyThermal

A lightweight Python library for thermal sensing and analytics on Linux platforms (x86_64 and ARM). It provides unified APIs for recording, visualization, and intelligent analysis of thermal data from Hikvision or compatible infrared sensors.

---

🌡️ Features

• Raw Frame Recording Capture and store radiometric thermal frames (e.g., 96×96, 16-bit raw) with timestamps.

• Colored Visualization Generate pseudo-color thermal images (e.g., 240×240 RGB) with adjustable color maps.

• Live Stream Interface Stream frames in real time, perform temperature conversion and display dynamically.

• Shared Memory Architecture Efficient zero-copy access to thermal data via shared memory interface.

• Thermal Object Detection Detect objects based on temperature ranges with clustering support:

  • Temperature-based object detection (default: 31-39°C for human body)
  • Object center detection and clustering
  • Temperature statistics per detected object (min / max / avg)

• YOLO v11 Detection (Optional) Advanced object and pose detection using YOLO v11:

  • Object detection with YOLO v11 (supports default and custom thermal models)
  • Pose/keypoint detection with 17 COCO keypoints
  • Support for custom thermal-specific models
  • Real-time inference on thermal images

• Offline Replay and Analysis (Future Development) Replay recorded sessions for algorithm benchmarking or dataset generation.

---

🚀 Installation

Quick Install (Recommended)

Install from source with automatic USB setup:

git clone https://github.com/AIoT-Infrastructure/pythermal.git
cd pythermal
pip install -e .

✨ Automatic USB Setup: When you install with pip install -e ., the package automatically:

• Sets up USB device permissions (copies udev rules to /etc/udev/rules.d/)
• Adds your user to the plugdev group
• Reloads udev rules

You may be prompted for your password during installation to complete the USB setup. After installation:

• Disconnect and reconnect your thermal camera
• Log out and log back in (or restart) for group changes to take effect

Alternative: Manual USB Setup

If you prefer to set up USB permissions manually, or if automatic setup didn't work:

# After installing the package
pythermal-setup-usb

This command will prompt for your password and set up USB permissions manually.

Full Setup (Optional)

For a complete setup including system dependencies (FFmpeg libraries) and native compilation:

cd pythermal
./setup.sh

This script will:

1. Detect your system architecture (x86_64 or ARM) and install required system dependencies (FFmpeg libraries)
2. Set up USB device permissions for the thermal camera
3. Compile the native thermal recorder (pythermal-recorder) for your architecture

Note: The native binaries are already included in the package, so compilation is only needed if you want to rebuild them.

Install from PyPI

Install directly on a Linux device (x86_64 or ARM, e.g., x86_64 desktop, Jetson, OrangePi, Raspberry Pi):

uv pip install pythermal

After installation, run pythermal-setup-usb to set up USB permissions.

Optional: YOLO Detection Support

To enable YOLO v11 object and pose detection, install with the yolo extra:

uv pip install pythermal[yolo]

Or install ultralytics separately:

pip install ultralytics>=8.0.0

✅ Bundled Native Runtime The package ships with the native thermal recorder (pythermal-recorder) and required shared libraries (.so files) for both x86_64 (pythermal/_native/linux64/) and ARM (pythermal/_native/armLinux/). The library automatically detects your system architecture and uses the appropriate binaries.

---

🧠 Quick Start

1. Initialize Thermal Device

The ThermalDevice class manages the thermal camera initialization by starting pythermal-recorder in a separate process and providing access to thermal data via shared memory.

from pythermal import ThermalDevice

# Create and start thermal device
device = ThermalDevice()
device.start()  # Starts pythermal-recorder subprocess and initializes shared memory

# Access shared memory for reading thermal data
shm = device.get_shared_memory()

# When done, stop the device
device.stop()

Or use as a context manager:

with ThermalDevice() as device:
    shm = device.get_shared_memory()
    # Use shared memory...
    # Device automatically stops on exit

---

2. Live View

Display real-time thermal imaging feed:

from pythermal import ThermalLiveView

viewer = ThermalLiveView()
viewer.run()  # Opens OpenCV window with live thermal feed

Or with a shared device:

from pythermal import ThermalDevice, ThermalLiveView

device = ThermalDevice()
device.start()

viewer = ThermalLiveView(device=device)
viewer.run()  # Uses the shared device

device.stop()

Controls:

• Press q to quit
• Press t to toggle between YUYV view and temperature view
• Move mouse over image to see temperature at cursor position

---

3. Record Thermal Frames

from pythermal import ThermalRecorder
import time

rec = ThermalRecorder(output_dir="recordings", color=True)
rec.start()              # Starts device and begins recording
rec.record_loop(duration=10)  # Record for 10 seconds
rec.stop()               # Stop recording

This records both:

• Raw temperature frames (96×96, uint16)
• YUYV visual frames (240×240)
• Colored RGB frames (240×240, uint8 RGB) if color=True

---

4. Access Thermal Data Directly

from pythermal import ThermalDevice, ThermalSharedMemory

device = ThermalDevice()
device.start()

shm = device.get_shared_memory()

# Check for new frame
if shm.has_new_frame():
    # Get metadata
    metadata = shm.get_metadata()
    print(f"Frame {metadata.seq}: {metadata.min_temp:.1f}°C - {metadata.max_temp:.1f}°C")
    
    # Get YUYV frame
    yuyv_frame = shm.get_yuyv_frame()
    
    # Get temperature array (96x96, uint16)
    temp_array = shm.get_temperature_array()
    
    # Get temperature map in Celsius (96x96, float32)
    temp_celsius = shm.get_temperature_map_celsius()
    
    # Mark frame as read
    shm.mark_frame_read()

device.stop()

---

5. Detect Objects in Thermal Images

Detect objects based on temperature ranges and visualize them with clustering:

from pythermal import ThermalDevice, detect_object_centers, cluster_objects

device = ThermalDevice()
device.start()
shm = device.get_shared_memory()

if shm.has_new_frame():
    metadata = shm.get_metadata()
    temp_array = shm.get_temperature_array()
    
    # Detect objects in temperature range (default: 31-39°C for human body)
    objects = detect_object_centers(
        temp_array=temp_array,
        min_temp=metadata.min_temp,
        max_temp=metadata.max_temp,
        temp_min=31.0,  # Minimum temperature threshold
        temp_max=39.0,  # Maximum temperature threshold
        min_area=50     # Minimum area in pixels
    )
    
    # Cluster nearby objects together
    clusters = cluster_objects(objects, max_distance=30.0)
    
    # Each object contains: center_x, center_y, width, height, area,
    # avg_temperature, max_temperature, min_temperature
    for obj in objects:
        print(f"Object at ({obj.center_x:.1f}, {obj.center_y:.1f}): "
              f"{obj.avg_temperature:.1f}°C")

device.stop()

See examples/detect_objects.py for a complete visualization example.

---

6. YOLO v11 Object and Pose Detection

Detect objects and human poses using YOLO v11 models:

from pythermal import ThermalDevice
from pythermal.detections.yolo import YOLOObjectDetector, YOLOPoseDetector
import cv2

device = ThermalDevice()
device.start()
shm = device.get_shared_memory()

# Initialize YOLO detectors (default models auto-download on first use)
object_detector = YOLOObjectDetector(model_size="nano")  # Options: nano, small, medium, large, xlarge
pose_detector = YOLOPoseDetector(model_size="nano")

if shm.has_new_frame():
    yuyv_frame = shm.get_yuyv_frame()
    bgr_frame = cv2.cvtColor(yuyv_frame, cv2.COLOR_YUV2BGR_YUYV)
    
    # Object detection
    objects = object_detector.detect(bgr_frame)
    for obj in objects:
        print(f"Detected {obj['class_name']} with confidence {obj['confidence']:.2f}")
    
    # Pose detection
    poses = pose_detector.detect(bgr_frame)
    for pose in poses:
        print(f"Detected person with {len(pose['keypoints'])} keypoints")
    
    # Visualize
    vis_image = object_detector.visualize(bgr_frame, objects)
    # or
    vis_image = pose_detector.visualize(bgr_frame, poses)

device.stop()

Using Custom Thermal Models

Place your custom YOLO v11 models (.pt files) in:

pythermal/pythermal/detections/yolo/models/

Quick usage:

# Use custom model from models directory
detector = YOLOObjectDetector(model_path="custom_thermal_object.pt")

📖 For detailed instructions on finding the models directory, uploading custom models, training, and troubleshooting, see the YOLO Detection Guide.

See examples/yolo_object_detection.py and examples/yolo_pose_detection.py for complete examples.

---

🧩 Command Line Interface

Command	Description
pythermal-preview	Live preview with temperature overlay
pythermal-setup-usb	Set up USB device permissions for thermal camera

Examples:

# Live preview
pythermal-preview

# Set up USB permissions (if not done during install)
pythermal-setup-usb

The pythermal-setup-usb command will:

• Copy udev rules to /etc/udev/rules.d/
• Add your user to the plugdev group
• Reload udev rules

After running, disconnect and reconnect your thermal camera, and log out/in for changes to take effect.

---

🧰 API Overview

Core Classes

Class	Purpose
ThermalDevice	Manages thermal camera initialization via subprocess and shared memory access
ThermalSharedMemory	Reads thermal data from shared memory (YUYV frames, temperature arrays, metadata)
ThermalRecorder	Records raw and colored frames to files
ThermalLiveView	Displays live thermal imaging feed with OpenCV
FrameMetadata	Named tuple containing frame metadata (seq, flag, dimensions, temperatures)

Detection Classes

Class	Purpose
DetectedObject	Dataclass representing a detected object with center, size, and temperature stats
BackgroundSubtractor	Background subtraction for motion detection using running average
ROI	Region of Interest definition with optional temperature thresholds
ROIManager	Manages multiple ROIs for zone monitoring and filtering
YOLOObjectDetector	YOLO v11 object detector (requires ultralytics package)
YOLOPoseDetector	YOLO v11 pose/keypoint detector (requires ultralytics package)

Detection Functions

Function	Purpose
detect_object_centers	Detect object centers from temperature map based on temperature range
detect_moving_objects	Detect moving objects using background subtraction
cluster_objects	Cluster detected objects that are close to each other

Shape Analysis Functions

Function	Purpose
calculate_aspect_ratio	Calculate aspect ratio (width/height) of detected object
calculate_compactness	Calculate compactness (circularity approximation)
calculate_circularity	Calculate true circularity from contour
calculate_convexity_ratio	Calculate convexity ratio from contour
filter_by_aspect_ratio	Filter objects by aspect ratio
filter_by_compactness	Filter objects by compactness
filter_by_area	Filter objects by area (min/max)
filter_by_shape	Filter objects by multiple shape criteria

YOLO Detection Methods

Method	Purpose
YOLOObjectDetector.detect()	Detect objects in image, returns list of detections with bbox, class, confidence
YOLOObjectDetector.visualize()	Draw bounding boxes and labels on image
YOLOPoseDetector.detect()	Detect poses/keypoints in image, returns list of poses with 17 keypoints
YOLOPoseDetector.visualize()	Draw skeleton, keypoints, and bounding boxes on image

---

🧪 Requirements

• Python ≥ 3.9
• Linux environment (x86_64 or ARM, e.g., x86_64 desktop, Jetson, OrangePi, Raspberry Pi)
• NumPy, OpenCV (auto-installed via pip)
• Thermal camera connected via USB
• Proper USB permissions (automatically set up during pip install -e ., or manually via pythermal-setup-usb)

Optional Dependencies

• ultralytics ≥ 8.0.0: Required for YOLO v11 detection features

  pip install ultralytics
  # or
  pip install pythermal[yolo]
  

---

⚙️ Architecture

Native Runtime

The library uses a native binary (pythermal-recorder) that runs as a separate process and writes thermal data to shared memory (/dev/shm/yuyv240_shm). The Python library communicates with this process via shared memory for efficient zero-copy data access.

Bundled Files

The package includes native files for both x86_64 and ARM architectures:

x86_64 (pythermal/_native/linux64/):

pythermal/_native/linux64/
├── pythermal-recorder      # Main thermal recorder executable
├── libHCUSBSDK.so            # Hikvision USB SDK library
├── libhpr.so                 # Hikvision processing library
├── libusb-1.0.so*            # USB library dependencies
└── libuvc.so                  # UVC library

ARM (pythermal/_native/armLinux/):

pythermal/_native/armLinux/
├── pythermal-recorder      # Main thermal recorder executable
├── libHCUSBSDK.so            # Hikvision USB SDK library
├── libhpr.so                 # Hikvision processing library
├── libusb-1.0.so*            # USB library dependencies
└── libuvc.so                  # UVC library

The library automatically detects your system architecture and loads the appropriate binaries.

Shared Memory Layout

The shared memory (/dev/shm/yuyv240_shm) contains:

Offset          Size            Content
0               FRAME_SZ        YUYV frame data (240×240×2 bytes)
FRAME_SZ        TEMP_DATA_SIZE  Temperature array (96×96×2 bytes, uint16)
FRAME_SZ+TEMP   ...             Metadata:
                                - seq (4 bytes, uint32)
                                - flag (4 bytes, uint32, 1=new, 0=consumed)
                                - width (4 bytes, uint32)
                                - height (4 bytes, uint32)
                                - min_temp (4 bytes, float)
                                - max_temp (4 bytes, float)
                                - avg_temp (4 bytes, float)
                                - reserved (4 bytes)

Process Management

The ThermalDevice class:

1. Starts pythermal-recorder as a subprocess
2. Waits for shared memory to become available
3. Provides access to thermal data via ThermalSharedMemory
4. Automatically cleans up the process on exit

Troubleshooting

• FileNotFoundError: pythermal-recorder not found Make sure you've run setup.sh to compile the native binaries, and that the package was installed correctly.

• PermissionError: pythermal-recorder is not executable Run chmod +x on the executable, or reinstall the package.

• TimeoutError: Shared memory did not become available

  • Check that the thermal camera is connected via USB
  • Verify USB permissions are set up correctly (run pythermal-setup-usb or setup.sh)
  • Try disconnecting and reconnecting the camera
  • Log out and log back in (or restart) after setting up USB permissions
  • Check that no other process is using the thermal camera

• RuntimeError: Thermal recorder process exited unexpectedly Check the process output for error messages. Common issues:

  • Camera not detected
  • Missing USB permissions (run pythermal-setup-usb to fix)
  • Missing shared libraries (check LD_LIBRARY_PATH)

• USB Permission Issues

  • If you get permission errors accessing the thermal camera:
    1. Run pythermal-setup-usb (or sudo pythermal-setup-usb)
    2. Disconnect and reconnect your thermal camera
    3. Log out and log back in (or restart your system)
    4. Verify with lsusb that your camera is detected

• ImportError: ultralytics package is required for YOLO detection Install the ultralytics package:

  pip install ultralytics
  # or
  pip install pythermal[yolo]
  

• FileNotFoundError: Model file not found

  • For custom models, ensure the .pt file is in pythermal/pythermal/detections/yolo/models/
  • Or provide the absolute path to the model file
  • Default models are automatically downloaded on first use (check internet connection)
  • See YOLO Detection Guide for detailed troubleshooting

---

📦 Directory Structure

pythermal/
├── pythermal/
│   ├── __init__.py
│   ├── device.py              # ThermalDevice class (manages subprocess)
│   ├── thermal_shared_memory.py  # Shared memory reader
│   ├── record.py              # ThermalRecorder class
│   ├── live_view.py           # ThermalLiveView class
│   ├── detections/            # Object detection module
│   │   ├── __init__.py
│   │   ├── utils.py           # Shared utilities and shape analysis
│   │   ├── temperature_detection.py  # Temperature-based detection
│   │   ├── motion_detection.py       # Background subtraction and motion detection
│   │   └── roi.py             # ROI management and zone monitoring
│   ├── usb_setup/             # USB setup scripts (included in package)
│   │   ├── setup.sh           # USB permissions setup script
│   │   ├── setup-thermal-permissions.sh
│   │   └── 99-thermal-camera.rules  # udev rules file
│   │   ├── roi.py             # ROI management and zone monitoring
│   │   └── yolo/              # YOLO v11 detection module
│   │       ├── __init__.py
│   │       ├── object_detection.py   # YOLO object detection
│   │       ├── pose_detection.py    # YOLO pose detection
│   │       └── models/              # Custom thermal models directory
│   │           └── README.md         # Instructions for custom models
│   └── _native/
│       ├── linux64/           # x86_64 binaries
│       │   ├── pythermal-recorder
│       │   └── *.so            # Native libraries
│       └── armLinux/          # ARM binaries
│           ├── pythermal-recorder
│           └── *.so            # Native libraries
├── examples/                  # Example scripts
│   ├── live_view.py
│   ├── record_thermal.py
│   ├── detect_objects.py      # Object detection visualization example
│   ├── detect_motion.py       # Motion detection example
│   └── detect_roi.py          # ROI zone monitoring example
├── setup.sh                   # Full setup script (permissions, dependencies, compilation)
├── setup.py                   # Python package setup (includes automatic USB setup)
│   ├── detect_roi.py          # ROI zone monitoring example
│   ├── yolo_object_detection.py  # YOLO object detection example
│   └── yolo_pose_detection.py   # YOLO pose detection example
├── setup.sh                   # Setup script for permissions and compilation
├── setup.py                   # Python package setup
└── README.md

---

📚 References

@inproceedings{zeng2025thermikit,
  title={ThermiKit: Edge-Optimized LWIR Analytics with Agent-Driven Interactions},
  author={Zeng, Lan and Huang, Chunhao and Xie, Ruihan and Huang, Zhuohan and Guo, Yunqi and He, Lixing and Xie, Zhiyuan and Xing, Guoliang},
  booktitle={Proceedings of the 2025 ACM International Workshop on Thermal Sensing and Computing},
  pages={40--46},
  year={2025}
}

---

📄 License

This library is released under the Apache 2.0 License for research and non-commercial use. Only the compiled native library (.so) is shipped; no vendor source or headers are distributed.

---

💡 Acknowledgements

🏫 Developed by AIoT Lab, CUHK
📧 Device Access: [e-mail]