PINGTile

Utility to tile sonar mosaics and maps.

Table of Contents

• Installation
• Quick Start
• Input and Output Expectations
• Usage
• Parameter Reference
• Expected Output Layout
• Troubleshooting
• Upload Dataset to Roboflow

Installation

1. Install Miniforge.
2. Open the Miniforge prompt.
3. Install PINGInstaller:

   pip install pinginstaller
   

4. Install PINGTile.

   python -m pinginstaller pingtile
   

Quick Start

1. Create a working folder with:
   • a raster mosaic directory (for example, *.tif sonar files)
   • a polygon label file (for example, *.shp)
2. Use the script in Usage as your starter script.
3. Update the Parameters block values for your paths, class mapping, and tile settings.
4. Select the pingtile interpreter/environment in VS Code.
5. Run the script with F5.

Input and Output Expectations

• Sonar inputs:
  • GeoTIFF rasters (.tif or .tiff) discovered recursively under sonarDir.
• Label inputs:
  • Vector polygons from inFileMask (for example, Shapefile).
  • The class field must exist and match classFieldName.
• CRS/projection:
  • Inputs are reprojected to epsg_out for processing.
• Outputs:
  • Tiled images in images.
  • Tiled label rasters in labels.
  • Optional plots in plots.
  • Optional COCO JSON in json/_annotations.coco.json.

Usage

1. Copy the following script to some location on your computer:

'''
Copyright (c) 2025 Cameron S. Bodine
'''

#########
# Imports

import os, sys
from joblib import Parallel, delayed, cpu_count

# For Package
from pingtile.imglbl2tile import doImgLbl2tile
from pingtile.utils import mask_to_coco_json

import rasterio as rio
import json

############
# Parameters

map = r'Z:\tmp\pingtile_test\map\Model_Training_Substrate_Polygons_Export.shp'
sonarDir = r'Z:\tmp\pingtile_test\mosaic'

outDirTop = r'Z:\tmp\pingtile_test'
outName = 'Hudson'

classCrossWalk = {
    '0':0,
    'U':1,
    'G':2,
    'B_C':3,
    'B':4
}

windowSize_m = [
                (12,12),
                (18,18),
                (24,24),
                ]

windowStride = 3
classFieldName = 'Substrate_'
minArea_percent = 0.5
target_size = (512, 512) #(1024, 1024)
threadCnt = 0.75
epsg_out = 32616
doPlot = True
lbl2COCO = True
allowNoMapTiles = False  # Set True to also export sonar-covered tiles that have no map overlap.

if not os.path.exists(outDirTop):
    os.makedirs(outDirTop)


###############################################
# Specify multithreaded processing thread count
if threadCnt==0: # Use all threads
    threadCnt=cpu_count()
elif threadCnt<0: # Use all threads except threadCnt; i.e., (cpu_count + (-threadCnt))
    threadCnt=cpu_count()+threadCnt
    if threadCnt<0: # Make sure not negative
        threadCnt=1
elif threadCnt<1: # Use proportion of available threads
    threadCnt = int(cpu_count()*threadCnt)
    # Make even number
    if threadCnt % 2 == 1:
        threadCnt -= 1
else: # Use specified threadCnt if positive
    pass

if threadCnt>cpu_count(): # If more than total avail. threads, make cpu_count()
    threadCnt=cpu_count();
    print("\nWARNING: Specified more process threads then available, \nusing {} threads instead.".format(threadCnt))

print("\nUsing {} threads for processing.\n".format(threadCnt))


# Find all sonar files
sonarFiles = []
for root, dirs, files in os.walk(sonarDir):
    for file in files:
        if file.lower().endswith('.tif') or file.lower().endswith('.tiff'):
            sonarFiles.append(os.path.join(root, file))


for windowSize in windowSize_m:

    # windowStride_m = windowStride*windowSize[0]
    windowStride_m = windowStride
    # minArea = minArea_percent * windowSize[0]*windowSize[1]

    dirName = f"{windowSize[0]}_{windowSize[0]}"
    outDir = os.path.join(outDirTop, dirName)
    outSonDir = os.path.join(outDir, 'images')
    outMaskDir = os.path.join(outDir,'labels')
    pltDir = os.path.join(outDir,'plots')

    if not os.path.exists(outSonDir):
        os.makedirs(outSonDir)
        os.makedirs(outMaskDir)
        os.makedirs(pltDir)

    for sonarFile in sonarFiles:

        print(f"\nProcessing {os.path.basename(sonarFile)} with windowSize: {windowSize} and windowStride_m: {windowStride_m}...\n")

        doImgLbl2tile(inFileSonar=sonarFile,
                      inFileMask=map,
                      outDir=outDir,
                      outName=outName,
                      epsg_out=epsg_out,
                      classCrossWalk=classCrossWalk,
                      windowSize=windowSize,
                      windowStride_m=windowStride_m,
                      classFieldName=classFieldName,
                      minArea_percent=minArea_percent,
                      target_size=target_size,
                      threadCnt=threadCnt,
                      doPlot=doPlot,
                      allowNoMapTiles=allowNoMapTiles
                      )

# Convert masks to COCO format
if lbl2COCO:
    

    for windowSize in windowSize_m:

        dirName = f"{windowSize[0]}_{windowSize[0]}"
        outDir = os.path.join(outDirTop, dirName)
        outSonDir = os.path.join(outDir, 'images')
        outMaskDir = os.path.join(outDir,'labels')
        pltDir = os.path.join(outDir,'plots')
        outJsonDir = os.path.join(outDir,'json')

        if not os.path.exists(outJsonDir):
            os.makedirs(outJsonDir)

        print(f"\nConverting to COCO format for windowSize: {windowSize}...\n")

        # Get the mask files
        maskFiles = []
        for root, dirs, files in os.walk(outMaskDir):
            for file in files:
                if file.lower().endswith(('.tif', '.tiff', '.png', '.jpg', '.jpeg')):
                    maskFiles.append(os.path.join(root, file))

        maskFiles=maskFiles[:10] # Debug limit to 10 files

        # Build categories list / lookup from classCrossWalk
        # categories_info passed to mask_to_coco_json should map id -> name
        categories_info = {v: str(k) for k, v in classCrossWalk.items()}
        # COCO categories (exclude background id 0 if present)
        categories = [{"id": v, "name": str(k)} for k, v in classCrossWalk.items() if v != 0]

        coco = {
            "info": {"description": outName or ""},
            "licenses": [],
            "images": [],
            "annotations": [],
            "categories": categories
        }

        annotation_id = 1
        image_id = 1

        for mask_path in maskFiles:
            base = os.path.splitext(os.path.basename(mask_path))[0]

            # try to find corresponding image filename in images folder (same base name)
            matched_image = None
            for ext in ('.png', '.jpg', '.jpeg', '.tif', '.tiff'):
                candidate = os.path.join(outSonDir, base + ext)
                if os.path.exists(candidate):
                    matched_image = os.path.basename(candidate)
                    break
            if matched_image is None:
                # fallback to mask basename (acceptable as file_name in COCO)
                matched_image = os.path.basename(mask_path)

            # read mask to get width/height
            try:
                with rio.open(mask_path) as src:
                    width, height = src.width, src.height
            except Exception as e:
                print(f"Skipping {mask_path}: cannot read ({e})")
                continue

            image_info = {
                "id": image_id,
                "file_name": matched_image,
                "width": width,
                "height": height
            }

            # mask_to_coco_json should return (annotations_list, next_annotation_id)
            anns, annotation_id = mask_to_coco_json(mask_path, image_info, categories_info, annotation_id)

            if anns:
                coco["images"].append(image_info)
                coco["annotations"].extend(anns)
                image_id += 1

        out_json = os.path.join(outJsonDir, f"_annotations.coco.json")
        with open(out_json, "w") as f:
            json.dump(coco, f)

2. Open the file with Visual Studio Code.
3. Update the Parameters as necessary:

############
# Parameters

map = r'Z:\tmp\pingtile_test\map\Model_Training_Substrate_Polygons_Export.shp'
sonarDir = r'Z:\tmp\pingtile_test\mosaic'

outDirTop = r'Z:\tmp\pingtile_test'
outName = 'Hudson'

classCrossWalk = {
    '0':0,
    'U':1,
    'G':2,
    'B_C':3,
    'B':4
}

windowSize_m = [
                (12,12),
                (18,18),
                (24,24),
                ]

windowStride = 3
classFieldName = 'Substrate_'
minArea_percent = 0.5
target_size = (512, 512) #(1024, 1024)
threadCnt = 0.75
epsg_out = 32616
doPlot = True
lbl2COCO = True

4. Ensure the pingtile environment is selected as the Interpreter see this.
5. Run the script in debug mode by pressing F5.

Parameter Reference

Expected Output Layout

After running for multiple window sizes, the output directory will look similar to:

outDirTop/
    12_12/
        images/
        labels/
        plots/
        json/
            _annotations.coco.json
    18_18/
        images/
        labels/
        plots/
        json/
            _annotations.coco.json

Troubleshooting

• No tiles generated:
  • Lower minArea_percent.
  • Confirm label polygons overlap the sonar raster extent.
• Empty or sparse labels:
  • Verify classFieldName exists in the label file.
  • Confirm classCrossWalk keys match label attribute values.
• CRS mismatch symptoms (offset labels/images):
  • Ensure both raster and vector sources are georeferenced.
  • Set epsg_out to a projected CRS appropriate for your area.
• Slow execution:
  • Reduce number of windowSize_m entries.
  • Increase windowStride.
  • Tune threadCnt (for example, 0.5 to 0.9).
• GDAL/raster I/O issues:
  • Run from the pingtile conda environment.
  • Confirm rasterio and GDAL are installed from compatible channels.

Upload Dataset to Roboflow

It is possible to upload your dataset to Roboflow with the following script:

import glob
from roboflow import Roboflow

# Initialize Roboflow client
rf = Roboflow(api_key="ADD_ROBOFLOW_API_KEY_HERE") #More info: https://docs.roboflow.com/developer/authentication/find-your-roboflow-api-key

# Directory path and file extension for images
dir_name = r"Z:\tmp\pingtile_test\12_12\json"
file_extension_type = ".png"

# Annotation file path and format (e.g., .coco.json)
annotation_filename = r"Z:\tmp\pingtile_test\12_12\json\_annotations.coco.json"

# Get the upload project from Roboflow workspace
project = rf.workspace().project("ADD_PROJECT_NAME_HERE")

# Upload images
image_glob = glob.glob(dir_name + '/*' + file_extension_type)
for image_path in image_glob:
    try:
        result = project.single_upload(
            image_path=image_path,
            annotation_path=annotation_filename,
        )
        # Roboflow returns a dict; check for an error key or a successful upload URL
        if result.get("error"):
            print(f"Upload failed for {image_path}: {result['error']}")
        else:
            print(f"Uploaded {image_path} -> {result.get('image') or result}")
    except Exception as e:
        print(f"Error uploading {image_path}: {e}")