• HBB to OBB conversion: converts YOLO-format horizontal bounding boxes to oriented bounding boxes, in absolute px or, with --normalize, relative to [0, 1] for Ultralytics training (details).
• HBB detection: produce the horizontal boxes in the first place with any Ultralytics detector, local, from the Ultralytics catalogue, or from Hugging Face (details).
• Segmentation-based: uses state-of-the-art SAM models for accurate object boundary detection.
• Multiple model support: SAM, SAM2, SAM2.1, SAM3, Mobile SAM, and FastSAM families (details).
• Model ensemble: combine multiple models via majority voting for enhanced accuracy.
• Confidence scoring: a per-OBB quality score flags fallbacks and low-confidence conversions for triage, optionally combined with the detector confidence from the input, written as an extra column or to a side-car directory that leaves the label files standard (details).
• Polygon output: optionally save the segmentation contour behind each OBB, row-aligned with the OBB file, as a tighter object outline for downstream masking (details).
• Evaluation tools: assess OBB accuracy against ground truth on mean and median IoU, orientation error, and the share of matched boxes above IoU 0.5, 0.75, 0.85 and 0.9, overall and per class.
• Hyperparameter optimization: search SAM inference resolutions, HBB scale factors and opening kernels for the best settings on your data, one sweep at a time or a whole benchmark from a config file.
• Provenance records: --save_provenance writes the command, the versions, a digest of the source that ran and the SHA-256 of every checkpoint, so a released annotation set can be regenerated rather than trusted.
• Visualization tools: render HBBs, segmentation masks, derived contours, and resulting OBBs.
• Interactive viewer: pan and zoom over annotated frames, toggle each layer, and compare two annotation sets side by side.
• Format conversion utilities: convert between YOLO, DOTA, Pascal VOC, COCO and LabelMe annotations, in either direction, for both box kinds.

• Improved morphological operations: more advanced operations for better mask refinement, beyond the signed opening/closing kernel.
• Support for other segmentation models: extend compatibility beyond the SAM/FastSAM families.

HBB2OBB integrates with and complements several specialized tools:

• Geo-trax 🚀: georeferenced vehicle trajectory extraction pipeline for high-altitude drone imagery, built on YOLO detection and multi-object tracking. Its vehicle detector supplies the HBB inputs for vehicle use cases (car, bus, truck, motorcycle), and hbb2obb-detect runs it by default (details).

• Stabilo ⚖️: Python library for video and trajectory stabilization using robust homography transformations. Supports various feature detectors, RANSAC algorithms, and user-defined masks.

• Stabilo-Optimize 🎯: benchmarking and hyperparameter optimization framework for Stabilo. Evaluates stabilization performance through ground truth-free assessment using random perturbations.

Miniconda:

conda create -n hbb2obb python=3.11 -y
conda activate hbb2obb

uv (fastest; then uv pip install hbb2obb):

uv venv --python 3.11
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

git clone https://github.com/rfonod/hbb2obb.git
cd hbb2obb && pip install -e '.[dev]'

The -e flag makes your local changes take effect without reinstalling; the [dev] extra adds pytest and ruff. For a plain install, use pip install . instead.

Unlike other SAM models, SAM 3 weights (sam3.pt) are not auto-downloaded by Ultralytics. To use SAM 3:

1. Request access on the SAM 3 model page on Hugging Face.
2. Once approved, download sam3.pt.
3. Place sam3.pt in the models/ directory (relative to where you run the conversion).

See the Ultralytics SAM 3 documentation for more.

Run hbb2obb --help / hbb2obb-eval --help for the full list. Key conversion arguments:

• --hbb_dir / -hd: directory of HBB annotations in YOLO TXT format (default: <img_source>/../labels_hbb).
• --obb_dir / -od: directory to save OBB annotations (default: <img_source>/../labels_obb).
• --sam_models / -sm: SAM model(s) to use (e.g. sam_b, sam_l, sam2_b, sam2.1_b, sam3, mobile_sam, FastSAM-s).
• --imgsz: SAM inference resolution (default: 1280).
• --scale_factors / -sf: factor(s) to scale HBBs (single value, or two values for short/long sides).
• --opening_kernel_percentage / -okp: morphological kernel size as a fraction of the mask's smaller dimension. Positive opens (erodes then dilates, removing thin protrusions), negative closes (dilates then erodes, filling holes and rejoining a fragmented mask), 0 disables it. Closing matters because only the largest contour is kept, so a vehicle split by glare or an occluding pole loses the smaller piece otherwise.
• --save_confidence: append a per-OBB confidence score as a 10th column in the output TXT files.
• --confidence_dir / -cd: write those scores to their own directory instead, one score per line, row-aligned with the labels. Use it when the label files have to stay strictly standard, since Ultralytics and other YOLO OBB readers reject a 10th column. Give it bare for img_source/../labels_confidence.
• --save_img, --viz_dir, --show_confidence, and --hide_hbb / --hide_obb / --hide_masks / --hide_segments / --hide_class_labels: visualization controls.
• --normalize / -n: write the coordinates relative to [0, 1] instead of in absolute px, which is what Ultralytics reads. --precision / -p sets the decimals (default: 10). Applies to --save_polygon output too, so one run never mixes the two conventions.
• --device: inference device for the SAM model(s), e.g. cpu, 0, cuda:0, mps (default: Ultralytics picks).
• --model_kwargs / -k: extra Ultralytics inference kwargs as key1=value1,key2=value2.

Key evaluation arguments:

• --excluded_classes / -e: class IDs to exclude from evaluation.
• --iou_threshold / -t: IoU threshold for a match (default: 0.1).
• --class_agnostic / -ca: ignore class-label matching (useful for re-classified GT).
• --exclude_edge_cases / -exc, --edge_tolerance / -et, --img_width / -iw, --img_height / -ih: edge-case handling.
• --map_path / -mp: path to a label map YAML mapping class IDs to names.

Converting HBB to OBB: hbb2obb() processes a single image and returns the OBB annotations as a NumPy array:

from pathlib import Path
from hbb2obb.converter import hbb2obb, save_obb_annotations, save_polygon_annotations

img_path = Path("/path/to/images/img1.jpg")

# Single SAM model
obb_annotations = hbb2obb(
    img_path=img_path,
    hbb_dir="/path/to/hbb/annotations",
    sam_models="sam_b",
    imgsz=1280,
    scale_factors=0.05,
    opening_kernel_percentage=0.15,
    save_img=True,
    viz_dir="/path/to/save/visualizations",
)

# Model ensemble with per-side scale factors
obb_annotations = hbb2obb(
    img_path=img_path,
    hbb_dir="/path/to/hbb/annotations",
    sam_models=["sam_b", "sam_l", "sam2_b", "sam2.1_b"],
    scale_factors=[0.1, 0.05],  # short side / long side
)

# Writes <obb_dir>/img1.txt, deriving the filename from img_path
save_obb_annotations(obb_annotations, "/path/to/save/obb/annotations", img_path)

# Also return per-OBB confidence scores, and write them as a 10th column
obb_annotations, confidences = hbb2obb(img_path=img_path, sam_models="sam_b", return_confidence=True)
save_obb_annotations(obb_annotations, "/path/to/save/obb/annotations", img_path, confidences=confidences)

# Also return the segmentation contours, and write them as polygon annotations
obb_annotations, contours = hbb2obb(img_path=img_path, sam_models="sam_b", return_contours=True)
save_polygon_annotations(contours, obb_annotations, "/path/to/save/polygons", img_path)

Evaluating OBB predictions:

from pathlib import Path
from hbb2obb.evaluator import evaluate_obb, print_results

results = evaluate_obb(
    gt_dir=Path("/path/to/ground_truth_annotations"),
    pred_dir=Path("/path/to/predictions"),
    iou_threshold=0.1,
    class_agnostic=True,  # optional: match regardless of class label
    exclude_edge_cases=True,  # optional: drop boxes at the image edge
    img_width=3840,
    img_height=2160,
)

print_results(results, "/path/to/label_map.yaml")

Starting from HBB annotations and OBB ground truth in whatever format you were given:

# 1. Bring both into YOLO TXT, whatever they arrived as (--from is detected if omitted)
hbb2obb-convert project/voc_hbb --to yolo -o project/labels_hbb -mp project/label_map.yaml
hbb2obb-convert project/gt.json --from coco --to yolo -o project/labels_obb_gt -mp project/label_map.yaml

# 2. Optimize hyperparameters to find the best settings (add -ok to also sweep the opening kernel)
hbb2obb-optimize project/images project/labels_obb_gt -sm sam_b sam_l sam2_b -n multi_sam

# 3. Inspect the best parameters, then convert with them
cat project/benchmark_results/multi_sam/summary.txt
hbb2obb project/images --hbb_dir project/labels_hbb --obb_dir project/labels_obb \
  --sam_models sam_b sam_l --imgsz 1280 --scale_factors 0.05 \
  --opening_kernel_percentage 0.15 --save_confidence --save_provenance

# 4. Evaluate against ground truth, then look at where it went wrong
hbb2obb-eval project/labels_obb_gt project/labels_obb -mp project/label_map.yaml
hbb2obb-view project/images --compare project/labels_obb_gt --show_confidence

# 5. Ship the result in every format your consumers want
hbb2obb-convert project/labels_obb --to dota coco voc -o project/release -mp project/label_map.yaml

--model takes a registered detector (geotrax today, the default), any Ultralytics model name or .pt path, or a Hugging Face reference written as <user>/<repo>/<file>.pt. All three parts are required, even when the repo holds only one file. Weights land in models/ beside the SAM checkpoints. A registered detector brings the settings it was validated at: geotrax runs at --imgsz 1920 over its four reliable classes, while anything else starts from the Ultralytics defaults. --class_map renumbers a detector's classes to yours: --class_map '2=0,5=1,7=2,3=3' turns COCO's car, bus, truck and motorcycle into 0,1,2,3 and drops every other class.

Detected boxes are a starting point, not ground truth. If you have hand-drawn boxes already and only want the confidence a detector would give them, --merge_with keeps your geometry untouched and only attaches the score of the detection covering each box:

hbb2obb-detect /path/to/images --merge_with /path/to/labels_hbb --extras_dir /tmp/extras --overwrite

Your boxes stay exactly as they are, in their own order; a box no detection covers keeps 1.0. Detections that back no box of yours are counted and, with --extras_dir, written as their own set to review (hbb2obb-view /path/to/images --hbb_dir /tmp/extras); the merge never adds them for you. --overwrite is required before anything writes into a directory that already holds labels.

Green is the OBB, white its source HBB, red the segmentation polygon it was fitted to, orange a box flagged difficult; with --show_confidence the OBB is tinted green→red by score, the same gradient --save_img uses. The last two need the conversion to have been run with --save_polygon and --save_confidence, as the sample data in data/ was (see data/README.md for the commands behind every file there). Labels with no confidence column still color by score when the scores sit in a side-car directory: the viewer reads labels_confidence/ beside them, or wherever --confidence_dir points.

Drag with the left mouse button to pan. --crops writes a contact sheet of the individual objects instead.

--verify compares the formats by exact equality after rounding, not by a tolerance: every format is one rounding of a single canonical source, so any disagreement is a real one.

Only DOTA and Pascal VOC can express a per-box difficult flag, so writing either one from YOLO or COCO resets it; --difficult_from dota carries the flags across. --difficult_from confidence instead derives the flag from the conversion score, flagging everything below --difficult_below (fallback boxes score 0.0, so they are always flagged); the scores come from a trailing column on the source labels, or from --confidence_dir when the labels are standard ones with the scores in a side-car. The scores themselves stay out of the output. Only YOLO and COCO can carry a confidence, so DOTA and Pascal VOC drop it. Image dimensions come from --images, or from --img_width / --img_height, and are needed to denormalize relative YOLO coordinates. LabelMe stores class names rather than ids, so pass -mp when round-tripping through it to pin the ids.

A COCO file is named coco_annotations_<kind>.json unless --coco_name says otherwise, which is also how --verify pairs one with its directory: labels_<name>/ goes with coco_annotations_<name>.json beside it. Where a directory holds the canonical YOLO files next to derived ones, --from is detected as yolo.

The grid is the full product of --imgsz x --scale_factors x --opening_kernels, and each grid point is a complete SAM pass over the whole image set, so the cost multiplies quickly: the defaults (3 image sizes, 12 scale factors, 1 opening kernel) already amount to 36 passes, and sweeping three kernels instead of one triples that to 108. --opening_kernels / -ok defaults to the single value 0.15, so omitting it leaves the two-axis sweep and its grid size unchanged.

# Add the morphological kernel as a third axis (2 x 3 x 5 = 30 grid points).
# Negative values close instead of open, so one axis covers both directions with 0 in the middle.
hbb2obb-optimize /path/to/images /path/to/ground_truth -iz 960 1280 -sf 0.03 0.05 0.07 \
    -ok -0.3 -0.15 0.0 0.15 0.3

A run writes run_config.yaml, results.yaml, summary.txt and plot.png into <output_folder>/<name>, and summary.md, comparison.png and PROVENANCE.txt into the output folder itself. The plot gives each series a hue by image size and, when more than one opening kernel was swept, a lightness of that hue and a marker shape by kernel, so no two of the swept combinations share a colour. Marker area is the execution time.

--device (e.g. cpu, 0, cuda:0, mps) applies to every run and overrides any device set in the config.

Every grid point records more than the score it is ranked by, and --plot_metric / -pm chooses which of them the figures draw:

# Redraw an existing sweep against orientation error, running no SAM passes at all
hbb2obb-optimize -c benchmark.yaml --refresh --plot_metric median_angle_error

The search itself always ranks by average IoU, so a sweep stays comparable with one measured before these existed. Only the figures change. A metric knows whether higher or lower is better, so the Pareto front on comparison.png flips for an error metric, and a run plot stars the grid point that metric prefers, which is worth looking at precisely when it is not the one that won.

This matters when the scores are tight. Average IoU saturates on well-fitted boxes, so a benchmark can put every configuration inside a few thousandths of the next and still be hiding a real difference in the headings they recover.

# benchmark.yaml
img_source: data/images
gt_dir: data/labels_obb_gt
output_folder: data/benchmark_results

defaults:
  imgsz: [640, 960, 1280]
  scale_factors: [0.03, 0.04, 0.05, 0.06, 0.07]

runs:
  - sam_models: [sam_b]
  - sam_models: [sam_l]
  - sam_models: [sam_l, sam_b, sam2_b, sam2.1_b]

hbb2obb-optimize -c benchmark.yaml --dry_run   # the runs, the grid size, the total cost
hbb2obb-optimize -c benchmark.yaml --refresh   # only redraw the plots and the summary

Each run takes the defaults and overrides whatever it names; a run with no name takes one from its models, so [sam_l, sam_b] writes into sam_l-sam_b. Every sweep writes summary.md and PROVENANCE.txt, and a config-driven benchmark also leaves a copy of the configuration in the output folder, so the results re-run on their own from wherever they end up. --resume skips runs that already hold a complete grid, and PROVENANCE.txt then names the runs this invocation measured and the ones it kept. --refresh redraws the plots and the summary from the results on disk and rewrites no provenance.

data/benchmark.yaml is a working example: it is the file behind data/benchmark_results/.

# Works the same for detection
hbb2obb-detect /path/to/images --save_provenance

The record holds the exact command, the settings the run used, the versions of ultralytics, torch, OpenCV, NumPy, Shapely and matplotlib, and the SHA-256 of every checkpoint used. A benchmark also hashes the label sets its numbers were measured against.

The code is pinned three ways: the release version, the commit with git describe when there is a checkout, and a SHA-256 over the package source. The record states whether that commit can be checked out to get the code that ran, judging by the package directory alone. Use the commit to find the change; use the digest to prove you have the same code.

1. Load HBB annotations from YOLO TXT.
2. Scale bounding boxes: positive factors expand HBBs (recover cropped parts), negative factors shrink them (tighten conservative boxes); short and long sides can be scaled differently.
3. Segmentation: run SAM model(s) with the HBBs as prompts.
4. Mask aggregation: with an ensemble, combine masks by majority voting; clip to the scaled HBB region; apply morphological opening.
5. Contour extraction: extract the largest refined mask contour per object (optionally saved with --save_polygon).
6. OBB computation: fit a minimum-area oriented bounding box.
7. Fallback: if no valid mask is found inside an HBB, keep the original HBB as the OBB (confidence 0.0).
8. Confidence: score each OBB in [0, 1] (see Confidence Scores).
9. Visualization (optional): overlay HBBs, masks, contours, and OBBs (colored by confidence).

Key characteristics:

• Label preservation: OBBs inherit the class label of their source HBB (no re-classification).
• Corrective effects: the transformation can recover cropped parts (positive scaling) and produce tighter boxes through precise segmentation.