vision-synthesis

Production-grade 2.5D compositional scene synthesis, occlusion reasoning, and dataset engineering for Computer Vision.

🌟 Why vision-synthesis?

Most synthetic dataset generators either draw primitive geometric shapes (circles, squares) or rely on rigid 2D pasting that creates harsh borders, unnatural lighting, and incorrect ground-truth masks when objects overlap.

vision-synthesis is built from the ground up for modern vision pipelines:

1. Exact 2.5D Occlusion Reasoning: Automatically computes visible masks, trims bounding boxes, and calculates occlusion percentages when objects overlap or occlude one another.
2. Photometric Harmonization: Seamless Poisson cloning (cv2.seamlessClone), Reinhard color tone transfer, Gaussian alpha edge feathering, and directional contact drop shadows.
3. Curriculum Difficulty Engine: Quantifies scene complexity (edge clutter, occlusion ratios, contrast, object density) and generates structured datasets across easy, medium, hard, and extreme stages.
4. Dataset Integrity Auditor: Detects out-of-bounds boxes, degenerate polygons, severe class imbalances, and cross-split train/val leakage using perceptual difference hashing.
5. Universal Exporters: Instant one-click exports to COCO (instance segmentation & detection) and YOLO (YOLOv8/v11 formats with data.yaml).

📦 Installation

pip install vision-synthesis

Or install with development dependencies:

pip install vision-synthesis[dev]

🚀 Quickstart

1. Composing a Realistic 2.5D Scene

import cv2
import numpy as np
from vision_synthesis import (
    SceneComposer,
    RandomPlacementStrategy,
    PerspectivePlacementStrategy,
    render_scene_annotations,
)

# Load background plate and foreground RGBA object
background = cv2.imread("background.jpg")
foreground_rgba = cv2.imread("product_rgba.png", cv2.IMREAD_UNCHANGED)

# Configure composer with shadow casting and color tone matching
composer = SceneComposer(
    canvas_size=(640, 640),
    enable_shadows=True,
    enable_color_transfer=True,
)

# Define objects to inject with depth/scale specifications
specs = [
    {
        "image": foreground_rgba,
        "label": "bottle",
        "category_id": 1,
        "scale_range": (0.6, 1.0),
        "rotation_range": (-15.0, 15.0),
    },
    {
        "image": foreground_rgba,
        "label": "bottle",
        "category_id": 1,
        "scale_range": (0.4, 0.8),
    },
]

# Synthesize scene with ground-plane perspective
scene = composer.compose(
    background=background,
    foreground_specs=specs,
    placement_strategy=PerspectivePlacementStrategy(),
)

# Inspect visible instances and exact occlusion
for inst in scene.get_visible_instances():
    print(f"Object: {inst.label}, BBox: {inst.bbox.to_xyxy()}, Occlusion: {inst.occlusion_ratio:.1%}")

# Render visual inspection overlay
annotated_img = render_scene_annotations(scene)
cv2.imwrite("synthetic_output.jpg", annotated_img)

2. Exporting to COCO or YOLO

from vision_synthesis import CocoExporter, YoloExporter

# Export to standard COCO JSON
coco_exporter = CocoExporter(output_dir="./dataset_coco")
coco_exporter.export([scene])

# Export to YOLOv8/v11 segmentation format with data.yaml
yolo_exporter = YoloExporter(output_dir="./dataset_yolo")
yolo_exporter.export([scene], task="segment", class_names=["bottle"])

3. Auditing Dataset Quality & Leakage

from vision_synthesis import DatasetAuditor

auditor = DatasetAuditor()

# Audit annotation bounds and degenerate polygons
audit_report = auditor.audit_scene(scene)
print(f"Scene valid: {audit_report['is_valid']}, Warnings: {audit_report['warnings']}")

# Audit train/val split leakage via perceptual difference hashing
leakages = auditor.detect_leakage(train_scenes, val_scenes, max_distance=2)
if leakages:
    print(f"Warning: Found {len(leakages)} near-duplicate samples between train and validation splits!")

🏛️ Architecture Overview

src/vision_synthesis/
├── core/                  # BoundingBox, PolygonMask (Shapely boolean ops), SyntheticScene
├── composer/              # SceneComposer, Placement strategies (Perspective, Non-overlapping), OcclusionEngine
├── blending/              # PoissonBlender, Reinhard color transfer, AlphaFeather, ContactShadows
├── curriculum/            # ComplexityScorer, CurriculumScheduler (Easy -> Extreme)
├── auditor/               # Integrity validator, class balance auditor, dHash split leakage
├── exporters/             # CocoExporter, YoloExporter (detection & segmentation)
└── visualizer/            # Annotation renderer, contact sheets, comparison grids

🧪 Running Tests

pytest tests/ -v

📄 License

Licensed under the Apache License, Version 2.0.