sw-inference 1.0.0

Simple Python inference SDK for ONNX models

SW Inference SDK

Lightweight Python SDK for running inference with ONNX models.

Supports: Object detection and instance segmentation models (BBox-S, BBox-M, BBox-L, Segm-S, Segm-M).

Installation

Requirements: Python ≥ 3.10

pip install sw-inference[cpu]

For GPU support:

pip install sw-inference[gpu]   # minimal
pip install sw-inference[cuda]  # with pre-installed cuda and cudnn as python package

GPU support requires CUDA 12.x and CuDNN. See here for more details. If unsure, use the extra [cuda] to install everything in the python environment. This is only valid for Linux distributions.

Quick Start

Command Line Usage

Preparation: Unzip the downloaded model.

Run inference on an image and save visualization:

python examples/basic_inference.py \
  --model-path path/to/model_dir \
  --input-image-path image.jpg \
  --output-image-path output.jpg

Python API Usage

from sw_inference import SWInference
import cv2

# Load model
model = SWInference("path/to/model_dir", device="cuda")

# Run inference
image = cv2.imread("image.jpg")
detections = model.infer_image(image)

# Access results
print(f"Found {len(detections)} objects")
for i in range(len(detections)):
    bbox = detections.xyxy[i]
    confidence = detections.confidence[i]
    class_name = model.get_class_name(detections.class_id[i])
    print(f"{class_name}: {confidence:.2f}")

Process multiple images:

for img_path in image_paths:
    image = cv2.imread(img_path)
    detections = model.infer_image(image)
    # ... process detections

Patching

Patching enables high-resolution inference by splitting large images into overlapping patches, running inference on each patch, and merging the results. This is useful when objects in your images are small relative to the image size.

Automatic Configuration

The optimal patching configuration can be automatically included in the exported model based on your validation set (or test set, if available).

Manual Configuration

If you have an existing exported model and want to add or modify patching, add a patching section to your config_export.json:

{
  "patching": {
    "enabled": true,
    "resize_to_img_size": [512, 512],
    "minimum_crop_width": 256,
    "maximum_crop_width": 1024,
    "overlap_size": 50,
    "merge_threshold": 0.5,
    "num_scales": 1,
    "overlap_filter": "NMM"
  }
}

Configuration Options

Parameter	Type	Default	Description
enabled	bool	true	Enable/disable patching
resize_to_img_size	[int, int]	[512, 512]	Size to resize each patch to [width, height]
minimum_crop_width	int	required	Minimum patch crop width
maximum_crop_width	int	required	Maximum patch crop width
overlap_size	int	50	Overlap in pixels between adjacent patches
merge_threshold	float	0.5	IoU threshold for merging overlapping detections
num_scales	int	1	Number of patching scales (higher values increase inference time)
overlap_filter	string	"NMM"	Strategy for handling overlapping detections: "NMM" or "NMS"

Development

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest -v .

# Lint and format
./scripts/format.sh

License

MIT License - see LICENSE for details.