Auspex Seg

Instance segmentation. A polygon per object, with the class and box that identify it — trained on your own annotations.

Trained from random initialisation. No pretrained weights are downloaded and none are required, so the model you train carries nothing of anyone else's.

Install

pip install auspex-seg        # import name: auspex_seg

Wheels for Linux x86_64, Windows and macOS Apple Silicon, Python 3.10–3.13. A GPU makes training much faster, but it is not required: training and prediction both work on a CPU-only install (see Running on CPU below).

Use

from auspex_seg import Auspex

model = Auspex(labelspace="labelspace.yaml")
model.train(data="data.yaml", epochs=150)
results = model.predict("images/", save=True)   # detections carry box + polygons

model = Auspex(weights="best.pt")               # checkpoints are self-contained
model.val(data="data.yaml")

Or the console command: auspex-seg train --data data.yaml --labelspace labelspace.yaml.

Reads COCO JSON, CVAT-for-images 1.1 XML and Pascal VOC XML, mixed freely in one run. Predictions export back to CVAT or COCO for pre-labelling.

Boxes come free

You annotate polygons; you do not annotate boxes. An annotation with only segmentation is complete — the box is taken from the extent of its rings.

The box still exists inside the model, and that is deliberate rather than incidental: mask prototypes are shared across the whole image, so the only thing that makes one of them an instance is its box. Every predicted polygon therefore arrives with a box and a score alongside it, at no extra labelling cost.

A box-only source (Pascal VOC, or a COCO file without segmentation) is accepted and trains the detection half. auspex-seg warns rather than fails if no source supplies polygons at all.

Very small objects

If your objects are only a few pixels across at training resolution, the default detection grid (finest stride 8) cannot resolve them, and a mask cannot be assembled for an instance that was never located. Two approaches, in a train.yaml (micro_batch is not one of them — it is there because the finer grid needs a smaller batch to fit):

model:
  strides: [4, 8, 16, 32]   # adds a finer detection level
train:
  micro_batch: 2            # ~4x the anchors; halve the batch at 768 px
augment:
  native_crop_p: 0.5        # or: train on zoomed windows of large source images

Both are off by default, and they are trades rather than free wins. The finer level roughly doubled F1 on the smallest classes in one measured set, but long thin objects spanning much of the frame tended to fragment into several detections — and because the classification loss is normalised over every anchor, quadrupling the anchor count also pushes predicted confidence down, which hurts most when a dataset has few objects per image. On a set averaging under ten objects an image it suppressed detection entirely. native_crop_p costs no anchors and is the safer first move.

Running on CPU

Training and prediction both work on a CPU-only machine — no NVIDIA driver, a plain pip install torch. Same code path, same checkpoints; verified end to end on torch 2.13.0+cpu, including resuming a GPU-trained checkpoint on a CPU box.

It is slower, so two things are worth knowing.

img_size is the lever. Measured on a 16-core desktop CPU at micro-batch 4, seconds per optimizer step: 9.1 at 640, 5.0 at 448, 3.3 at 320. num_workers makes almost no difference (9.35 at 4 workers against 9.09 at none), and CPU bf16 autocast is 27x slower, not faster — precision resolves to fp32 on CPU automatically and should stay there. Your hardware will differ; the ratios are the useful part.

For prediction, turn off mask refinement. The refinement head runs once per detection, so it dominates CPU inference:

auspex-seg predict --weights best.pt --source images/ --device cpu --no-refine

4.01 s/image down to 0.87 s/image on the same 16-core desktop CPU, over 6 images and 1381 detections. Boxes, classes and scores come back identical. Masks soften at instance edges, and some very small instances lose their polygon altogether — 825 detections carried polygons with refinement on, 724 with it off, on that same run.

From random initialisation a useful run wants a few hundred epochs, so CPU training suits small datasets, pipeline bring-up, and machines where a GPU is not an option — it is not a substitute for one.

Data configuration

labelspace.yaml — your categories, in order; the order is the class id.

data.yaml — one entry per annotation source:

sources:
  - name: batch1
    path: annotations/batch1.json  # COCO .json | CVAT 1.1 XML | VOC xml dir
    images_root: images/
    split: train
    provides: {rect: true, polygon: true}

provides declares what a source actually labels, explicitly — it is never inferred from what happens to be in the file, because an image with zero annotations is a valid negative rather than an unlabelled one.

Sources are validated when training starts: the run prints the image count, how many of them actually carry polygons, and how many sources declared them, and warns when it finds none. That line is worth a glance before walking away from a long job — a manifest that declares polygons over box-only annotations trains a box detector and reports mask mAP of zero several hours later.

A detection's polygons list can be empty when its mask came out too small to trace, so index it defensively.

Growing a model over time: --transfer <checkpoint> continues from existing weights even when classes were added or reordered — the mask pathway is class-agnostic and transfers whole. --source-balance rebalances very unequal sources; --cache-records keeps memory flat on large datasets.

Known limits, stated up front

• RLE masks are not decoded. A COCO annotation whose segmentation is a dict is marked ignore: it trains nothing at all, neither mask nor box, and only suppresses the background signal where it sits. RLE is the native output of most auto-annotation tools, so an RLE export trains on nothing — check the parser's warning count before assuming a dataset came through intact.
• Holes train, but do not come back. A ring nested inside another IS subtracted when building the training target, so a donut trains as a donut and the predicted raw mask carries its hole. Polygon output uses only external contours, so the polygons you get back trace the outside and a donut reads as a disc. Use the mask if you need the hole.

Trial and licensing

Training is free for 30 days, in full — no key, no sign-up. The clock starts on your first training run.

Prediction, validation, calibration and export are never gated. They keep working afterwards, on any model you have already trained, forever.

After 30 days, training needs a credential:

export AUSPEX_SEG_LICENSE=AUSPEXSEG-...   # a licence key, or
export AUSPEX_SEG_LICENSE=HLN1....        # an account token
                                          # or write it to ~/.auspex-seg/license.key

Verified offline against a public key compiled into the wheel — nothing phones home, so training works on air-gapped machines. The trade is that a credential cannot be withdrawn before it expires, so they are dated.

Get one: https://heliontechltd.com/license

License

Proprietary, under an End-User License Agreement: licensed users may install and run it and train models on their own data (the resulting weights are theirs); copying, redistribution, modification and reverse engineering are prohibited. Full terms: https://heliontechltd.com/license