object-remove 0.1.0

From-scratch CPU reference implementation of a tiered on-device object-removal pipeline

object_remove

A from scratch, CPU implementation of an on device object removal pipeline (see docs/ARCHITECTURE.md for the full design): a polygon/scanline mask layer, three fill engines of increasing sophistication, two interchangeable blend backends, and a locked/queued session persistence layer.

Everything here is implemented from primitives (FAST corners, BRIEF descriptors, Hamming matching, PatchMatch, Laplacian/Poisson blending) — no OpenCV, no learned weights.

Quick start

pip install -r requirements.txt          # numpy, scipy, Pillow
PYTHONPATH=. python examples/demo.py     # writes before/after PNGs to examples/out/
PYTHONPATH=. python -m pytest -q         # 31 tests

import numpy as np
from object_remove import RemoveObjectJob, EngineTier

result = RemoveObjectJob(image_rgb).run(mask, tier=EngineTier.AUTO,
                                        progress=lambda stage, frac: ...)
removed = result.image          # float32 RGB in [0, 1]
print(result.tier_used)         # which engine AUTO picked

image_rgb is any (H, W, 3) array (uint8 or float); mask is (H, W) with nonzero marking the object to remove.

Module layout

Stage	Concern	This repo
A — mask	selection, brush, auto select	object_remove/mask/
B/D — scale/tiles	pyramid + tile scheduling	object_remove/pyramid/
C1 — legacy CPU fill	segment aware patch comparator	fillengines/patch_comparator.py
C2 — texture synthesis	self similar rigid shift	fillengines/self_similar_shift.py
C3 — patch match	multi scale PatchMatch	fillengines/patchmatch/
E — blend	multi band + Poisson	object_remove/render/
F — session	undo / persistence	object_remove/session/
— orchestration	tier selection, progress, cancellation	pipeline/remove_object_job.py

The three fill engines

• Tier 1 — SegmentAwarePatchFillEngine — greedy onion peel exemplar fill scored by segment aware SSD with byte mask validity. Cheap, correct on small holes; the low end / tiny hole fallback.
• Tier 2 — SelfSimilarShiftEngine — the fast default for repetitive texture. FAST corners + 256 bit BRIEF + Hamming matching find a rigid self similar offset; a 1D DP seam cuts the copied block in cleanly, run for x then y via a transpose trick.
• Tier 3 — PatchMatchEngine — multi scale, multi group PatchMatch inpainting (random init → propagation → random search, then patch voting reconstruction, coarse to fine). The quality path.

EngineTier.AUTO sends truly tiny holes to tier 1, otherwise tries the cheap tier 2 rigid shift first and falls back to tier 3 PatchMatch when the shift's boundary continuity score is poor.

The two blend backends

• MultiBandBlender — Laplacian pyramid blend with a fixed 4 tap kernel, [0.15439, 0.15133, 0.14252, 0.12896], and the mask feathered per pyramid level.
• PoissonBlend / PoissonBlend2 — gradient domain blend (sigma, multiplier, algo_n 1|2 variant), solved as a sparse Poisson system. Poisson is the safe default for single image inpainting because it reads only the (valid) hole boundary; the compositor repairs the background under the hole so no backend ever samples the object being removed.

Session persistence

SessionManager gives each edit a session id and an RLE mask history. Directory removal is dispatched onto a background queue and run under a lock — never an inline delete from the caller. A separate EmptySessionSweeper GCs orphaned empty session dirs left by aborted/crashed edits.

Status & scope

Implemented and tested: the full CPU pipeline across all stages. Auto select is explicitly an interface + CPU fallback: a real segmentation model can be dropped in behind AutoSelectModel, which ships a working color flood fallback so tap to select runs end to end without one.

Engines run in pure Python/numpy for clarity; they are correct, not tuned for speed. The hot inner loops (PatchMatch sweep, Poisson solve) are the obvious targets for a native/GPU port.

object_remove/
  mask/        brush_rasterizer, polygon_selection, selection_enhancer,
               connected_components, auto_select
  pyramid/     image_pyramid, scale_scheduler, tile_scheduler
  fillengines/ patch_comparator (T1), self_similar_shift (T2), patchmatch/ (T3)
  render/      multiband_blender, poisson_blender, compositor
  session/     undo_session_manager, empty_session_sweeper
  pipeline/    remove_object_job