remback 0.0.1

Face‑aware background removal with a fine‑tuned SAM.

Remback

A Python package for removing backgrounds from profile pictures using a fine-tuned Segment Anything Model (SAM).

Installation

Installation of package

pip install remback

Manual retrieval of checkpoint

https://huggingface.co/duriantaco/remback/tree/main

Note: It should automatically download it, but if you do run into a SSL error, just manually download it from the path above.

Usage

Command-Line Interface (CLI)

Remove the background from an image:

remback --image_path /path/to/input.jpg --output_path /path/to/output.jpg --checkpoint /path/to/checkpoint.pth

• --image_path: Path to the input image (required).
• --output_path: Path to save the output image (default: output.jpg).

CLI flags

Flag	Default	Meaning
--sharp	0	Unsharp‑mask strength (0 = off)
--contrast	1.0	Multiply pixel values after cut‑out
--debug_mask	—	Path to save the binary mask for inspection

Python API

Use it in your Python scripts:

from remback.remover import BackgroundRemover

remover = BackgroundRemover()
remover.remove_background("input.jpg", "output.jpg")

Fine‑Tuning

Remback starts from Meta’s sam_vit_b checkpoint and fine‑tunes it exclusively for portrait / upper‑body shots.

Component	Status
Image encoder blocks 0‑8	Frozen
Image encoder blocks 9‑11	Trainable
Prompt encoder	Trainable
Mask decoder	Trainable

• Loss mix

Loss	Weight
Binary‑cross‑entropy	0.35
Dice	0.35
BoundaryLoss*	0.30

* BoundaryLoss drives sharper transitions by comparing Sobel edges of the logits and ground truth.

• Optimiser / schedule

• AdamW (lr 3 e‑5, weight‑decay 1 e‑4)

• mixed‑precision + GradScaler

• early‑stop on val mIoU (patience = 2 epochs)

Post‑Processing Pipeline

1. Prompt box expansion
   MTCNN face box is padded
   +120 % left/right, +5 % up, +20 % down → hair & shoulders included.

2. Raw mask threshold
   logits > 0.10 → binary mask.

3. Largest‑component keep
   Removes spurious blobs outside the subject.

4. Morphology

   • open (5×5 ellipse, 1 iter) – clears pepper noise
   • close (5×5 ellipse, 1 iter) – seals pin‑holes

5. Alpha matt­ing
   Gaussian blur (σ ≈ 0.5) then apply:

   res[alpha < 0.40] = 255
   

Comparison to Other Tools

Unlike general-purpose tools like rembg, Remback is optimized for images with faces:

1. Uses MTCNN for face detection to guide segmentation.
2. Employs custom BoundaryLoss for sharper edges around complex areas like stray hair etc.

Requirements

1. Python 3.8+
2. Dependencies (installed automatically): torch, opencv-python, numpy, mtcnn, segment-anything.

Benchmark Results

Remback

Rembg

We tested Remback against other methods. Here’s the table with mIoU and Accuracy (higher is better lah):

Method	mIoU	Accuracy
Remback	0.9584	0.9696
Original SAM	0.3864	0.5757
MTCNN	0.3164	0.4730
Rembg	0.8468	0.8841

Notes

The fine-tuned model is included in the package. If no face is detected, it will raise an error.

Acknowledgments & Licenses

Dependency	License	Notes
Segment Anything (SAM) © Meta AI	Apache 2.0	https://github.com/facebookresearch/segment-anything
MTCNN face detector	MIT	https://github.com/ipazc/mtcnn
PyTorch	BSD‑style	https://pytorch.org
OpenCV	Apache 2.0	https://opencv.org

Remback only redistributes weights you fine‑tuned yourself; the original SAM checkpoint is downloaded from the official Meta repository under Apache 2.0.

To Do

• [] Parameterise thresholds (mask_thresh, alpha_cut) via CLI/‑‑config
• [] Batch mode: accept a folder / glob, stream results
• [] Dynamic quant‑int8 checkpoint & flag --cpu_fast
• [] ONNX export script (remback.export_onnx) + doc
• [] Add hair‑refiner head (1‑layer UNet on top of SAM logits)

Citation

@misc{remback2025,
  title  = {Remback: Face‑aware background removal with a fine‑tuned SAM},
  author = {oha},
  year   = {2025},
  note   = {https://pypi.org/project/remback}
}