Upscaler

Local, open-source image upscaling + sharpening. Runs entirely on your machine (CPU, NVIDIA CUDA, or Apple-Silicon MPS) on top of pretrained Real-ESRGAN weights — no cloud, no API keys.

Upscaling, deblur, a Gradio GUI, and an ONNX backend all work end-to-end. See docs/PROJECT_NOTES.md for full planning, design decisions, the "why it can make photos worse" lesson, and the train-your-own-model playbook (incl. AMD/Windows/ROCm).

Install

Python 3.9–3.12 recommended (PyTorch wheels).

On a Windows PC with an AMD GPU? See docs/SETUP-WINDOWS-AMD.md for a full GPU-accelerated setup (WSL2 + ROCm) — dramatically faster than CPU/MPS for video.

pip install "local-upscaler[gui]"

That's it — the upscaler command is now available, and model weights download automatically (with checksum verification) the first time you use them. Extras: [gui] (GUI), [onnx] (ONNX backend), [face] (face restore), [video] (bundled ffmpeg).

git clone https://github.com/Maty3k/Upscaler.git && cd Upscaler
python -m venv .venv && source .venv/bin/activate   # .venv\Scripts\activate on Windows
pip install -e ".[gui]"     # or ".[dev]" for tests

Usage

CLI

# 4x upscale (default model)
upscaler photo.jpg -o photo_4x.png

# 2x, and apply a sharpening pass afterwards
upscaler photo.jpg --scale 2 --sharpen

# deblur motion blur (NAFNet) before upscaling
upscaler blurry.jpg --deblur --scale 4

# stronger sharpen, explicit device
upscaler photo.jpg --sharpen 1.5 --device mps

# batch a whole folder
upscaler ./input_dir -o ./output_dir --scale 4

# anime / illustration model
upscaler art.png --model realesrgan-x4plus-anime

# restore faces after upscaling (GFPGAN; needs the [face] extra)
upscaler portrait.jpg --scale 4 --face --face-strength 0.8

# upscale a video frame-by-frame (offline, keeps audio; needs ffmpeg)
upscaler video clip.mp4 -o clip_2x.mp4 --scale 2

# ONNX Runtime backend (exports once from the .pth, then torch-free + often
# faster on CPU). Works with --deblur and batching too.
upscaler photo.jpg --scale 4 --onnx

upscaler --list-models

Convert formats (no AI)

upscaler convert photo.png -o photo.webp        # format from extension
upscaler convert photo.png -f JPEG -q 80        # explicit format + quality
upscaler convert photo.png -o out.webp --lossless
upscaler convert ./folder -o ./out -f WebP      # batch a directory

Supports PNG / JPEG / WebP / AVIF / HEIC / JPEG 2000 / TIFF / GIF / BMP / ICO / ICNS / TGA / PCX / DIB / SGI / PPM (AVIF needs Pillow ≥ 11.2 or pillow-heif; HEIC needs pillow-heif). Alpha is flattened onto a white background for formats that can't store it (JPEG/BMP/PPM/PCX).

Remove background & batch

upscaler removebg photo.jpg -o cutout.png          # transparent PNG (needs [onnx])
upscaler removebg ./folder -o ./out --feather 2     # batch a directory
upscaler batch ./folder -o ./out --op upscale --scale 2     # upscale every image
upscaler batch ./folder -o ./out --op convert -f WebP       # convert every image
upscaler batch ./folder -o ./out --op removebg              # cut out every image

batch runs one operation over many images and skips unreadable files without aborting. The GUI also has Colorize (DDColor) and Inpaint / object removal (LaMa) tabs — both fully local; Colorize needs the [face] extra, Inpaint needs only torch.

Video (frame-by-frame)

upscaler video clip.mp4 -o clip_2x.mp4 --scale 2          # keeps audio
upscaler video clip.mp4 -o clip_2x_60.mp4 --scale 2 --fps 60   # + smooth to 60fps
upscaler video clip.mp4 -o clip_4k.mp4 --scale 4 --size 3840    # fit longest edge to 4K
upscaler video clip.mp4 -o test.mp4 --scale 2 --start 0 --end 5  # trim: first 5s only
upscaler video ./clips -o ./out --scale 2                 # batch a whole folder

Offline frame-by-frame upscaling (split → upscale each frame → re-encode + mux audio). --fps adds motion-interpolated frames (ffmpeg minterpolate) for smoother motion — duration unchanged, audio stays in sync, but it's slow. Needs ffmpeg (system install, or pip install -e ".[video]" for a bundled binary). It's a render-and-wait feature — minutes per minute of footage — and since frames are upscaled independently, very fine detail can shimmer slightly between frames (a temporal model would be needed to fully remove that).

Image ⇄ PDF

upscaler pdf build a.png b.png c.png -o out.pdf   # images → multi-page PDF
upscaler pdf build ./folder -o out.pdf            # all images in a directory
upscaler pdf extract in.pdf -o ./pages --dpi 200  # PDF pages → PNGs
upscaler pdf extract in.pdf                        # → ./in_pages/ next to the PDF

Weights download automatically on first use and are cached under upscaler/weights/ (override with UPSCALER_WEIGHTS_DIR).

GUI (drag-and-drop)

pip install -e ".[gui]"
python app.py            # opens a local web UI at http://127.0.0.1:7860

A full local web app with a tab per tool: Upscale & Enhance (with deblur / denoise, JPEG de-blocking, face restore), Colorize (DDColor), Remove Objects (LaMa inpainting), Remove BG, Video upscaling, Convert & Documents (formats + image ⇄ PDF), Batch, a Lian Li Screen composer for the 8.8″ case panel, and a Library of everything you export. Runs entirely on your machine — nothing is uploaded anywhere. (PDF support uses pypdfium2, included in the .[gui] extra or installable on its own via .[pdf].)

Library

from PIL import Image
from upscaler import Upscaler, enhance

# reuse one loaded model across many images
up = Upscaler(scale=4, device="auto")
up.upscale_file("in.jpg", "out.png")

# one-shot upscale + sharpen
result = enhance(Image.open("in.jpg"), scale=2, sharpen=1.0)
result.save("out.png")

How it works

• upscaler/models/rrdbnet.py — the RRDBNet generator, vendored so we don't depend on the fragile basicsr/realesrgan stack. Layer names match the official checkpoints, which load with strict=True.
• upscaler/models/registry.py + weights.py — model registry and lazy, integrity-checked weight download.
• upscaler/engine.py — device selection and tiled inference (large images are processed in padded tiles to bound memory and avoid seams).
• upscaler/models/nafnet.py + deblur.py — vendored NAFNet and the deblur stage. NAFNet's channel attention pools globally, so it runs on the whole image (not tiled) and is applied at native resolution before upscaling.
• upscaler/pipeline.py + sharpen.py — enhance(): optional deblur → upscale → optional unsharp mask.
• upscaler/onnx_export.py + onnx_engine.py — export each model to ONNX with dynamic shapes (one-time, needs torch) and run it via ONNX Runtime. The engines import only onnxruntime/numpy/Pillow, so cached .onnx files run torch-free. Verified to match the torch output (≤1/255 per pixel).

Performance notes

• CPU works but is slow on large images; keep --tile at 512 or lower.
• Apple Silicon: --device mps is much faster than CPU.
• CUDA: add --fp16 for a speed/memory win.
• AMD / Intel GPU on native Windows: torch can't reach these, but the ONNX engine can via DirectML — pip uninstall onnxruntime then pip install -e ".[directml]", and add --onnx (CLI) or tick the ONNX checkbox (GUI Upscale/Video → Advanced). For maximum AMD speed use WSL2 + ROCm instead: see docs/SETUP-WINDOWS-AMD.md.

Testing

pip install -e ".[dev]"
pytest        # architecture + tiling tests; run on CPU, no weights download

Roadmap

• Phase 0 — scaffold, packaging, license
• Phase 1 — Real-ESRGAN upscaling (lib + CLI), tiling, lazy weights, unsharp sharpen
• Phase 2 — model-based deblur stage (NAFNet) for genuinely blurry input
• Phase 3 — Gradio drag-and-drop GUI (app.py)
• Phase 4 — ONNX Runtime path for faster, PyTorch-free CPU inference (--onnx)

Licensing

This project is Apache-2.0 (see LICENSE). Pretrained weights are downloaded at runtime and never redistributed in this repo; each carries its own upstream terms. The core Real-ESRGAN weights are BSD-3-Clause, but several optional models are not: the community upscalers (4x-UltraSharp, Remacri, NMKD) and the CodeFormer face restorer are non-commercial — their dropdown entries say so; check upstream terms before commercial use. Credit to Xintao Wang et al. for Real-ESRGAN and to BasicSR for the RRDBNet architecture, and to Chen et al. / megvii-research for NAFNet (MIT). NAFNet deblur weights are mirrored on Hugging Face (nyanko7/nafnet-models); the upstream originals are on the official NAFNet Google Drive.