linux-rt-upscaler 0.1.1

Real-time AI upscaler for any Linux window using CuNNy

Real‑Time Upscaler for Linux

A real‑time AI upscaler for any application window on GNU/Linux. It uses CuNNy neural networks to perform 2× upscaling, then scales the result to full screen while preserving aspect ratio. Mouse clicks and motion are automatically forwarded to the original window.

Based on

Based on RealTimeSuperResolutionScreenUpscalerforLinux by L65536, with the following differences:

• Full‑screen scaling – The upscaled image now fills the monitor, applying a second scaling layer while preserving aspect ratio.
• Click/motion forwarding – Mouse clicks and motion are forwarded to the original window with proper coordinate transformation. This can be deactivated with the -d option.

Features

• AI‑Powered Upscaling – Uses the CuNNy neural network family, trained specifically for high‑quality 2× upscaling (ideal for visual novels, illustrations, and general content).
• Model Selection – Choose between quality/performance trade‑offs:
  • 8x32 (highest quality but slowest),
  • fast (balanced),
  • veryfast (fastest but lowest quality).
• Attach to Any Window – Either grab the currently active window or launch a new program and capture its window automatically.
• Full‑Screen Output – The upscaled image is displayed in a transparent overlay that covers your entire monitor, scaled to fill the screen while preserving aspect ratio.
• Input Forwarding – Click, move, and drag on the upscaled image as if you were interacting directly with the original window.
• Hardware Accelerated – GPU compute via Compushady (Vulkan) works on NVIDIA, AMD, and Intel GPUs.
• Low Overhead – Minimal CPU/GPU usage; the final scaling pass uses hardware bilinear filtering.

Requirements

• GNU/Linux with X11 (Wayland is not supported)
• Vulkan-capable GPU from any vendor (NVIDIA, AMD, Intel)
• Vulkan drivers (libvulkan-dev on Debian/Ubuntu)
• X11 development libraries (libx11-dev)
• Python 3.8 or newer

Installation

Option 1: Install from PyPI (Recommended)

# Install system dependencies (Debian/Ubuntu)
sudo apt install libvulkan-dev libx11-dev

# Install with pipx (recommended)
pipx install linux-rt-upscaler

# Or with regular pip
pip install linux-rt-upscaler

Option 2: Install from source

# Clone the repository
git clone https://github.com/baronsmv/linux-rt-upscaler.git
cd linux-rt-upscaler

# Install system dependencies (Debian/Ubuntu)
sudo apt install libvulkan-dev libx11-dev

# Install Python packages
pip install -r requirements.txt

# Install the package in development mode
pip install -e .

Usage

After installation, the upscale command will be available globally:

# Upscale the currently active window
upscale

# Run a command and upscale its window
upscale <command>

# Choose a specific model
upscale -m fast

# Disable mouse‑click forwarding (also enables dimming/click‑through)
upscale -d

# Show help
upscale -h

Command‑Line Options

Option	Description
-m, --model	Select upscaling model: 8x32 (best quality, slower), fast (balanced), veryfast (fastest, lower quality). Default is 8x32.
-d, --disable-click-forward	Disable forwarding mouse clicks to the original window. Also enables overlay dimming (20% opacity when mouse leaves source) and click‑through.
-h, --help	Show the help message and exit.

Controls

• Exit: Press Ctrl+C in the terminal where the upscaler is running.
• Dimming/Click‑through (only when -d is used):
  • The overlay becomes semi‑transparent (20% opacity) when the mouse leaves the source window.
  • Clicks then pass through to whatever window is underneath (e.g., your desktop or other applications).

How It Works

1. Window Selection – Uses X11 to find the target window by PID or WM_CLASS.
2. Capture – Grabs the window's pixels using a fast custom C library.
3. AI Upscaling – Four CuNNy compute shaders (written in HLSL, compiled via Compushady) produce a 2× larger image.
4. Aspect‑Preserving Scaling – A lightweight bilinear compute shader scales the upscaled image to fill the monitor, adding black bars to maintain the original aspect ratio.
5. Display – The result is rendered in a transparent overlay window that bypasses the window manager (so it always stays on top).
6. Input Forwarding – Mouse events are transformed using the scaling ratios and sent to the original window via XSendEvent.

Acknowledgments

• L65536 – For the original RealTimeSuperResolutionScreenUpscalerforLinux project, which provided the foundational scripts and CuNNy integration
• CuNNy – Neural network upscaling models, especially the NVL variants trained for visual novel content.
• Compushady – Python library for GPU compute (Vulkan backend).
• PySide6 – Qt bindings used for the overlay window.
• python‑xlib – X11 client library for window management and input forwarding.