linux-rt-upscaler 0.3.0.post3

Real-time AI upscaler for any Linux window

Real-Time Upscaler for Linux

A real-time AI upscaler for any application window on GNU/Linux. It uses CuNNy neural networks to perform 2x (or 4x) upscaling, then scales the result to full screen while preserving aspect ratio. Mouse clicks and motion are automatically forwarded to the original window.

Now with full XWayland support – works seamlessly under Wayland compositors!

Results at 400% magnification

Features

• AI-Powered Upscaling – Uses the CuNNy (Convolutional upscaling Neural Network) models, trained specifically for high-quality 2x upscaling of visual novels and illustrations.
• Complete Model Selection – Choose from 9 variants, offering a range of quality/performance trade-offs:
  • 8x32 – Highest quality, slowest.
  • 4x32
  • 4x24
  • 4x16
  • 4x12
  • 3x12
  • fast – Default. Recommended for slow machines.
  • faster
  • veryfast – Fastest option, lowest quality.
• Tile‑Based Processing – Divides each frame into tiles, re‑upscaling only the regions that change, drastically reducing GPU and CPU load for mostly static content.
• Attach to Any Window – Either grab the currently active window, select from visible windows or launch a new program and capture its window automatically.
• Flexible Output Geometry – Control the overlay size, scaling mode, offset and borders, with interactive zoom and pan.
• Input Forwarding – Click, move, and drag on the upscaled image as if interacting directly with the original window.
• Hardware Accelerated – Custom Vulkan compute backend works on NVIDIA, AMD, and Intel GPUs.
• Global Hotkeys & On-Screen Display – Switch models, zoom, pan, take lossless screenshots and more using keyboard shortcuts.

Requirements

• GNU/Linux (X11 or Wayland with XWayland)
• Vulkan-capable GPU (NVIDIA, AMD, Intel)
• Python 3.10 or newer

[!NOTE] Python 3.14 compatibility

Previously, we advised against Python 3.14 due to a suspected Vulkan backend issue (#1). Recent testing shows it should now work without problems.

Python 3.14 is supported; however, our testing has been limited. If you encounter any issues, please report them.

For a safe fallback, you can still use Python 3.13:

pipx install --python=python3.13 --fetch-missing-python linux-rt-upscaler

Installation

Install with pipx (recommended)

pipx install linux-rt-upscaler

Install from source

Required development packages (click to expand)

Debian / Ubuntu / Linux Mint

sudo apt update
sudo apt install gcc make libvulkan-dev libx11-dev libxcb1-dev libx11-xcb-dev \
    libxext-dev libxdamage-dev libxfixes-dev

Fedora / RHEL / CentOS

sudo dnf install gcc make vulkan-loader-devel libX11-devel libxcb-devel libX11-xcb-devel \
    libXext-devel libXdamage-devel libXfixes-devel

Arch Linux / CachyOS

sudo pacman -S base-devel vulkan-devel libx11 libxcb libxext libxdamage libxfixes

openSUSE

sudo zypper install gcc make vulkan-devel libX11-devel libxcb-devel libX11-xcb-devel \
    libXext-devel libXdamage-devel libXfixes-devel

---

git clone https://github.com/baronsmv/linux-rt-upscaler.git
cd linux-rt-upscaler

pip install -e .

Usage

After installation, the upscale command will be available globally:

# Upscale the currently active window
upscale

# Interactively select from visible windows
upscale -s

# Run a command and upscale its window
upscale -- <command>

# Choose a specific model (examples)
upscale -m 8x32      # Highest quality, slowest
upscale -m 4x24      # A balanced option
upscale -m veryfast  # Maximum performance

# Perform 4x upscaling (two 2x passes)
upscale -2

# Set a custom overlay geometry (50% size, fitted)
upscale -o 50%

# Crop 100 pixels from top and left, then upscale
upscale --crop-top 100 --crop-left 100

# Shift the overlay 100 pixels right and 50 down
upscale --offset-x 100 --offset-y 50

For a full list of options and examples:

upscale --help

Controls

Shortcut	Action
Alt+Shift+S	Toggle overlay visibility / pause processing
Alt+Shift+M	Switch to the next upscaling model
Alt+Shift+G	Cycle output geometry (fit → stretch → cover)
Alt+Shift+P	Take a lossless screenshot (--screenshot-dir DIR defines the location)
Alt+Shift++ / -	Zoom in / Zoom out
Alt+Shift+↑ / ↓ / ← / →	Pan the upscaled content
Alt+Shift+Escape	Exit the application

All hotkeys can be customised in the configuration file.

Profiles

You can define named configuration profiles in your YAML config file. Profiles let you quickly switch settings for different games or applications without typing long command lines each time.

Create a config file (e.g., ~/.config/linux-rt-upscaler/config.yaml) and add a top-level profiles key. Each profile is a dictionary with an optional match section and an options section.

If no profile is selected manually, the program checks all profiles that have a match section against the title of the target window. If a profile matches (any match criterion is sufficient), its options are applied automatically.

# General defaults (lowest priority)
model: fast
select: false

profiles:
  game:
    match:
      title: "Danganronpa"     # Exact match (case-insensitive)
      title_contains: "ronp"   # Or substring match (case-insensitive)
      title_regex: "Dangan.*"  # Or regular expression (case-insensitive)
    options:
      model: 4x24
      double_upscale: true

A more detailed example is included here.

How It Works

1. Window Selection – Uses X11 to find the target window by PID or WM_CLASS.
2. Capture – Grabs the window's pixels using a fast custom C library.
3. AI Upscaling – CuNNy compute shaders (written in HLSL, compiled via Compushady) produce a 2x (or 4x) larger image.
4. Aspect-Preserving Scaling – A lightweight Lanczos2 compute shader scales the upscaled image to fill the monitor, adding black bars to maintain the original aspect ratio.
5. Display – The result is rendered in a transparent overlay window that bypasses the window manager (so it always stays on top).
6. Input Forwarding – Mouse events are transformed using the scaling ratios and sent to the original window via XSendEvent.

Future Plans

• Standalone GUI application – Create a windowed app interface for easier management.
• Addition of more models – Parse and include other models and shaders.
• Native Wayland support – Support pure Wayland windows without XWayland.

Known Issues

Mouse forwarding may not work with Wine, Proton <10 and other applications

Synthetic mouse events (clicks, motion, wheel) sent by the overlay are ignored by:

• Wine and Proton versions older than 10.0 (Proton 10 works correctly).
• Certain native applications like Firefox.

For more details, see issue #7.

Motivation

While real-time upscaling tools like Magpie and Lossless Scaling remain Windows-exclusive, projects such as lsfg-vk are successfully bringing their frame generation capabilities to Linux.

This project tackles the other half of the equation: AI-powered upscaling to deliver a native solution Linux has been missing, an experience similar to Gamescope that applies intelligent upscaling (similar to Anime4K) to any application.

Acknowledgments

This project stands on the shoulders of several open-source works, mantained by amazing people and communities:

• L65536, for the original RealTimeSuperResolutionScreenUpscalerforLinux, which demonstrated the feasibility of real‑time CuNNy upscaling on Linux. This project began from that proof‑of‑concept and has since evolved into a complete rewrite with a custom Vulkan backend, tile‑based processing, and more features.
• funnyplanter, for the incredible CuNNy neural network upscaling models, especially the Magpie NVL variants trained on visual novel artwork.
• Compushady, which served as an invaluable foundation during early development. The current release uses a custom, tailored Vulkan backend that builds on those early lessons and grows to meet the project’s specific needs.
• PySide6, the Qt binding that powers the entire graphical overlay window.
• xcffib, the low‑level XCB binding used for all window management and event forwarding.
• screeninfo, providing the physical dimensions and positions of all connected monitors, and used for scaling factor detection.
• psutil, helping locate the target window by matching process IDs and window classes when attaching to a running application or launching a new one.
• Pillow, used for saving lossless screenshots and for rendering the on‑screen display text as an image, which is then uploaded to the GPU.