Flash

Flash is a Python SDK for developing cloud-native AI apps where you define everything -- hardware, remote functions, and dependencies -- using local code.

import asyncio
from runpod_flash import Endpoint, GpuType


@Endpoint(name="hello-gpu", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=["torch"])
async def hello():
    import torch

    gpu_name = torch.cuda.get_device_name(0)
    print(f"Hello from your GPU! ({gpu_name})")
    return {"gpu": gpu_name}


asyncio.run(hello())
print("Done!")

Write @Endpoint decorated Python functions on your local machine. Deploy them with flash deploy, then call them by running the same script. Flash handles GPU/CPU provisioning and worker scaling on Runpod Serverless.

Setup

Install Flash

pip install runpod-flash
# or
uv tool install runpod-flash
# or, with Nix (flakes) — no Python setup required
nix profile install github:runpod/flash
# ...or run it without installing:
nix run github:runpod/flash -- --help

Flash requires Python 3.10+ on macOS or Linux. Windows support is in development.

Fastest start: with Nix (flakes) you can run flash straight from GitHub — nix run github:runpod/flash -- --help — with no Python setup, no virtualenv, and every dependency pinned for you. See the Nix guide for install steps and details.

Authentication

flash login

This saves your API key and allows you to use the Flash CLI and call @Endpoint functions.

Coding agent integration

flash init writes an AGENTS.md at your project root containing CLI-first rules for AI coding tools (Cursor, Codex, Aider, Amp, Jules, etc.). It also creates CLAUDE.md as a symlink to AGENTS.md so Claude Code picks up the same rules.

If AGENTS.md or CLAUDE.md already exist in your project, Flash leaves them alone — your file, your rules.

Existing projects (already past flash init):

# from your project root
python -c "from runpod_flash.rules import install_agent_files; from pathlib import Path; install_agent_files(Path.cwd())"

Tools using other conventions: GitHub Copilot reads .github/copilot-instructions.md and Cursor (legacy) reads .cursorrules. If you use those, symlink or copy AGENTS.md:

ln -s ../AGENTS.md .github/copilot-instructions.md
ln -s AGENTS.md .cursorrules

Opt out: Delete AGENTS.md. No flash subcommand other than flash init (or an explicit call to install_agent_files(...)) will re-create it.

There is no --no-rules flag or flash rules subcommand by design: the rules are small, the cost of an unwanted file is one rm, and an opt-out flag would advertise the existence of the file to users who would otherwise never notice. If you have a use case that needs init-time suppression (corporate policy, regulated environments), open an issue and we'll revisit.

Claude Code skill bundle (optional): For richer Claude Code integration beyond static rules, install the cross-tool skill bundle:

npx skills add runpod/skills

See the SKILL.md file in the runpod/skills repository.

Quickstart

Create gpu_demo.py:

import asyncio
from runpod_flash import Endpoint, GpuType


@Endpoint(
    name="flash-quickstart",
    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
    workers=3,
    dependencies=["numpy", "torch"],
)
def gpu_matrix_multiply(size):
    import numpy as np
    import torch

    device_name = torch.cuda.get_device_name(0)
    A = np.random.rand(size, size)
    B = np.random.rand(size, size)
    C = np.dot(A, B)

    return {"matrix_size": size, "result_mean": float(np.mean(C)), "gpu": device_name}


async def main():
    print("Running matrix multiplication on Runpod GPU...")
    result = await gpu_matrix_multiply(1000)
    print(f"Matrix size: {result['matrix_size']}x{result['matrix_size']}")
    print(f"Result mean: {result['result_mean']:.4f}")
    print(f"GPU used: {result['gpu']}")


if __name__ == "__main__":
    asyncio.run(main())

Deploy, then run:

flash deploy
python gpu_demo.py

How it works

Flash has two modes: deploy and dev.

Deploy and run (flash deploy + python script.py)

Deploy packages your code and provisions endpoints on Runpod. After deploying, run your script directly and Flash routes calls to your deployed endpoints via implicit resolution:

flash deploy                 # build, upload, provision endpoints
python gpu_demo.py           # calls deployed endpoints automatically

Flash resolves endpoints by matching the app name (defaults to the current directory name) and environment (defaults to production). Configure with env vars or .env:

FLASH_APP=my-project         # defaults to current directory name
FLASH_ENV=staging            # defaults to "production"

Dev mode (flash dev)

For local development and testing, flash dev starts a hybrid dev server that runs your FastAPI app locally while provisioning live ephemeral workers on Runpod:

flash dev                    # starts local server + provisions workers
flash dev --port 3000        # custom port
flash dev --auto-provision   # provision all endpoints at startup

What Flash does

• Remote execution: @Endpoint functions run on Runpod Serverless GPUs/CPUs
• Implicit endpoint resolution: python script.py routes to deployed endpoints automatically
• Auto-scaling: workers scale from 0 to N based on demand
• Dependency management: packages install automatically on remote workers
• Two patterns: queue-based (@Endpoint) for batch work, load-balanced (Endpoint() + routes) for REST APIs
• Concurrency control: max_concurrency lets each worker process multiple jobs simultaneously

Documentation

Full documentation: docs.runpod.io/flash

• Quickstart - First GPU workload in 5 minutes
• Create endpoints - Queue-based, load-balancing, and custom Docker endpoints
• CLI reference - flash dev, flash deploy, flash build
• Configuration - All endpoint parameters

Flash apps

When you're ready to move beyond scripts and build a production-ready API, you can create a Flash app (a collection of interconnected endpoints with diverse hardware configurations) and deploy it to Runpod.

Follow this tutorial to build your first Flash app.

Flash CLI

flash --help

Learn more about the Flash CLI.

Examples

Browse working examples: github.com/runpod/flash-examples

Requirements

• Python 3.10-3.14
• macOS or Linux (Windows support in development)
• A Runpod account (email must be verified) with an API key

Contributing

We welcome contributions! See RELEASE_SYSTEM.md for development workflow.

git clone https://github.com/runpod/flash.git
cd flash
pip install -e ".[dev]"

# use conventional commits
git commit -m "feat: add new feature"
git commit -m "fix: resolve issue"

Prefer a one-command, reproducible environment? With Nix (flakes):

nix develop      # dev shell: Python 3.14, uv, ruff, mypy, bandit, shellcheck
                 # run `flash-help` inside for the available check-* helpers
nix flake check  # hermetic gate: ruff lint + format, nixfmt, shellcheck

See the Nix guide for installation and the full workflow.

Support

• Discord - Community support
• GitHub Issues - Bug reports

License

MIT License - see LICENSE for details.