Flash

Flash is a Python SDK for developing cloud-native AI apps where you define everything -- hardware, remote functions, and dependencies -- using local code.

import asyncio
from runpod_flash import Endpoint, GpuType

@Endpoint(name="hello-gpu", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=["torch"])
async def hello():
    import torch
    gpu_name = torch.cuda.get_device_name(0)
    print(f"Hello from your GPU! ({gpu_name})")
    return {"gpu": gpu_name}

asyncio.run(hello())
print("Done!")

Write @Endpoint decorated Python functions on your local machine. Deploy them with flash deploy, then call them by running the same script. Flash handles GPU/CPU provisioning and worker scaling on Runpod Serverless.

Setup

Install Flash

pip install runpod-flash
# or
uv tool install runpod-flash

Flash requires Python 3.10+ on macOS or Linux. Windows support is in development.

Authentication

flash login

This saves your API key and allows you to use the Flash CLI and call @Endpoint functions.

Coding agent integration

flash init writes an AGENTS.md at your project root containing CLI-first rules for AI coding tools (Cursor, Codex, Aider, Amp, Jules, etc.). It also creates CLAUDE.md as a symlink to AGENTS.md so Claude Code picks up the same rules.

If AGENTS.md or CLAUDE.md already exist in your project, Flash leaves them alone — your file, your rules.

Existing projects (already past flash init):

# from your project root
python -c "from runpod_flash.rules import install_agent_files; from pathlib import Path; install_agent_files(Path.cwd())"

Tools using other conventions: GitHub Copilot reads .github/copilot-instructions.md and Cursor (legacy) reads .cursorrules. If you use those, symlink or copy AGENTS.md:

ln -s ../AGENTS.md .github/copilot-instructions.md
ln -s AGENTS.md .cursorrules

Opt out: Delete AGENTS.md. No flash subcommand other than flash init (or an explicit call to install_agent_files(...)) will re-create it.

There is no --no-rules flag or flash rules subcommand by design: the rules are small, the cost of an unwanted file is one rm, and an opt-out flag would advertise the existence of the file to users who would otherwise never notice. If you have a use case that needs init-time suppression (corporate policy, regulated environments), open an issue and we'll revisit.

Claude Code skill bundle (optional): For richer Claude Code integration beyond static rules, install the cross-tool skill bundle:

npx skills add runpod/skills

See the SKILL.md file in the runpod/skills repository.

Quickstart

Create gpu_demo.py:

import asyncio
from runpod_flash import Endpoint, GpuType

@Endpoint(
    name="flash-quickstart",
    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
    workers=3,
    dependencies=["numpy", "torch"]
)
def gpu_matrix_multiply(size):
    import numpy as np
    import torch

    device_name = torch.cuda.get_device_name(0)
    A = np.random.rand(size, size)
    B = np.random.rand(size, size)
    C = np.dot(A, B)

    return {
        "matrix_size": size,
        "result_mean": float(np.mean(C)),
        "gpu": device_name
    }

async def main():
    print("Running matrix multiplication on Runpod GPU...")
    result = await gpu_matrix_multiply(1000)
    print(f"Matrix size: {result['matrix_size']}x{result['matrix_size']}")
    print(f"Result mean: {result['result_mean']:.4f}")
    print(f"GPU used: {result['gpu']}")

if __name__ == "__main__":
    asyncio.run(main())

Deploy, then run:

flash deploy
python gpu_demo.py

How it works

Flash has two modes: deploy and dev.

Deploy and run (flash deploy + python script.py)

Deploy packages your code and provisions endpoints on Runpod. After deploying, run your script directly and Flash routes calls to your deployed endpoints via implicit resolution:

flash deploy                 # build, upload, provision endpoints
python gpu_demo.py           # calls deployed endpoints automatically

Flash resolves endpoints by matching the app name (defaults to the current directory name) and environment (defaults to production). Configure with env vars or .env:

FLASH_APP=my-project         # defaults to current directory name
FLASH_ENV=staging            # defaults to "production"

Dev mode (flash dev)

For local development and testing, flash dev starts a hybrid dev server that runs your FastAPI app locally while provisioning live ephemeral workers on Runpod:

flash dev                    # starts local server + provisions workers
flash dev --port 3000        # custom port
flash dev --auto-provision   # provision all endpoints at startup

What Flash does

• Remote execution: @Endpoint functions run on Runpod Serverless GPUs/CPUs
• Implicit endpoint resolution: python script.py routes to deployed endpoints automatically
• Auto-scaling: workers scale from 0 to N based on demand
• Dependency management: packages install automatically on remote workers
• Two patterns: queue-based (@Endpoint) for batch work, load-balanced (Endpoint() + routes) for REST APIs
• Concurrency control: max_concurrency lets each worker process multiple jobs simultaneously

Documentation

Full documentation: docs.runpod.io/flash

• Quickstart - First GPU workload in 5 minutes
• Create endpoints - Queue-based, load-balancing, and custom Docker endpoints
• CLI reference - flash dev, flash deploy, flash build
• Configuration - All endpoint parameters

Flash apps

When you're ready to move beyond scripts and build a production-ready API, you can create a Flash app (a collection of interconnected endpoints with diverse hardware configurations) and deploy it to Runpod.

Follow this tutorial to build your first Flash app.

Flash CLI

flash --help

Learn more about the Flash CLI.

Examples

Browse working examples: github.com/runpod/flash-examples

Requirements

• Python 3.10-3.13
• macOS or Linux (Windows support in development)
• A Runpod account (email must be verified) with an API key

Contributing

We welcome contributions! See RELEASE_SYSTEM.md for development workflow.

git clone https://github.com/runpod/flash.git
cd flash
pip install -e ".[dev]"

# use conventional commits
git commit -m "feat: add new feature"
git commit -m "fix: resolve issue"

Support

• Discord - Community support
• GitHub Issues - Bug reports

License

MIT License - see LICENSE for details.