import letify

let = letify.Launcher()
colab = let.providers.colab_pro_plus

@let.function(device=colab.G4, host="remote")
def train(lr, bs):
    import torch
    ...
    return {"loss": loss}

print(train(lr=1e-4, bs=32))

No session to create. No environment to install. No files to upload. No scope to open, and no machine left running when you close the lid. 🎉

💡 Why declare instead of connect

Renting a GPU normally means going to it. You open a notebook or an SSH session, rebuild your environment there, copy your data across, and work inside somebody else's machine for as long as it lives. The training is the small part. The rest is infrastructure work that produces no results.

Declaring inverts that. You say what a function needs and where it belongs; the session, the environment, the transfer and the teardown are arranged for you.

🧠 You never leave the environment you were working in

Your editor, your debugger, your notes, your data and your git history stay where they are. The declaration sends one function to the card and brings the result back, so the remote GPU is a detail of one function rather than a place you move into.

That continuity is the point. You are not two people, one of whom lives in a browser tab with a different Python and no working directory. There is nothing to keep in sync and nothing to copy back before the session dies.

@let.function(device=colab.G4, host="remote")
def train(lr, bs):
    ...

train(lr=1e-4, bs=32)     # the same file you were already editing

⚡ There is no infrastructure step

Nothing is turned on, and nothing is left running by accident. A call starts the session it needs and ends it when the work is done, so forgetting to stop a GPU is not a mistake you can make. Neither is installing packages: the environment comes from the uv.lock you already have, cached so the second session does not pay for it.

The failure mode this removes is the expensive one. A forgotten instance bills overnight, and a session you spent twenty minutes preparing dies with everything in it.

📈 Nothing is tied to one server

A declaration names an accelerator shape, not a machine. So the same code runs on a Colab runtime, a lab box over SSH, an Elice allocation or this laptop by changing one line of configuration, and when one account runs out you add another rather than rewriting anything.

That is also how the work scales sideways. Capacity is what the provider entry declares it has, so a sweep spreads across every card available to you, across accounts and across machines:

[colab_pro.devices]
G4 = { count = 2 }            # two sessions on this account

[lab_a100.devices]
A100 = { indices = "0-3" }    # four cards in the shared box are ours

Six configurations then run six ways at once if six cards exist, on hardware that never had to be the same hardware. Nothing about the declaration changes when the pool grows.

💸 What that makes affordable

The same card costs wildly different amounts depending on which door you rent it through.

A factor of four for identical silicon. The cheap door is a notebook, though: no persistent disk, eviction at any moment, and whichever accelerator happens to be free. Everything above is what makes that door usable, so the cheapest option stops being the inconvenient one.

# open a browser, pick a GPU, hope it's free
!pip install -q torch transformers  # 4 min
!gdown ...                           # 18 min
!git clone https://github.com/me/repo
%cd repo
# ... finally start working
# ... session dies, start over

@let.function(device=colab.G4, host="remote",
              volumes=[cache])
def train(lr, bs):
    ...

train(lr=1e-4, bs=32)

📦 Installation

uv add letify                 # core, no provider dependencies
uv add "letify[colab]"        # Google Colab
uv add "letify[modal]"        # Modal
uv add "letify[shell]"        # SSH, tunnels, Elice Cloud
uv add "letify[gcs]"          # Google Cloud Storage cache
uv add "letify[s3]"           # S3 compatible cache
uv add "letify[all]"          # everything

The Python package is pure Python. A provider whose package is missing simply reports itself unavailable, and the rest keeps working.

One optional piece is native. host="local" needs letify-core, a Rust workspace that stands in for the CUDA driver, built with python letify-core/build.py. If you only ship functions to remote machines you never need it.

🚀 Quickstart

1. Declare your accounts once

Put accounts in ~/.letify, so they belong to your machine and never to the repository.

[colab_pro_plus]
kind = "colab"
account = "you@example.com"

[lab_a100]
kind = "shell"
address = "gpu.lab.example.edu"
user = "researcher"
key = "~/.ssh/id_ed25519"
persistent = true

🔐 Secrets are referenced, never written. Use access_token_env = "MY_TOKEN" or access_token_keyring = "service/user".

2. Look around

$ letify providers
colab_pro_plus  colab   ephemeral   channel=persistent
lab_a100        shell   persistent  channel=persistent
local           local   persistent  channel=persistent

$ letify devices
{
  "colab_pro_plus": ["A100", "G4", "H100", "L4", "T4", "v5e1", "v6e1"],
  "lab_a100":       ["A100"],
  "local":          ["CPU", "GeForce_RTX_4050"]
}

3. Declare and run

import letify

let = letify.Launcher()
env = letify.Env()                      # reads uv.lock
colab = let.providers.colab_pro_plus
cache = colab.volume("hf-cache")        # survives the session

@let.function(device=colab.G4, host="remote", env=env, volumes=[cache])
def train(lr, bs):
    ...
    return {"loss": loss}

print(train(lr=1e-4, bs=32))

That is the whole program. 🍰

🧭 What a declaration says

Three words, and none of them is a mechanism.

@let.function(
    device=colab.G4,       # where the accelerator is, with provider and account
    host="remote",         # where the host code runs
    lifetime="call",       # how long the session lives
)

device carries the provider, the account and the accelerator in one value, because those are one decision. Core count and memory come with the shape the provider registered, so there is nothing to ask for.

host is the CUDA word for the CPU side. "local", the default, keeps Python and the libraries in this process and forwards only CUDA calls. "remote" ships the function to the machine that holds the device.

lifetime is how long the session lives. "call", the default, ends it with the call, counting a search space as one call. "process" keeps it so a run of separate calls does not pay session start each time.

⚠️ letify never silently changes the mode. Ask for something a provider cannot serve and you get an exception naming the reason. Ask for something slow and you get a warning with the numbers, and then it runs, because the choice is yours.

🌍 Providers

Provider
├── 💻 Local      your machine            persistent
├── ☁️  Modal      serverless GPU          persistent
└── 🐚 Shell      any machine over SSH    ephemeral by default
    ├── 📓 Colab   via the official CLI
    ├── 🕳️  Tunnel  Tailscale or frp, for NAT
    └── 🇰🇷 Elice   Elice Cloud, allocated by API

Multiple accounts are first class. Each configuration entry is one account, and entries of the same kind coexist. Two Colab accounts means twice the concurrent sessions.

a = let.providers.colab_pro_plus
b = let.providers.colab_pro

@let.function(device=a.G4, host="remote")
def train(lr): ...

@let.function(device=b.L4, host="remote")      # different account, same program
def evaluate(ckpt): ...

Or do not pick at all:

@let.function(device=let.providers.any.A100, host="remote")
def train(lr): ...

🔭 Sweeps

Fan-out is a declared space, not a .map() call. Passing a space where a scalar is expected says that argument varies.

space = letify.grid(lr=[1e-4, 3e-4, 1e-3], bs=[16, 32])   # 6 points
pairs = letify.zip(lr=[1e-4, 3e-4], bs=[16, 32])          # 2 points
both  = letify.grid(lr=[1e-4]) | letify.grid(lr=[1e-3])   # union

Then consume it with the language you already know. 🐍

@let.function(device=colab.G4, host="remote")
async def train(lr, bs):
    ...

results = await train(space)              # list, in input order

async for r in train(space):              # streamed, as each finishes
    print(r)

🧵 Sync or async is declared at the def, not at the call. A plain def blocks. An async def gives you a coroutine, so await and asyncio.gather work exactly as they always do. letify adds no future type of its own, and there is no .remote(), .spawn() or .map() to remember.

💾 Caching that actually helps

A volume is a content addressed blob store. Contents are named by their hash, and mutable names live in a separate tiny namespace, exactly like Git objects and refs.

cache = colab.volume("hf-cache")

@let.function(device=colab.G4, host="remote", volumes=[cache])
def train(lr): ...

Why this shape:

The payoff is where it hurts most, which is session start:

All of that time is billed as GPU time. That is why an ephemeral provider with a volume attached behaves like a persistent one.

🔗 Values that stay put

A session is one living process, so a value can stay in it.

@let.function(device=colab.G4, host="remote", lifetime="process", keep_remote=True)
def build_model():
    return load_model()          # 14 GB, stays on the remote machine

@let.function(device=colab.G4, host="remote", lifetime="process")
def evaluate(model, batch):
    return model(batch)          # the handle resolves in place

model = build_model()            # a Handle, not 14 GB
evaluate(model=model, batch=...)

Large arguments are content addressed too. Pass the same tensor to ten calls and it crosses the network once, because the runtime is asked by digest whether it already holds it.

A handle names the session that holds it. Passing one to a different session raises rather than quietly copying the object across, since that would be an unrequested transfer of everything it points at.

💸 Your bill cannot run away

Nothing has to be torn down by hand.

A call ends its own session. That is the default, and a sweep counts as one call, so six points start one set of sessions and end them once.

lifetime="process" is the opt in, for a run of separate calls that would otherwise pay session start each time. It ends when your process does, because a timer that ended it sooner would overrule what you declared.

The lease is the backstop. The session holds a deadline that this process keeps renewing. Kill your script, lose your laptop, crash your kernel, and the worker exits on its own, which frees the card. The grace period is long enough that a flaky connection does not kill a training run. Whether it also stops the billing depends on what the provider charges for: docs/guide/06-cost.md says which providers are covered and which are not.

🚫 There is deliberately no detached mode. A detached run whose remote side gets preempted loses its results. Instead, the local process stays the owner, and durability comes from checkpoints in the store.

🧪 Test without a GPU, without mocks

The local provider starts the same worker behind the same framed protocol a remote runtime would. Your tests exercise the real path.

def test_train_returns_a_loss():
    let = letify.Launcher(home=False)

    @let.function(device=let.providers.local.CPU, host="remote")
    def train(lr):
        return {"loss": 1.0 / lr}

    assert train(lr=2.0)["loss"] == 0.5

🛠️ CLI

letify login shell lab        # declare an account, and reference it here
letify logout lab             # take the account off this machine
letify providers              # who is declared, storage, channel kind
letify devices                # what each one offers
letify status                 # what is running right now
letify usage                  # what is left on each account
letify utilization            # how busy each instance's GPU is
letify check lab              # does this machine answer?
letify probe lab              # is host="local" worth using here?
letify efficiency 0.5 3 150   # the formula, from measured terms

📚 Documentation

🚧 Status

Alpha, and honest about it. What works today:

✅ Declarations, sync and async, sweeps, pooling, session lifetimes and the lease ✅ Persistent sessions: handles resolve in later calls, large arguments travel once ✅ Content addressed storage, configuration and secrets ✅ The Local and Colab providers ✅ letify-core, verified on a real GPU: the agent opens the driver, the local driver forwards an allocation and a copy in both directions, and the bytes match

Not finished yet:

🚧 letify-driver covers the entry points a PyTorch process needs to start up and run one kernel. Anything else names itself and returns CUDA_ERROR_NOT_SUPPORTED, so a real run prints the list of what to build next 🚧 Modal and Elice follow each published interface but have not been run against the live services 🚧 Unified memory cannot be forwarded at all, so a paged optimizer needs host="remote"

The full list is at the end of docs/SPEC.md.

🤝 Contributing

Spec driven and test driven: settle docs/SPEC.md, write the failing test, then write the code. Performance work uses ResearchTree, where one branch is one experiment and one pull request is its lab note. Read CLAUDE.md before opening one.