anycloud-sdk 0.1.21

Python SDK for anycloud — submit jobs, run workloads on any cloud

anycloud Python SDK

Submit jobs, run workloads on any cloud.

Install

pip install anycloud-sdk

For the latest pre-release version:

pip install --pre anycloud-sdk

Quick start

import anycloud

ac = anycloud.Client()

job = ac.submit("my-training:latest", gpu="h100:8", env={"LR": "0.01"})
job.wait()
print(job.logs())

Chaining jobs

prep = ac.submit("prep:latest")
prep.wait()

train = ac.submit("train:latest", gpu="h100:8", env={"LR": "0.01"})
train.wait()

eval_job = ac.submit("eval:latest")
eval_job.wait()

Fan-out / fan-in

from anycloud import Submission

split = ac.submit("split:latest")
split.wait()

shards = ac.submit_many([
    Submission(image="worker:latest", env={"LR": lr})
    for lr in ["0.1", "0.01", "0.001"]
])
shards.wait()

merge = ac.submit("merge:latest")
merge.wait()

Decorator (run a Python function remotely)

Skip the image-build loop — decorate a function, and AnyCloud clones your repo on the remote VM:

@anycloud.function(image="ghcr.io/acme/trainer:latest", gpu="h100:8")
def train(lr: float):
    ...

jobs = train.map([0.1, 0.01, 0.001])  # fan out across args
jobs.wait()

Requires your code in a git repo (committed + pushed) and git installed in the image. See reference.

When a remote function fails, JobFailedError.logs includes the labeled bootstrap error:

from anycloud import JobFailedError

try:
    train.submit(0.01).wait()
except JobFailedError as e:
    print(e.state, e.logs)

Buckets

Chain data between jobs using bucket handles. Output buckets are auto-created by the server; upload() auto-creates input buckets.

data  = ac.bucket("training-data")
model = ac.bucket("model-output")

data.upload("~/datasets/imagenet")  # auto-creates bucket

prep  = ac.submit("prep:latest", input=data, output=model)
prep.wait()

train = ac.submit("train:latest", input=model, output=model, gpu="h100:8")
train.wait()

model.download("~/checkpoints/")

Bucket I/O streams through the local anycloud api server using a credential registered with anycloud credentials new — no extra installs needed. Use Client(credentials="<name>") so the SDK knows which credential to pass.