anycloud-sdk 0.1.3rc1

Python SDK for anycloud — submit jobs, run workloads on any cloud

anycloud Python SDK

Submit jobs, run workloads on any cloud.

Install

pip install anycloud-sdk

For the latest pre-release version:

pip install --pre anycloud-sdk

Quick start

import anycloud

ac = anycloud.Client()

job = ac.submit("my-training:latest", gpu="h100:8", env={"LR": "0.01"})
job.wait()
print(job.logs())

Chaining jobs

prep = ac.submit("prep:latest")
prep.wait()

train = ac.submit("train:latest", gpu="h100:8", env={"LR": "0.01"})
train.wait()

eval_job = ac.submit("eval:latest")
eval_job.wait()

Fan-out / fan-in

from anycloud import JobGroup

split = ac.submit("split:latest")
split.wait()

shards = JobGroup([ac.submit("worker:latest", env={"LR": lr}) for lr in ["0.1", "0.01", "0.001"]])
shards.wait_all()

merge = ac.submit("merge:latest")
merge.wait()

Buckets

Chain data between jobs using bucket handles. Output buckets are auto-created by the server; upload() auto-creates input buckets.

data  = ac.bucket("training-data")
model = ac.bucket("model-output")

data.upload("~/datasets/imagenet")  # auto-creates bucket

prep  = ac.submit("prep:latest", input=data, output=model)
prep.wait()

train = ac.submit("train:latest", input=model, output=model, gpu="h100:8")
train.wait()

model.download("~/checkpoints/")

Install with the cloud extra for your provider: pip install anycloud-sdk[aws] (or [gcp], [azure], [all]).