bladekit — Blade AI Fabric SDK + CLI

Run Python functions on the GPU fleet, from your laptop.

Install

Distribution bladekit, import bladekit, command blade (bladekit is installed too, as an alias of the same command). Python ≥ 3.10.

pip install bladekit

Also served, with the forge-shadow compatibility package, by the branded index https://console.blade.sh/sdk/ (docs/INSTALL.md in the source repository).

import bladekit

app = bladekit.app("demo")

@app.function(gpu="A100", image="fabric/torch-2.x")
def embed(texts):
    from sentence_transformers import SentenceTransformer
    return SentenceTransformer("all-MiniLM-L6-v2").encode(texts).tolist()

@app.local_entrypoint()
def main():
    print(embed.remote(["hello", "world"]))

$ blade setup                # browser flow: mints + saves a token (defaults to https://cp.run.blade.sh)
$ blade setup --api-url https://cp.run.stg.blade.sh   # ... or against another plane
$ echo "$FABRIC_TOKEN" | blade token set   # headless alternative; or: --from-env FABRIC_TOKEN
$ blade run script.py        # streams logs live, returns the result
$ blade logs <run_id> -f     # follow logs with resume
$ blade status <run_id>      # state + queue position
$ blade runs                 # list runs
$ blade cancel <run_id>
$ blade nodes                # fleet: GPUs free/busy
$ blade --version            # installed bladekit version

Object API

• bladekit.app(name, image=...) (or the class, bladekit.App) — namespace of remote functions.
• @app.function(gpu=, cpu=, memory=, timeout=, image=, idle_timeout=) — register a function. image is a catalogue alias (fabric/base) or a raw registry ref with an explicit tag/digest. Note: when the platform enforces a registry allowlist (FORGE_IMAGE_REGISTRY_ALLOWLIST), a raw ref (or an Image-DSL Image.from_registry(...) base) outside the allowed registries is rejected with a 400 IMAGE_REGISTRY_FORBIDDEN — catalogue aliases are always allowed.
• @app.local_entrypoint() — the function blade run invokes locally.
• fn.remote(*args, **kwargs) — blocking; returns the result or re-raises the remote exception (with the remote traceback).
• fn.spawn(*args, **kwargs) — non-blocking; returns a handle with .get(timeout=) / .cancel().
• fn.local(...) — run locally without the fleet.

Successive .remote() calls of the same function in one process reuse the same run (hot container, FIFO calls). At the end of blade run the SDK POSTs /complete to drain the container immediately.

Container lifecycle hooks

An @app.cls() class is the stateful counterpart of @app.function: it is instantiated once per container and its methods (marked with @app.method()) reuse that instance across calls. Three decorators run code at defined moments in the container's life:

• @bladekit.build() — runs once at image-build time to bake expensive, deterministic setup into the image layer (download weights, warm a compile cache), so every container starts with it already on disk. Until the control-plane image-build path executes it, the runner falls back to running it once per container, before the enter hooks — semantically weaker (per-container, not baked in) but correct.
• @bladekit.enter() — runs once per container before the first call (the usual place to load a model onto the GPU). Re-runs on a fresh instance after a blade serve hot reload.
• @bladekit.exit() — runs once per container at graceful teardown, after the last call drains and before the process exits (flush a cache, volume.commit(), close a handle, emit a final metric). Exit hooks are best-effort: an exception in one is logged and the remaining hooks still run, so a failing teardown never wedges shutdown.

Within each kind, hooks run in definition order; across kinds the order is always build → enter → (calls) → exit.

@app.cls(gpu="RTX-A4500", image="fabric/torch")
class Embedder:
    MODEL = "BAAI/bge-small-en-v1.5"

    @bladekit.build()
    def fetch_weights(self):          # once at build time (fallback: per container)
        from huggingface_hub import snapshot_download
        snapshot_download(self.MODEL)

    @bladekit.enter()
    def load(self):                   # once per container
        from sentence_transformers import SentenceTransformer
        self.model = SentenceTransformer(self.MODEL, device="cuda")

    @app.method()
    def embed(self, texts):
        return self.model.encode(texts).tolist()

    @bladekit.exit()
    def flush(self):                  # once at teardown, best-effort
        print("shutting down")

Embedder().embed.remote(["hello", "world"])

Parametrized classes

bladekit.parameter() turns one @app.cls definition into many independent instances, one per distinct set of parameter values — the equivalent of modal.parameter. Declare parameters as typed class attributes; a parametrized class defines no __init__ (the values are set for you). Each distinct value set gets its own run and its own container pool, and the bound values travel with every call so AI Fabric routes it to the matching container.

@app.cls(gpu="RTX-A4500", image="fabric/torch")
class LLM:
    model_name: str = bladekit.parameter(default="Qwen/Qwen2.5-1.5B-Instruct")
    temperature: float = bladekit.parameter(default=0.7)

    @bladekit.enter()
    def load(self):
        # each instance reads its own bound parameters
        self.pipe = load_pipeline(self.model_name)

    @app.method()
    def generate(self, prompt):
        return self.pipe(prompt, temperature=self.temperature)

# one definition, two parametrized instances → two container pools
LLM(model_name="Qwen/Qwen2.5-1.5B-Instruct").generate.remote("hi")
LLM(model_name="meta-llama/Llama-3.1-8B-Instruct").generate.remote("hi")

• The value type comes from the annotation (or an explicit type=) and must be one of str, int, float, bool, bytes. A parameter without a default is required at call time.
• Instantiate with keyword arguments only; unknown or wrongly-typed values are rejected with bladekit.InvalidArgumentError.

Fan-out: map, spawn_map

• fn.map(iterable, ...) — blocking fan-out of f(x) over the GPU fleet; yields results in input order (order="index", default) or arrival order.

• fn.starmap(iterable, ...) — like map, but each element is a tuple splatted as positional args: (a, b) → f(a, b).

• fn.spawn_map(iterable, ...) — spawn + map: fire the whole fan-out and return a MapHandle immediately, without blocking on results. The batch is created, all inputs are pushed, and it is finalized server-side; you then poll it at your leisure. fn.spawn_starmap(iterable, ...) is the tuple-splatting variant.

  handle = classify.spawn_map(urls)      # returns at once — nothing awaited
  ...                                    # do other work; the fleet churns
  for label in handle.results():         # collect later (order/return_exceptions here)
      ...
  print(handle.status())                 # inspect progress
  handle.cancel()                        # stop the batch
  

  The batch is durable server-side, so a handle survives a client disconnect — re-attach from another process with blade map resume <map_id>. map() blocks and streams; spawn_map() hands you the handle and gets out of the way.

Dynamic batching: @bladekit.batched

@bladekit.batched auto-collects individual calls into a batch and invokes the decorated function once per batch, then fans the per-call results back to each caller. The function is written to take lists and return a list; callers still invoke one item at a time. Stack it under @app.function (Modal order):

@app.function(gpu="RTX-A4500", image="fabric/torch")
@bladekit.batched(max_batch_size=32, wait_ms=50)
def embed(prompts: list[str]) -> list[list[float]]:
    # `prompts` is the whole batch; return one vector per prompt.
    return model.encode(prompts).tolist()

vec = embed.remote("hello")     # one call in, one vector out — batched under the hood

A batch is dispatched as soon as either trigger fires: it reaches max_batch_size inputs, or wait_ms milliseconds elapse after the batch's first input. This is distinct from @app.concurrent, which forwards N concurrent requests to a long-lived server, each handled independently — @bladekit.batched fuses calls into one invocation so you amortize a fixed per-call cost (a model forward pass, a tokenizer round-trip) across the whole batch.

Contract: positional args are transposed by position and keyword args by name, so every call in a batch must share the same call shape; the function must return a list of exactly len(batch) results (else the whole batch fails with BATCH_RESULT_LENGTH_MISMATCH). If the function raises, every call in the batch receives that exception.

Cloud bucket mounts

bladekit.CloudBucketMount mounts an external object-store bucket you own (S3 / GCS / R2 / MinIO / Scaleway — anything speaking the S3 API) as a filesystem path inside a function or deployed replica. Unlike bladekit.Volume (an AI Fabric-managed network volume with generations and commit()), a bucket mount is a live view of your bucket — AI Fabric only mounts it, it never stores the objects. The runner FUSE-mounts each bucket at container start and unmounts it on teardown.

Pass it in the same volumes= map as a Volume, keyed by mount path:

import bladekit, pathlib

app = bladekit.App("bucket-demo")

creds = bladekit.Secret.from_name("aws-credentials")  # holds the bucket keys

@app.function(
    image="fabric/base",
    volumes={
        "/mnt/data": bladekit.CloudBucketMount(
            "my-data-bucket",
            secret=creds,
            read_only=True,       # mount without write access
            key_prefix="raw/2024/",   # mount only this sub-path
        ),
    },
)
def count_parquet():
    return sum(1 for _ in pathlib.Path("/mnt/data").glob("*.parquet"))

@app.local_entrypoint()
def main():
    print(count_parquet.remote())

bladekit.CloudBucketMount(bucket_name, *, secret, read_only=False, key_prefix=None, bucket_endpoint_url=None, requester_pays=False):

• secret — required. Credentials come only from a bladekit.Secret, never inline. The wire spec carries the secret name, never a value; the control plane resolves it and injects its keys (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, plus optional AWS_SESSION_TOKEN) as container env vars, and the runner reads those at mount time. A credential never appears in the run body, argv, or a log line.
• read_only=True — mounted without write access.
• key_prefix="raw/2024/" — mount only objects under this bucket-relative key prefix (must end with /).
• bucket_endpoint_url= — S3 endpoint override for non-AWS stores (R2, MinIO, GCS-S3, Scaleway); omit for AWS S3.
• requester_pays=True — for buckets with S3 Requester Pays.

Store the credentials first with blade secret create aws-credentials AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=…. At most 8 bucket mounts per function; mount paths share the volume namespace (absolute, unique, outside reserved paths).

Errors

• bladekit.APIError (and subclasses) carry the API envelope code + human message.
• bladekit.RunError (and subclasses NodeLostError, ContainerFailedError, VersionMismatchError, …) describe platform failures with actionable hints.
• A remote user exception is re-raised as the original exception when possible, otherwise wrapped in bladekit.RemoteFunctionError — never confused with a platform error.

Config

blade setup (browser token flow) and blade token set both write ~/.blade/config.toml. FABRIC_API_URL / FABRIC_TOKEN environment variables take precedence; FABRIC_CONFIG points the CLI at another file. blade token show says where each value came from.

blade token set never accepts the token as a positional argument — that would leak it into ps aux, shell history and process monitors. Pass it on stdin (echo "$TOK" | blade token set) or name the variable (--from-env FABRIC_TOKEN).

Inside a container, read the platform's variables under their FABRIC_* names (FABRIC_SERVER_PORT, FABRIC_VOLUMES, FABRIC_PREEMPT_GRACE_SECONDS, FABRIC_MAX_SESSION_SECONDS, FABRIC_TURN_URL / FABRIC_TURNS_URL / FABRIC_TURN_SECRET). FABRIC_* and FORGE_* are reserved: Image.env() rejects both prefixes.

Compatibility and migration (from forge / forge-shadow)

The SDK used to be forge-shadow (import forge, command forge). Nothing an existing user does breaks on upgrade; the old names are deprecated and keep working for a transition period (see "Removal" below).

Before	Now	The old one…
pip install forge-shadow	pip install bladekit	installs bladekit + the shims below
import forge	import bladekit	works, DeprecationWarning once
forge.<x>, forge.<module>	bladekit.<x>, bladekit.<module>	same objects, same modules
forge command	blade command	same subcommands and options, notice on stderr
ForgeError	FabricError	the same class, DeprecationWarning on use
FORGE_TOKEN, FORGE_API_URL, FORGE_CONFIG, FORGE_VERBOSE, …	FABRIC_*	read when the FABRIC_* one is unset
~/.forge/config.toml	~/.blade/config.toml	copied to the new path on first use, left in place
image forge/torch	image fabric/torch	still accepted, same image

Details:

• forge-shadow is now a thin compatibility distribution (source in sdk/compat/forge-shadow/), published at exactly the same version as bladekit, which it requires (bladekit==<version>). It ships the forge import shim and the forge command. bladekit alone does not ship a forge package: two distributions owning one forge/ directory is what pip deletes during an upgrade, and forge is another project's name on public PyPI. pip install -U forge-shadow therefore upgrades an existing install in place; new installs should use bladekit.
• The forge import shim resolves every forge.<module> to the already imported bladekit.<module> object, so isinstance/except checks and pickles that name forge.* modules (written by an older client) keep working. New pickles name bladekit.*, which needs bladekit on the other side — every image the control plane builds has it, because the image's forge-shadow==<version> pulls it in.
• Environment: FABRIC_<NAME> wins; FORGE_<NAME> is the fallback. When both are set to different values the SDK uses FABRIC_<NAME> and warns once. The control plane still injects FORGE_* into containers, so apps deployed with the old SDK keep running unchanged, and importing bladekit in a container also exposes the user-facing ones under their FABRIC_* names.
• Catalog aliases: the SDK sends fabric/<name> to the control plane as forge/<name> (the only spelling the server knows for now); blade image list shows fabric/<name>.
• Unchanged on purpose (server side, renamed separately): HTTP headers (X-Forge-*, Forge-Signature), hosts, the forge-runner / forge-sandboxd processes and their [forge-runner] log prefix, the fg_ token prefix, the __forge_*__ attributes the runner reads, and .forgeignore.

Removal. The deprecated names (forge import and command, forge-shadow, ForgeError, FORGE_* fallback, ~/.forge migration, forge/ alias rewrite) are kept for a transition period whose end is not decided yet; they are removed in a future major release, announced in the changelog.

.forgeignore

Bundling tars the script's directory. Add a .forgeignore to exclude paths; it uses exactly the .gitignore syntax and matching rules (* never crosses /, a leading or middle / anchors at the bundle root, a trailing / matches directories only, ** spans directories, ! re-includes). .git, .venv, __pycache__ are always excluded.

Image.from_dockerfile(context_dir=...) filters its build context with a .dockerignore at the context root instead, matched exactly like docker build (patterns are relative to the context root, so *.py only matches top-level files; use **/*.py for any depth). ! is not supported there yet.