Flow CLI & SDK

Run GPU workloads on Mithril from the shell or from Python. Flow procures GPUs through Mithril, spins up InfiniBand-connected instances, and runs your workloads. It is one engine with two projections: the CLI is what you meet first, and import flow is the same engine for programs. Every CLI verb runs on the SDK.

Public repo

First hour

The same three commands flow --help teaches:

uv tool install flow-compute   # or: pipx install flow-compute
flow setup                     # Mithril account, API key and project
flow tutorial                  # Guided first GPU run with health checks
flow run 'nvidia-smi'          # Run a command on a GPU

flow run without -i bids on the cheapest NVIDIA GPU instance type with live capacity, prints the choice with its family and hourly price, and asks before any bid above $20/hour. The implicit default never picks a TPU or an accelerator family Flow does not know, since the first command is nvidia-smi; name other hardware with -i (-i v6e-1, -i cpu). flow tutorial prices its GPU test the same way and asks before launching it; --yes is the only scripted consent and --skip-example leaves the test out. Add --dry-run to see the configuration without bidding:

$ flow run 'nvidia-smi' --dry-run
╭─ Task Configuration ─────────────────────────────────────────────╮
│ Name           run-0972ae                                        │
│ Command        nvidia-smi                                        │
│ Instance Type  a100                                              │
│ Market Price   $8.00/hr (a100, cheapest live NVIDIA GPU)         │
│ Priority       Med                                               │
│ Max Bid        $16.00/hr                                         │
│ Code Upload    auto (timeout 600s) → /workspace (host+container) │
╰──────────────────────────────────────────────────────────────────╯

✓ Configuration is valid

Then watch, inspect and stop what you started:

flow status              # List and monitor tasks
flow logs <task> -f      # Stream task logs
flow ssh <task>          # SSH into a running task
flow cancel <task>       # Cancel a task

$ flow status
╭─ ❊ Flow • 9 completed ────────────────────────────────────────────╮
│                                                                   │
│     #  Status        Task                          Type           │
│     1  ✓ completed   cli-welcome                   1×A100         │
│     2  ✓ completed   post-training-sanity-1        1×A100·40G     │
│     3  ✓ completed   pretraining-pending-a100-40g  1×A100·40G     │
│     4  ✓ completed   post-training-fine-tune       8×H100         │
│     5  ✓ completed   inference-serving             1×H100         │
│     6  ✓ completed   pretraining-train-8xa100      8×A100         │
│     7  ✓ completed   post-training-sanity-2        1×A100·40G     │
│     8  ✓ completed   reasoning-rollout-b200        8×B200         │
│     9  ✓ completed   pretraining-2048-h100         256×8×H100     │
│                                                                   │
╰───────────────────────────────────────────────────────────────────╯

The same first hour from Python:

import flow

task = flow.run("nvidia-smi", instance_type="a100")
for line in task.log_stream():
    print(line, end="")

Use flow with a coding agent

Paste this into Claude Code, Codex or any other coding agent before it runs Flow for you:

You are operating Flow, the `flow` CLI that runs GPU workloads on Mithril. Follow these rules.

Setup
- Install with `uv tool install flow-compute`.
- The user creates an API key at console.mithril.ai and exports `MITHRIL_API_KEY`; never print
  it. If it is missing, ask the user to set it or to run `flow setup` in their own terminal.
- Ask the user which project to bill. Pass it as `--project <name or id>` before the command on
  every call (`flow --project research status --json`), or export `MITHRIL_PROJECT` once.
- Export `FLOW_NO_INTERACTIVE=1`, keep `FLOW_FORCE_INTERACTIVE` unset and never pass an
  interactive flag, so a command that would prompt fails instead of waiting.

Reading the CLI
- Learn commands, options, capabilities and exit codes from `flow schema --json`, not help
  text. Never use what it marks `"deprecated": true`; use the `replacement` it names.
- Pass `--json` to every command that has the `json` capability. stdout is then one envelope,
  `{"data": ..., "error": null}` on success; on failure `error` carries `code`, `message` and
  `hint`. Check `error` first, since `data` can still hold results. With `--follow`, or on a
  command whose `json_modes` is only `stream`, stdout is one JSON object per line instead.
- Retry only `TRANSIENT` (exit 6), with backoff. `AUTH_REQUIRED` (exit 2): ask the user to fix
  the credentials. Any other `error.code`: change the request or report it; never rerun it as is.
- `APPROVAL_PENDING` (exit 7): it waits for the user's approval. Tell them, stop, never resend it.

Spending
- Commands that create tasks, VMs, bids or reservations spend money. Before each, tell the user
  the project, instance type and count, price cap and expected run time; wait for an explicit yes.
- Preview at no cost: `flow run 'python train.py' -i 8xh100 -m 20 --dry-run --json`. `-m` caps
  the bid per instance-hour; spot bills the market's clearing price, never more than the cap.
- An envelope's `project` names the project its command acted in. Before a spend, check that the
  preview's `project` is the one the user approved; if it is not, stop and ask.
- `--yes` is the only consent the CLI accepts. Pass it only for a spend the user approved;
  `CONFIRMATION_REQUIRED` (exit 3) means the CLI wants the user's decision, not yours.
- Pass `--idempotency-key <key>` to `flow run` so a retry returns the first task, not a second.
- A spot task keeps its machine, and its bill, after its command exits. Once its work is done,
  cancel what you started: `flow cancel <task> --yes --json`. A reservation bills until its end
  time and cannot be ended early.

Operating
- `flow status --json` lists tasks and `flow logs <task> --json` shows output.
  `flow ssh <task> -- <command>` runs a command on the machine and takes no `--json`.
- Full command index, SDK reference and recipes: `flow setup --claude` or `flow setup --codex`.

Background

There's a paradox in GPU infrastructure today: Massive GPU capacity sits idle, even as AI teams wait in queues—starved for compute. Mithril, the AI-compute omnicloud, dynamically allocates GPU resources from a global pool (spanning Mithril's first-party resources and 3rd-party partner cloud capacity) using efficient two-sided auctions, maximizing surplus and reducing costs. Mithril seamlessly supports both reserved-in-advance and just-in-time workloads—maximizing utilization, ensuring availability, and significantly reducing costs.

Why choose Flow

Status quo GPU provisioning involves quotas, complex setups, and queue delays, even as GPUs sit idle elsewhere or in recovery processes. Flow addresses this:

Dynamic Market Allocation – Efficient two-sided auctions ensure you pay the lowest market-driven prices rather than inflated rates.

Simplified Batch Execution – An intuitive interface designed for cost-effective, high-performance batch workloads without complex infrastructure management.

Provision from 1 to thousands of GPUs for long-term reservations, short-term "micro-reservations" (minutes to weeks), or spot/on-demand needs—all interconnected via InfiniBand. High-performance persistent storage and built-in Docker support further streamline workloads, ensuring rapid data access and reproducibility.

Why Flow + Mithril?

"The tremendous demand for AI compute and the large fraction of idle time makes sharing a perfect solution, and Mithril's innovative market is the right approach." — Paul Milgrom, Nobel Laureate (Auction Theory and Mechanism Design)

Workflows

Research mode

• flow run → reproducible batch jobs from a command, a YAML file or a SLURM script.
• flow dev → interactive development on an allocated GPU workstation.
• import flow → the same engine for pipelines and orchestration.

# Launch a batch job on discounted H100s
flow run 'python train.py' -i 8xh100

# Run a saved configuration
flow run task.yaml

# Frictionlessly leverage an existing SLURM script
flow run job.slurm

# Serverless-style decorator
@app.function(instance_type="a100")
def train(lr: float) -> dict:
    ...

Infrastructure mode

• flow run 'python train.py' -i 8xh100 -N 20 → run on a 20-node GPU cluster
• flow volume create -s 10000 --interface file → provision 10 TB of persistent, high-speed storage
• flow ssh instance -- nvidia-smi → run across all nodes in parallel

$ flow run 'python train.py' -i 8xh100 -N 20 --dry-run
╭─ Task Configuration ────────────────────────────────────────────╮
│ Name          flow-task-0fd2a7                                  │
│ Command       python train.py                                   │
│ Instances     20 × 8xh100                                       │
│ Market Price  $192.00/hr (h100)                                 │
│ Priority      Med                                               │
│ Max Bid       $384.00/hr per instance ($7680.00/hr total)       │
│ Code Upload   auto (timeout 600s) → /workspace (host+container) │
╰─────────────────────────────────────────────────────────────────╯

✓ Dry run passed; nothing was submitted

Ideal Use Cases

• Rapid Experimentation – Quick iterations for research sprints.
• Instant Elasticity – Scale rapidly from one to thousands of GPUs.
• Collaborative Research – Shared dev environments with per-task cost controls.

Flow is not yet ideal for: always‑on ≤100 ms inference, strictly on‑prem regulated data, or models that fit on laptop or consumer-grade GPUs.

Architecture (30‑s view)

Your intent ⟶ Flow Execution Layer ⟶ Global GPU Fabric

Flow abstracts complex GPU auctions, InfiniBand clusters, and multi-cloud management into a single seamless and unified developer interface.

Installation

Requirements

• Python 3.10 or later
• Recommended: use uv to auto-manage a compatible Python when installing the CLI
• Linux or macOS for native CLI workflows. On Windows, use WSL2; native Windows is not fully validated.

1) Install uv — optional but recommended

Installation guide: docs.astral.sh/uv/getting-started/installation

• macOS/Linux:

  curl -LsSf https://astral.sh/uv/install.sh | sh
  

• Windows: use WSL2 for the Flow CLI, then run the macOS/Linux command inside WSL.

  curl -LsSf https://astral.sh/uv/install.sh | sh
  

2) Install Flow

• Global CLI (uv):

  uv tool install flow-compute
  flow setup
  

• Global CLI (pipx):

  pipx install flow-compute
  flow setup
  

Providers

Flow talks to one provider per configuration. Mithril is the default and the only provider installed by the base package; flow setup --provider (or FLOW_PROVIDER) selects the active one. The SkyPilot provider is an optional extra, pip install 'flow-compute[skypilot]', and is not installed by default. Switching providers means switching configurations; a single config does not span providers.

Under the Hood (Advanced)

• Bid Caps – Protect budgets automatically.
• Self-Healing – Spot nodes dynamically migrate tasks.
• Docker/Conda – Pre-built images or dynamic install.
• Multi-cloud Ready – Mithril (with Oracle, Nebius integrations internal to Mithril), and more coming
• SLURM Compatible – Run #SBATCH scripts directly.

Python SDK

Advanced Task Configuration

# Distributed training example (32 GPUs, Mithril groups for InfiniBand connectivity by default)
task = flow.run(
    "torchrun --nproc_per_node=8 train.py",
    instance_type="8xa100",
    num_instances=4,  # Total of 32 GPUs (4 nodes × 8 GPUs each)
    env={"NCCL_DEBUG": "INFO"}
)

# Mount S3 data + persistent volumes
task = flow.run(
    "python analyze.py",
    instance_type="a100",
    data_mounts={
        "/datasets": "s3://ml-bucket/imagenet",  # S3 via s3fs
        "/models": "volume://pretrained-models"   # Persistent storage
    }
)

The vocabulary is the same on every surface (instance_type, env, data_mounts, ...); see docs/vocabulary.md.

Key Features Summary

• Distributed Training – Multi-node InfiniBand clusters auto-configured
• Code Upload – Automatic with .flowignore (or .gitignore fallback)
• Live Debugging – SSH into running instances (flow ssh)
• Cost Protection – Built-in max_price_per_hour safeguards
• Jupyter Integration – Connect notebooks to GPU instances

Documentation: https://docs.mithril.ai/cli-and-sdk/quickstart

Further Reading

• Restoring the Promise of Public Cloud for AI
• Introducing Mithril
• Spot Auction Mechanics