Ophelian

Declarative, multi-cloud ML pipelines that route to the cheapest GPU.

User guide · Quickstart · Concepts · Cookbook · CLI · Examples · Changelog · Discussions

Overview

Ophelian is a small, opinionated Python framework for taking ML and AI prototypes to production without rewriting them every time the runtime changes. Declare a Pipeline, pick an env, and Ophelian compiles and runs it — locally, on AWS, GCP, or Azure.

Quick example

from ophelian import Pipeline, Train, Auto

pipe = Pipeline([
    Train(
        model="meta-llama/Llama-3.2-1B",
        data="s3://my-bucket/dataset.jsonl",
        epochs=3,
    ),
])

# Picks the cheapest A100 across AWS / GCP / Azure right now.
pipe.run(env=Auto(cheapest_gpu="A100"))

The same source runs on your laptop, on EC2 Spot, on a GCE preemptible VM, or on an Azure Spot VM. Ophelian handles checkpointing, artifact persistence (S3 / GCS / Azure Blob), structured logs, and a rich summary at the end.

More copy-paste recipes — LLM fine-tuning, distributed training, deploy targets, custom envs — live in the Cookbook.

Highlights

• One pipeline, every cloud. The same Pipeline(...) runs on local Docker, AWS (EC2 / EKS), GCP (GCE), and Azure (Azure VM) with no code change.
• Cost router. Auto(cheapest_gpu="A100") consults live pricing across AWS, GCP, and Azure and selects the cheapest provider/region for the requested GPU class, with a static fallback when APIs are unreachable.
• Spot-aware. Checkpointing and auto-resume work identically on EC2 Spot, GCE preemptible, and Azure Spot. Re-run with OPHELIAN_RESUME_FROM=<run_id>.
• Pluggable adapters. PyTorch, HuggingFace Transformers, scikit-learn, and XGBoost are first-class; third-party adapters register via entry-points.
• Local-first. Every cloud code path has a Local*Driver mirror, so the full test suite exercises the framework offline — no cloud credentials required.
• Honest cost telemetry. Every terminal run appends one row to ~/.ophelian/ledger.jsonl; ophelian costs renders showback as a rich table, Markdown, JSON, or CSV.
• Strictly typed. Pydantic v2 throughout; the entire codebase passes mypy --strict.
• Apache 2.0, no vendor lock-in, Python 3.11+.

Five envs, one API

from ophelian import Standalone, AWS, GCP, Azure, Auto

Standalone(local=True)                                     # local Docker / in-process

AWS(region="us-east-1", instance="g5.xlarge", spot=True)   # EC2 / S3

GCP(project="p", region="us-central1",
    machine_type="n1-standard-4", gpu_type="nvidia-tesla-t4",
    preemptible=True)                                      # GCE / GCS

Azure(subscription_id=..., resource_group="ml",
      region="eastus", vm_size="Standard_NC6s_v3",
      spot=True)                                           # Azure VM / Blob

Auto(cheapest_gpu="A100",
     regions=["us-east-1", "us-central1", "eastus"])       # cost router

The same Pipeline(...) runs on every one of them.

Provenance and cost

The auto-router reports exactly where each price came from — live API, on-disk cache, or static fallback table — so you can audit the decision before spending money on a GPU:

Auto router selected azure/eastus Standard_NC24ads_A100_v4 @ 0.735 USD/h (spot)
  | data: aws=cached@2h gcp=static azure=live | considered: 8 quotes

Need a hard guarantee? Auto(..., require_live=["azure"]) raises instead of silently falling back to a stale or static price.

Every terminal run appends a row to ~/.ophelian/ledger.jsonl. Query it with:

ophelian costs --by team --since 2026-01-01 --format markdown

The ledger schema is versioned and append-only; downstream dashboards, invoicing, and internal showback tools tail the file without us imposing a backend.

Why not just use SageMaker / Vertex AI / Azure ML?

Installation

pip install ophelian                  # core, runs locally
pip install 'ophelian[aws]'           # + EC2 / S3
pip install 'ophelian[gcp]'           # + GCE / GCS
pip install 'ophelian[azure]'         # + Azure VM / Blob
pip install 'ophelian[huggingface]'   # + Transformers + PyTorch
pip install 'ophelian[all]'           # every adapter and provider

Requires Python 3.11+. Full extras list (pytorch, sklearn, xgboost, otel, …) in pyproject.toml.

Verify the install and confirm the CLI is on your $PATH:

ophelian version

Documentation

The hosted user guide lives at https://ophelianio.github.io/ophelian/. Per-topic pages:

• Quickstart — install and run your first pipeline.
• Concepts — pipelines, nodes, envs, providers, drivers, stores, run_id.
• Envs — per-cloud reference (AWS, GCP, Azure, Standalone, Auto).
• Cookbook — copy-paste recipes for common patterns.
• CLI — ophelian run, dry-run, costs, version.
• Cost ledger — schema, showback, integrations.
• Observability — JSON logs, OpenTelemetry, lifecycle events.
• Troubleshooting — when it doesn't work.

Runnable end-to-end examples in examples/. Release history in CHANGELOG.md.

Contributing

Contributions are welcome — from bug reports to new envs and adapters. Start with the contributing guide to set up a local development environment, then browse good first issues. We follow the Contributor Covenant 2.1.

Community

• Questions → GitHub Discussions
• Bugs and features → Issues
• Security → see SECURITY.md — please do not file a public issue

Citation

@software{ophelian_2026,
  author  = {Falva, Luis and the Ophelian contributors},
  title   = {{Ophelian: a declarative, multi-cloud ML pipeline framework}},
  year    = {2026},
  version = {1.1.1},
  license = {Apache-2.0},
  url     = {https://github.com/ophelianio/ophelian},
}

License

Copyright © 2024–2026 Luis Falva and the Ophelian contributors. Licensed under the Apache License, Version 2.0 — see LICENSE and NOTICE for third-party attributions.