dstack 0.18.7rc2

dstack is an open-source orchestration engine for running AI workloads on any cloud or on-premises.

dstack is an open-source container orchestration engine designed for running AI workloads across any cloud or data center. It simplifies dev environments, running tasks on clusters, and deployment.

Cloud and on-prem

dstack allows workload orchestration on both cloud and on-prem servers. Supported cloud providers include AWS, GCP, Azure, OCI, Lambda, TensorDock, Vast.ai, RunPod, and CUDO. For on-prem servers, see fleets.

Accelerators

dstack supports NVIDIA GPUs and Google Cloud TPUs out of the box.

Major news ✨

• [2024/07] dstack 0.18.7rc1: Fleets (Release)
• [2024/05] dstack 0.18.4: Google Cloud TPU, and more (Release)
• [2024/05] dstack 0.18.3: OCI, and more (Release)
• [2024/05] dstack 0.18.2: On-prem clusters, private subnets, and more (Release)
• [2024/04] dstack 0.18.0: RunPod, multi-node tasks, and more (Release)

Installation

Before using dstack through CLI or API, set up a dstack server.

Install the server

The easiest way to install the server, is via pip:

pip install "dstack[all]" -U

Configure backends

If you have default AWS, GCP, Azure, or OCI credentials on your machine, the dstack server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml.

See the server/config.yml reference for details on how to configure backends for all supported cloud providers.

Start the server

To start the server, use the dstack server command:

$ dstack server

Applying ~/.dstack/server/config.yml...

The admin token is "bbae0f28-d3dd-4820-bf61-8f4bb40815da"
The server is running at http://127.0.0.1:3000/

Note It's also possible to run the server via Docker.

Add on-prem servers

If you'd like to use dstack to run workloads on your on-prem servers, see on-prem fleets command.

How does it work?

1. Define configurations

dstack supports three types of run configurations:

• Dev environments — for interactive development using a desktop IDE
• Tasks — for any kind of batch jobs or web applications (supports distributed jobs)
• Services— for production-grade deployment (supports auto-scaling and authorization)

Each type of run configuration allows you to specify commands for execution, required compute resources, retry policies, auto-scaling rules, authorization settings, and more.

Configuration can be defined as YAML files within your repo.

2. Run configurations

Run any defined configuration either via dstack CLI or API.

dstack automatically handles provisioning, interruptions, port-forwarding, auto-scaling, network, volumes, run failures, out-of-capacity errors, and more.

3. Manage fleets

Use fleets to provision and manage clusters and instances, both in the cloud and on-prem.

More information

For additional information and examples, see the following links:

• Docs
• Examples
• Changelog
• Discord

Contributing

You're very welcome to contribute to dstack. Learn more about how to contribute to the project at CONTRIBUTING.md.

License

Mozilla Public License 2.0