dstack 0.18.7

dstack is an open-source orchestration engine for running AI workloads on any cloud or on-premises.

dstack is an open-source container orchestration engine for AI. It accelerates the development, training, and deployment of AI models, and simplifies the management of clusters.

Cloud and on-prem

dstack is easy to use with any cloud or on-prem servers. Supported cloud providers include AWS, GCP, Azure, OCI, Lambda, TensorDock, Vast.ai, RunPod, and CUDO. For using dstack with on-prem servers, see fleets.

Accelerators

dstack supports NVIDIA GPU and Google Cloud TPU out of the box.

Major news ✨

• [2024/07] dstack 0.18.7rc1: Fleets (Release)
• [2024/05] dstack 0.18.4: Google Cloud TPU, and more (Release)
• [2024/05] dstack 0.18.3: OCI, and more (Release)
• [2024/05] dstack 0.18.2: On-prem clusters, private subnets, and more (Release)
• [2024/04] dstack 0.18.0: RunPod, multi-node tasks, and more (Release)

Installation

Before using dstack through CLI or API, set up a dstack server.

Install the server

The easiest way to install the server, is via pip:

pip install "dstack[all]" -U

Configure backends

If you have default AWS, GCP, Azure, or OCI credentials on your machine, the dstack server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml.

See the server/config.yml reference for details on how to configure backends for all supported cloud providers.

Start the server

To start the server, use the dstack server command:

$ dstack server

Applying ~/.dstack/server/config.yml...

The admin token is "bbae0f28-d3dd-4820-bf61-8f4bb40815da"
The server is running at http://127.0.0.1:3000/

Note It's also possible to run the server via Docker.

Add on-prem servers

If you'd like to use dstack to run workloads on your on-prem servers, see on-prem fleets command.

How does it work?

1. Define run configurations

dstack supports three types of run configurations:

• Dev environments — for interactive development using a desktop IDE
• Tasks — for any kind of batch jobs or web applications (supports distributed jobs)
• Services— for production-grade deployment (supports auto-scaling and authorization)

Each type of run configuration allows you to specify commands for execution, required compute resources, retry policies, auto-scaling rules, authorization settings, and more.

Configuration can be defined as YAML files within your repo.

2. Run configurations

Run any defined configuration either via dstack CLI or API.

dstack automatically handles provisioning, interruptions, port-forwarding, auto-scaling, network, volumes, run failures, out-of-capacity errors, and more.

3. Manage fleets

Use fleets to provision and manage clusters and instances, both in the cloud and on-prem.

More information

For additional information and examples, see the following links:

• Docs
• Examples
• Changelog
• Discord

Contributing

You're very welcome to contribute to dstack. Learn more about how to contribute to the project at CONTRIBUTING.md.

License

Mozilla Public License 2.0