dstack 0.13.1rc2

dstack is an open-source toolkit for running GPU workloads on any cloud.

Orchestrate GPU workloads effortlessly on any cloud

Docs • Examples • Discord

dstack is an open-source toolkit and orchestration engine for running GPU workloads on any cloud.

Supported providers: AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, and DataCrunch.

Latest news ✨

• [2023/12] dstack 0.13.0: Disk size, CUDA 12.1, Mixtral, and more (Release)
• [2023/11] dstack 0.12.3: Vast.ai integration (Release)
• [2023/10] dstack 0.12.2: TensorDock integration (Release)
• [2023/09] RAG with Llama Index and Weaviate (Example)
• [2023/08] Fine-tuning with QLoRA (Example)
• [2023/07] Deploying with TGI (Example)
• [2023/07] Deploying with vLLM (Example)

Installation

Before using dstack through CLI or API, set up a dstack server.

Install the server

The easiest way to install the server, is via pip:

pip install "dstack[all]" -U

Configure credentials

If you have default AWS, GCP, or Azure credentials on your machine, the dstack server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml.

For further details on setting up the server, refer to installation.

Start the server

To start the server, use the dstack server command:

$ dstack server

Applying configuration from ~/.dstack/server/config.yml...

The server is running at http://127.0.0.1:3000/
The admin token is "bbae0f28-d3dd-4820-bf61-8f4bb40815da"

Note It's also possible to run the server via Docker.

More information

For additional information and examples, see the following links:

• Docs
• Examples
• Discord

Licence

Mozilla Public License 2.0