dstack 0.12.5rc1

dstack simplifies training, fine-tuning, and deploying generative AI models on any cloud.

Train and deploy generative AI on any cloud

Docs • Learn • Blog • Discord

dstack simplifies training, fine-tuning, and deployment of generative AI models on any cloud.

Supported providers: AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, and DataCrunch.

Latest news ✨

• [2023/12] Leveraging spot instances effectively (Learn)
• [2023/11] Access the GPU marketplace with Vast.ai (Blog)
• [2023/10] Use world's cheapest GPUs with TensorDock (Blog)
• [2023/09] RAG with Llama Index and Weaviate (Learn)
• [2023/08] Fine-tuning Llama 2 using QLoRA (Learn)
• [2023/08] Serving Stable Diffusion using FastAPI (Learn)
• [2023/07] Serving LLMs using TGI (Learn)
• [2023/07] Serving LLMs using vLLM (Learn)

Installation

Before using dstack through CLI or API, set up a dstack server.

Install the server

The easiest way to install the server, is via pip:

$ pip install "dstack[all]" -U

Another way to install the server is through Docker.

Configure the server

If you have default AWS, GCP, or Azure credentials on your machine, the dstack server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml. For further details, refer to server configuration.

Start the server

To start the server, use the dstack server command:

$ dstack server

Applying configuration...
---> 100%

The server is running at http://127.0.0.1:3000/.
The admin token is bbae0f28-d3dd-4820-bf61-8f4bb40815da

More information

For additional information and examples, see the following links:

• Docs
• Learn
• Blog
• Discord

Licence

Mozilla Public License 2.0