Last released Nov 15, 2024
SGLang is yet another fast serving framework for large language models and vision language models.
Last released Oct 9, 2023
Running large language models like OPT-175B/GPT-3 on a single GPU. Focusing on high-throughput large-batch generation.
Supported by
