Last released Jul 24, 2024
An asynchronous chat engine using vLLM with a async producer-consumer pattern.
Last released Mar 7, 2024
Supported by