4 projects
smg-grpc-servicer
SMG gRPC servicer implementations for LLM inference engines (vLLM, SGLang, MLX, TokenSpeed)
smg-grpc-proto
SMG gRPC proto definitions for SGLang, vLLM, TRT-LLM, and MLX
smg
High-performance Rust-based inference gateway for large-scale LLM deployments
genai-bench
A powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.
