13 projects
smg-grpc-proto
SMG gRPC proto definitions for SGLang, vLLM, TRT-LLM, and MLX
smg
High-performance Rust-based inference gateway for large-scale LLM deployments
sglang-router
High-performance Rust-based load balancer for SGLang with multiple routing algorithms and prefill-decode disaggregation support
smg-mcp
Model Context Protocol (MCP) implementation for Shepherd Model Gateway
smg-auth
Authentication and authorization for Shepherd Model Gateway
wfaas
Workflow as a Service - workflow orchestration for LLM applications
smg-multimodal
Multimodal (image/video) processing for LLM inference
openai-protocol
OpenAI-compatible protocol types and utilities
reasoning-parser
Reasoning and chain-of-thought parser for LLM outputs
tool-parser
Tool/function call parser for LLM outputs
llm-tokenizer
Tokenizer utilities for LLM inference
kv-index
KV cache index for LLM inference - radix tree and page table implementations
amg
app manager tool
