lmux-groq

Groq provider for lmux. Talks to the Groq Chat Completions API directly over httpx.

Supports chat completions and streaming.

Part of the lmux ecosystem: standardized interface, cost tracking on every response, and registry-based routing across providers.

Auth

Set GROQ_API_KEY in your environment. The default GroqEnvAuthProvider reads it automatically.

from lmux_groq import GroqProvider

provider = GroqProvider()

Usage

Chat

from lmux import UserMessage

response = provider.chat("llama-3.3-70b-versatile", [UserMessage(content="Hello")])
print(response.content)
print(response.cost)

Streaming

for chunk in provider.chat_stream("llama-3.3-70b-versatile", [UserMessage(content="Hello")]):
    if chunk.delta:
        print(chunk.delta, end="")

Async

All methods have async variants: achat, achat_stream.

Registry

Use with the lmux registry to route across multiple providers:

from lmux import Registry

registry = Registry()
registry.register("groq", provider)
response = registry.chat("groq/llama-3.3-70b-versatile", messages)

Provider Params

from lmux_groq import GroqParams

response = provider.chat(
    "llama-3.3-70b-versatile",
    messages,
    provider_params=GroqParams(service_tier="flex"),
)

Constructor Options

GroqProvider(
    auth=...,          # AuthProvider[str], default: GroqEnvAuthProvider()
    base_url=...,      # Optional base URL override
    timeout=...,       # Request timeout in seconds
    max_retries=...,   # Max retry attempts
    default_headers=...,  # Optional headers included with every request
    transport=...,        # Optional httpx.BaseTransport for the sync client (proxies, testing)
    async_transport=...,  # Optional httpx.AsyncBaseTransport for the async client
)

default_headers is useful for gateway authentication, tracing, and routing. Groq-managed authorization and content-type headers take precedence over caller values, case-insensitively.