XAI Plugin for Stream Agents

This package provides xAI (Grok) integration for the Stream Agents ecosystem, enabling you to use xAI's powerful language models in your conversational AI applications.

Features

Native xAI SDK Integration: Full access to xAI's chat completion and streaming APIs
Conversation Memory: Automatic conversation history management
Streaming Support: Real-time response streaming with standardized events
Multimodal Support: Handle text and image inputs
Event System: Subscribe to response events for custom handling
Easy Integration: Drop-in replacement for other LLM providers

Installation

uv add "vision-agents[xai]"
# or directly
uv add vision-agents-plugins-xai

Quick Start

import asyncio
from vision_agents.plugins import xai

async def main():
    # Initialize with your xAI API key
    llm = xai.LLM(
        model="grok-4",
        api_key="your_xai_api_key"  # or set XAI_API_KEY environment variable
    )

    # Simple response
    response = await llm.simple_response("Explain quantum computing in simple terms")

    print(f"\n\nComplete response: {response.text}")

if __name__ == "__main__":
    asyncio.run(main())

Advanced Usage

Conversation with Memory

from vision_agents.plugins import xai

llm = xai.LLM(model="grok-4", api_key="your_api_key")

# First message
await llm.simple_response("My name is Alice and I have 2 cats")

# Second message - the LLM remembers the context
response = await llm.simple_response("How many pets do I have?")
print(response.text)  # Will mention the 2 cats

Using Instructions

llm = LLM(
    model="grok-4",
    api_key="your_api_key"
)

# Create a response with system instructions
response = await llm.create_response(
    input="Tell me about the weather",
    instructions="You are a helpful weather assistant. Always be cheerful and optimistic.",
    stream=True
)

Multimodal Input

# Handle complex multimodal messages
advanced_message = [
    {
        "role": "user",
        "content": [
            {"type": "input_text", "text": "What do you see in this image?"},
            {"type": "input_image", "image_url": "https://example.com/image.jpg"},
        ],
    }
]

messages = LLM._normalize_message(advanced_message)
# Use with your conversation system

API Reference

XAILLM Class

Constructor

LLM(
    model: str = "grok-4",
    api_key: Optional[str] = None,
    client: Optional[AsyncClient] = None
)

Parameters:

model: xAI model to use (default: "grok-4")
api_key: Your xAI API key (default: reads from XAI_API_KEY environment variable)
client: Optional pre-configured xAI AsyncClient

Methods

`async simple_response(text: str, processors=None, participant=None)`

Generate a simple response to text input.

Parameters:

text: Input text to respond to
processors: Optional list of processors for video/voice AI context
participant: Optional participant object

Returns: LLMResponseEvent[Response] with the generated text

`async create_response(input: str, instructions: str = "", model: str = None, stream: bool = True)`

Create a response with full control over parameters.

Parameters:

input: Input text
instructions: System instructions for the model
model: Override the default model
stream: Whether to stream the response (default: True)

Returns: LLMResponseEvent[Response] with the generated text

Configuration

Environment Variables

XAI_API_KEY: Your xAI API key (required if not provided in constructor)

Realtime Speech-to-Speech

The plugin provides an xai.Realtime class for bidirectional voice conversations using xAI's Speech-to-Speech API. The API returns both synthesized speech and transcriptions of the user's input; it is not a standalone speech-to-text integration.

from vision_agents.plugins import xai

realtime = xai.Realtime(
    model="grok-voice-think-fast-2.0",
    voice="ara",
)

Realtime models

Model	Description
`grok-voice-think-fast-2.0`	Current flagship voice model and plugin default
`grok-voice-latest`	Rolling alias; scheduled to point to Think Fast 2.0 on August 5, 2026
`grok-voice-think-fast-1.0`	Previous generation; pin this model to retain 1.0 behavior

The model argument accepts model IDs directly, so applications can choose a pinned release for reproducible behavior or the grok-voice-latest alias for automatic upgrades.

Text-to-Speech (TTS)

The plugin also ships an xai.TTS class powered by xAI's Grok Voice API. It provides five expressive voices with inline speech tags for fine-grained delivery control.

Usage

from vision_agents.plugins import xai

# Default voice (eve) — energetic, upbeat
tts = xai.TTS()

# Specify a voice
tts = xai.TTS(voice="ara")   # warm, friendly
tts = xai.TTS(voice="leo")   # authoritative, strong
tts = xai.TTS(voice="rex")   # confident, clear
tts = xai.TTS(voice="sal")   # smooth, balanced

# Custom output format
tts = xai.TTS(
    voice="rex",
    codec="mp3",
    sample_rate=44100,
    bit_rate=192000,
)

# Explicit API key (otherwise reads XAI_API_KEY env var)
tts = xai.TTS(api_key="xai-your-key-here")

Configuration

Parameter	Type	Default	Description
`api_key`	str	env var	xAI API key. Falls back to `XAI_API_KEY` environment variable.
`voice`	str	`"eve"`	Voice ID: `"eve"`, `"ara"`, `"leo"`, `"rex"`, or `"sal"`.
`language`	str	`"en"`	BCP-47 language code or `"auto"` for detection.
`codec`	str	`"pcm"`	Output codec: `"pcm"`, `"mp3"`, `"wav"`, `"mulaw"`, `"alaw"`.
`sample_rate`	int	`24000`	Sample rate: `8000`–`48000` Hz.
`bit_rate`	int	`None`	MP3 bit rate (only used with `codec="mp3"`).
`base_url`	str	`None`	Override the xAI TTS API endpoint.
`session`	object	`None`	Optional pre-existing `aiohttp.ClientSession`.

Voices

Voice	Tone	Best For
`eve`	Energetic, upbeat	Demos, announcements, upbeat content (default)
`ara`	Warm, friendly	Conversational interfaces, hospitality
`leo`	Authoritative, strong	Instructional, educational, healthcare
`rex`	Confident, clear	Business, corporate, customer support
`sal`	Smooth, balanced	Versatile — works for any context

Speech tags

Add expressiveness to synthesized speech with inline and wrapping tags:

Inline tags (placed where the expression should occur):

Pauses: [pause] [long-pause] [hum-tune]
Laughter: [laugh] [chuckle] [giggle] [cry]
Mouth sounds: [tsk] [tongue-click] [lip-smack]
Breathing: [breath] [inhale] [exhale] [sigh]

Wrapping tags (wrap text to change delivery):

Volume: <soft>text</soft> <loud>text</loud> <shout>text</shout>
Pitch/speed: <high-pitch>text</high-pitch> <low-pitch>text</low-pitch> <slow>text</slow> <fast>text</fast>
Style: <whisper>text</whisper> <sing>text</sing>

MP3 output

MP3 decoding requires pydub. Install it via the mp3 extra:

uv add "vision-agents-plugins-xai[mp3]"

Requirements

Python 3.10+
xai-sdk
vision-agents-core
Optional: pydub (for MP3 decoding via the mp3 extra)

License

Apache-2.0

Metadata

Release files for vision-agents-plugins-xai 0.6.9

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for vision-agents-plugins-xai 0.6.9
File	Size	Uploaded
vision_agents_plugins_xai-0.6.9.tar.gz	17.0 kB	Aug 13, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for vision-agents-plugins-xai 0.6.9
File	Interpreter	ABI	Platform	Reset
vision_agents_plugins_xai-0.6.9-py3-none-any.whl 18.8 kB Aug 13, 2026	Python 3	none	any	Details

Total release size: 35.8 kB

Release files / vision_agents_plugins_xai-0.6.9.tar.gz

Download URL	vision_agents_plugins_xai-0.6.9.tar.gz
Size	17.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`8d49653bb5d2d814cb303810b9953218a001372131f7d441cc0cd405ff8cf1ec`
BLAKE2b-256 checksum How to use checksums	`92f71edfc1c6465aa706ecf766db0d6915d96750aca4ad60ee575bc309d2512b`
Upload date	Aug 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`

Release files / vision_agents_plugins_xai-0.6.9-py3-none-any.whl

Download URL	vision_agents_plugins_xai-0.6.9-py3-none-any.whl
Size	18.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`898ae66466938dfc5dec2237a5e6e9bcebb7b9bd00a55bc65ed9c45e23bf04ae`
BLAKE2b-256 checksum How to use checksums	`d776f0ce0edd317c3ba0dcc43e80ea5af47430d90276b0b6c2598e3c25d60ba7`
Upload date	Aug 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`