parallax-ai 0.2.2

Lightweight parallel AI API calls in Python.

Parallax-AI

Lightweight parallel AI API calls in Python.

Installation

You can install Parallax using pip:

pip install parallax-ai

Testing

vllm serve google/gemma-3-27b-it

python -m parallax_ai.test
>>>
ParallaxOpenAIClient.completions:
First Output Elapsed Time: 7.10s
Total Elapsed Time (500 requires): 7.10s
ParallaxOpenAIClient.icompletions:
First Output Elapsed Time: 2.85s
Total Elapsed Time (500 requires): 6.99s
ParallaxOpenAIClient.chat_completions:
First Output Elapsed Time: 7.32s
Total Elapsed Time (500 requires): 7.32s
ParallaxOpenAIClient.ichat_completions:
First Output Elapsed Time: 2.85s
Total Elapsed Time (500 requires): 7.09s
Vanilla OpenAI Client:
First Output Elapsed Time: 1.78s
Total Elapsed Time (500 requires): 672.02s

Usage (Compatible with any OpenAI-API–compatible server, e.g., vLLM)

Initialize Client

from parallax_ai import ParallaxOpenAIClient

# Initialize Client
parallax_client = ParallaxOpenAIClient(
    api_key=YOUR_API_KEY,
    base_url=YOUR_API_BASE_URL,
)

list_of_messages = [
    [{"role": "user", "content": "..."}],
    [{"role": "user", "content": "..."}],
    [{"role": "user", "content": "..."}],
    .
    .
    .
    [{"role": "user", "content": "..."}],
]

chat_completions: Returns a list of outputs in order (waits until all are finished)

outputs = parallax_client.chat_completions(list_of_messages, model="gpt-3.5-turbo")
for output in outputs:
    # PROCESS OUTPUT
    pass

ichat_completions: Returns outputs one by one in order (yields as soon as each finishes)

for output in parallax_client.ichat_completions(list_of_messages, model="gpt-3.5-turbo"):
    # PROCESS OUTPUT
    pass

ichat_completions_unordered: Returns outputs as they finish (order not guaranteed)

for output, index in parallax_client.ichat_completions_unordered(list_of_messages, model="gpt-3.5-turbo"):
    # PROCESS OUTPUT
    pass