parallem 0.0.3

A library for building LLM pipelines with built-in support for batch processing and multi-provider compatibility. Beta prerelease - use at your own risk.

ParaLLeM

Agents + Batch API.

• Batch API support (50% discount!)
• Expressive. Agents are simple Python functions.
• Concise, lightweight.
• Durable by default.
• Built for >1 million parallel requests.

Quickstart

pip install parallem

import parallem as pllm

# Place OPENAI_API_KEY in .env file
with pllm.resume_directory(
    ".pllm/simplest",
    provider="openai",
    strategy="sync",
    dashboard=True,
    load_dotenv=True,
) as orch:
    with orch.agent() as agt:
        resp = agt.ask_llm("Please name a power of 3.")
        print(resp.final_answer)

To switch to the Batch API, simply change strategy="sync" to strategy="batch".

Responses are saved and cached. On the subsequent runs, answers are instant.

Compatibility

Sync/Batch	OpenAI	Anthropic	Google
Simple	✅	✅	✅
Structured Output	✅	✅	✅
Function Calls	✅	✅	✅
Web Search	✅	✅	✅
Image Input	✅	✅	✅
MCP	✅	✅	-

Examples

• Agents are python functions
• Structured output
• Tool calls
• Web search
• Image input
• Local MCP
• Server MCP
• 1 million requests
• Ollama

Documentation

Please refer to the documentation.

parallem.org