Arova

One API. Every model. Zero ceremony.

See MIGRATION.md to switch from LiteLLM. See SECURITY.md for our security model.

Arova is a small, typed Python client for calling major hosted and local language-model providers through one stable interface. It uses native adapters where wire formats differ and one universal OpenAI-compatible adapter for the long tail of endpoints.

Quickstart

pip install arova
export OPENAI_API_KEY=sk-...

from arova import completion

response = completion(
    "openai/gpt-5.6-luna",
    [{"role": "user", "content": "Explain zero-copy I/O in one paragraph."}],
)
print(response.text, response.cost)

The model prefix selects a provider. A bare model name uses OpenAI by default, and fallback chains can mix providers: fallbacks=["groq/llama-3.3-70b-versatile", "opencompat/local-model"]. For asynchronous applications, use await arova.acompletion(...) or async for event in arova.astream(...).

Provider coverage

Arova includes native adapters for OpenAI, Anthropic, Gemini, Azure OpenAI, Bedrock, Mistral, Cohere, Groq, DeepSeek, and xAI. It also includes arova.opencompat, which can target any OpenAI-compatible endpoint by setting base_url, model, and key. This covers Together AI, Fireworks AI, OpenRouter, Hugging Face Inference Providers, Ollama, vLLM, LM Studio, Perplexity, Cerebras, SambaNova, NVIDIA NIM, DeepInfra, Novita, and deployment-specific endpoints without adding vendor SDKs. The detailed matrix and source notes are in RESEARCH.md.

from arova.providers.opencompat import OpenCompatProvider
from arova.types import ChatRequest, Message

# Manual base_url setup
provider = OpenCompatProvider(
    base_url="https://api.together.xyz/v1",
    api_key="...",
    provider_name="together",
)

# Or use a built-in preset!
from arova.providers import together, openrouter, ollama, vllm, fireworks
provider = together(api_key="...")

Smart Routing

Arova includes a lightweight heuristics-based smart_completion function that automatically analyzes your prompt and selects the best model for the task (e.g., routing math/logic to reasoning models, and general queries to fast/cheap models), complete with a pre-configured fallback chain.

from arova import smart_completion

# Automatically routes to reasoning models (e.g. o1-preview)
res1 = smart_completion([{"role": "user", "content": "solve this math equation"}])

# Automatically routes to coding models (e.g. claude-3.5-sonnet)
res2 = smart_completion([{"role": "user", "content": "write a python script"}])

Streaming

Streaming yields typed events rather than provider-specific dictionaries. Tool-call arguments may arrive over many deltas and can be reassembled with assemble_tool_calls.

from arova import Arova, TextDelta, Finish

client = Arova()
for event in client.stream("groq/llama-3.3-70b-versatile", [{"role": "user", "content": "Give me three names for a two-faced API."}]):
    if isinstance(event, TextDelta):
        print(event.text, end="", flush=True)
    elif isinstance(event, Finish):
        print(f"\nfinished: {event.reason}")

Tool calling and structured output

The same request types work across native adapters and compatible endpoints. Provider quirks are translated at the boundary.

from arova import completion

response = completion(
    "anthropic/claude-sonnet-4.0",
    [{"role": "user", "content": "What is the weather in Paris?"}],
    tools=[{
        "name": "get_weather",
        "description": "Return current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    }],
)
for call in response.tool_calls:
    print(call.name, call.arguments)

A JSON-schema response can be requested with response_format={"type": "json_schema", "name": "answer", "schema": {...}}. Support depends on the upstream model; Arova preserves the request and normalizes the response when the provider supports it.

Retries, fallbacks, and costs

Arova retries transient transport failures, 408/409/429 responses, and 5xx responses with jittered exponential backoff. A numeric Retry-After header takes precedence. A fallback chain is expressed as model strings, for example fallbacks=["groq/llama-3.3-70b-versatile", "opencompat/local"]. Every non-streaming response includes normalized usage and a deterministic cost estimate from the bundled static price table. Prices are a source-controlled snapshot, not a billing authority; see RESEARCH.md.

Error Handling

If a request completely fails (e.g., invalid API key, network error, or all fallbacks exhausted), Arova handles it depending on the method:

• completion / acompletion: Raises an arova.ProviderError which chains the final underlying exception (e.g., httpx.HTTPStatusError) so standard tracebacks reveal the exact upstream failure.
• stream / astream: Yields an ErrorEvent in the stream containing the failure details, rather than throwing an exception that crashes the active generator.

Caching and Observability

Arova supports pluggable caching and observability hooks without forcing you to install heavy third-party SDKs like Redis or Langfuse.

from arova import Arova
from arova.cache import InMemoryCache
from arova.hooks import ObservabilityHooks

class MyLogger(ObservabilityHooks):
    def on_request(self, request, provider_name):
        print(f"Sending to {provider_name}")
        
    def on_response(self, request, response):
        print(f"Success! Cost: {response.cost}")
        
    def on_error(self, request, error, provider_name):
        print(f"Failed: {error}")

client = Arova(
    cache=InMemoryCache(max_size=1000),
    hooks=MyLogger()
)

CLI

arova --help
arova models
arova cost groq llama-3.3-70b-versatile --input-tokens 1000 --output-tokens 250
arova chat --model openai/gpt-5.6-luna

Benchmarks vs LiteLLM

Arova's SDK overhead is significantly lower than LiteLLM's because it relies on Pydantic's V2 Rust core and avoids massive per-call dictionary mapping, translating directly to the provider.

You can run tests/benchmark.py to test object overhead. Arova aims for <1ms raw framework overhead per call on modern CPUs.

• LiteLLM: Heavy imports, implicit global state, slow dict parsing.
• Arova: ~12ms import time, zero global side-effects, type-safe throughout.

License

Arova is released under the MIT License. See LICENSE.