blablador

Python client and command-line tool for Blablador, the LLM service of the Jülich Supercomputing Centre.

Blablador's API is OpenAI-compatible, so this package is a thin layer over the official openai SDK: everything the SDK does works unchanged, and the package adds what Blablador needs on top:

• clients preconfigured for Blablador (Blablador, AsyncBlablador)
• typed model listing with Blablador's fields and the current alias-* names
• the model's reasoning, whichever backend serves it (blablador.reasoning())
• the rerank endpoint, which the OpenAI API does not have (client.rerank)
• fixes for requests the gateway rejects but the OpenAI API allows
• rate-limit handling that does not extend Blablador's blocks: no retry of HTTP 429 by default, and an opt-in wait for the time the server states
• a blablador command: models, chat, transcribe

Status: early. Requires Python 3.10 or newer. The client overrides internals of the openai SDK, so each release pins one minor version of openai (0.2.1: openai>=3.42,<3.43); the pin is raised once the tests pass with a newer release.

Install

pip install blablador

This installs the library and the blablador command. The development version installs from the repository: pip install "git+https://gitlab.jsc.fz-juelich.de/sdlaml/agents-on-hpc/blablador.git".

If you use uv, add it to your project, or install only the command, available everywhere:

uv add blablador
uv tool install blablador

Getting started

You need an API key: a Helmholtz Codebase token with the read_user scope (sign in with your institution's account). Tokens are valid for at most a year. Put it in the environment:

export BLABLADOR_API_KEY=...

Then use Blablador like openai.OpenAI, which it is a subclass of:

from blablador import Blablador

client = Blablador()  # reads BLABLADOR_API_KEY
completion = client.chat.completions.create(
    model="alias-fast",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(completion.choices[0].message.content)

Streaming, the Responses API, tool calls, embeddings, audio transcription and with_options() work as the OpenAI SDK documentation describes. AsyncBlablador is the same for openai.AsyncOpenAI.

From the shell:

blablador chat "What is the Jülich Supercomputing Centre?"

What the package adds

Models and aliases

Blablador's models change. The alias-* names (alias-fast, alias-code, alias-large, ...) stay, and are pointed at newer models as these replace older ones, so use aliases in code and read what is behind them at runtime rather than hardcoding a table.

client.models.list() returns blablador.BlabladorModels, which add Blablador's fields to the OpenAI ones:

Field	Meaning
root	the underlying model: a Hugging Face name, a path on the server, or None
max_model_len	context length in tokens, if reported
client_count	number of servers currently serving the model, if reported
is_alias	True for the alias-* names

for model in client.models.list():
    if model.is_alias:
        print(model.id, model.root, model.max_model_len)

client.models.aliases()  # {"alias-fast": "openai/gpt-oss-120b", ...}, one request

In chat completions, the response's model field holds the model's display name ("01 - GPT-OSS-120b - ..."), not the alias that was sent; embeddings responses name the alias. client.models.retrieve() does not work: Blablador answers 404 for a single model. The listing does not say what a model can do (chat, embeddings, rerank, speech); the names are the only hint.

Reasoning

Reasoning models return their reasoning next to the answer, in a field the OpenAI API does not define and whose name depends on the backend (reasoning from vLLM, reasoning_content from sglang). blablador.reasoning() returns it from a message or a streamed delta, or None:

import blablador

message = completion.choices[0].message
print(blablador.reasoning(message))  # how the model got there
print(message.content)  # the answer

for chunk in client.chat.completions.create(..., stream=True):
    if chunk.choices:
        print(blablador.reasoning(chunk.choices[0].delta) or "", end="")

Rerank

client.rerank.create() scores documents by relevance to a query (Blablador's /v1/rerank), best first. Use it to pick the best few of many candidates, for example after an embedding search:

texts = [
    "sbatch submits a batch script to Slurm",
    "JURECA has A100 GPUs",
    "The cafeteria opens at 11:30",
]
ranked = client.rerank.create(
    model="alias-qwen3-8b-reranking",
    query="How do I submit a job on JURECA?",
    documents=texts,
    top_n=2,  # optional: only the best two
)
for result in ranked.results:  # sorted by relevance_score, highest first
    print(f"{result.relevance_score:.3f}  {texts[result.index]}")

return_documents=True adds each document's text to its result. AsyncBlablador has the same method as a coroutine.

Requests the gateway rejects

Where the gateway is stricter than the OpenAI API, the client adjusts the request so that code written for OpenAI keeps working:

• Function tools without a description (optional for OpenAI, HTTP 400 from Blablador) are sent with "description": "", for chat completions and the Responses API.

Rate limits

Blablador limits requests per model and user, and blocks a key that exceeds the limit: 60 s for the first violation, doubling with each further one up to an hour. The OpenAI SDK would retry an HTTP 429 at once, which risks a longer block, so this client never retries 429 by default (timeouts, 5xx and connection errors are still retried, max_retries times). Instead it raises blablador.RateLimitError, a subclass of openai.RateLimitError, whose retry_after holds the wait in seconds if the server stated one, in a Retry-After header or in the error message ("try again in 57 seconds"):

import time
import blablador

try:
    completion = client.chat.completions.create(...)
except blablador.RateLimitError as err:
    time.sleep(err.retry_after or 60)  # then try again, once

For batch jobs, the client can do the waiting itself:

client = blablador.Blablador(rate_limit_retries=3, max_rate_limit_wait=600)

A request that gets a 429 then waits the stated time (plus a second) and is sent again, up to rate_limit_retries times. It is not retried, and the error is raised, when the server did not say how long to wait, since a guessed wait that ends too early may extend the block, or when the wait is longer than max_rate_limit_wait seconds (default 3600). The client records every stated block, also one it does not wait out. While the key is blocked, other requests through the same client, from any thread, and through its with_options() copies wait for the block to end instead of being sent into it, or raise RateLimitError at once if the rest of the block is longer than max_rate_limit_wait. When the block ends, all waiting requests are sent together, so with many threads the client may exceed the limit again; it does not pace requests. A request whose upload cannot be read again (a stream that cannot seek back) is not retried. This applies only to clients with rate_limit_retries set: a copy made with rate_limit_retries=0, like a plain client, sends every request. Clients created separately, and other processes using the same key, do not know about the block.

How Blablador states the wait in a 429 has not been confirmed yet (no real 429 has been recorded, since that would block the key). If it uses a form the client does not recognise, retry_after is None and rate_limit_retries raises instead of waiting.

Command line

The blablador command covers the quick uses: see which models are there, ask one question, transcribe a recording. It reads the API key from BLABLADOR_API_KEY, like the client, and every command sends one request. python -m blablador works too, blablador COMMAND --help lists the options, and blablador --version prints the versions of blablador and openai for bug reports.

blablador models                      # aliases, context length, the model behind each
blablador models --all --json         # every model, as JSON
blablador chat "What is JSC?"         # streams the answer
cat notes.txt | blablador chat "Summarize these notes"   # piped text follows the prompt
blablador chat --reasoning -m alias-code "Is 91 prime?"  # reasoning on stderr, answer on stdout
blablador transcribe meeting.m4a -l en                   # speech to text

blablador models

Lists the alias-* names with their context length and the model behind each, aliases only unless --all is given.

Option	Meaning
--all	also list the full model names, after the aliases
--json	print a JSON list with id, root, max_model_len, owned_by and client_count per model, for scripts

blablador chat [PROMPT ...]

Sends one prompt and streams the answer to stdout. The words of the prompt need no quotes. When stdin is not a terminal, it is read to its end and appended after the prompt, separated by a blank line, so PROMPT can be left out when the whole prompt is piped. There is no conversation: each call is a new chat.

Under srun, or from a script that starts blablador with an open stdin, nothing may ever close stdin, and the command would wait for it. After two seconds it says so on stderr; pass --no-stdin (or < /dev/null) there.

Option	Meaning
-m, --model MODEL	model or alias; default $BLABLADOR_MODEL, else alias-fast
-s, --system TEXT	system prompt
--reasoning	also print the model's reasoning, to stderr, so that > answer.txt keeps only the answer
--no-stream	wait for the complete answer and print it at once
--no-stdin	do not read stdin, even when it is not a terminal

blablador transcribe FILE

Prints the text of an audio file (wav, mp3, m4a, ...). Blablador has no alias-* name for speech to text, so the default names the model itself; if it is renamed, blablador models --all shows the current one.

Option	Meaning
-m, --model MODEL	speech-to-text model; default faster-whisper-large-v3
-l, --language CODE	language of the audio, e.g. en or de; optional

Exit status and errors

The exit status is 0 on success, 1 for a missing API key or an error from the API, 2 for wrong usage (unknown option, missing prompt or file) and 130 after Ctrl-C. Errors are printed to stderr without a traceback, with the same hints as the client (for example, that the key may have expired). The command does not retry rate-limited requests: a 429 is reported, with the wait if the server stated one.

Examples

• examples/quickstart.py: models, chat, streaming, tool calls, rerank and the async client, as sections you can run one at a time: python examples/quickstart.py [models|chat|stream|tools|rerank|async ...] (with uv: uv run python examples/quickstart.py ...)
• examples/agent_loop.py: a minimal coding agent, one bash tool and a loop. It runs the shell commands the model picks without any checks, so use a throwaway directory.

Configuration

Variable	Default
BLABLADOR_API_KEY	required: a Helmholtz Codebase token with read_user scope
BLABLADOR_BASE_URL	https://api.blablador.fz-juelich.de/v1
BLABLADOR_MODEL	alias-fast: the model blablador chat uses without -m

Arguments (Blablador(api_key=..., base_url=...)) take precedence over the variables. OPENAI_* variables are ignored, so an OpenAI setup in the same shell does not send its key or headers to Blablador. When the key is missing or rejected, the error says where to get a new one.

Development

In a virtual environment, with pip 25.1 or newer (for --group):

pip install -e . --group dev
pytest                     # offline tests, replaying recorded responses in tests/recordings/
pytest -m live             # a few requests against the real API (needs BLABLADOR_API_KEY)
ruff check . && ruff format --check .
python -m tests.record [SCENARIO ...]   # re-record tests/recordings/ from the live API

With uv, uv sync --group dev sets up the environment, and each command runs with uv run in front (uv run pytest).

GitLab CI runs ruff and the offline tests on Python 3.10 and 3.15, and installs the built wheel, for every push, merge request and tag; live tests run only locally. Blablador's rate limits are tight and repeated violations block the key for longer each time, so keep live tests and re-recordings to what you need. The recorder strips credentials and internal server paths; check the diff of tests/recordings/ before committing.

Planned and finished work is tracked with tatr in tasks/.

License

MIT, see LICENSE.