litellm-ibm-bob 0.1.0

LiteLLM custom provider for IBM Bob (bobshell) backend

litellm-ibm-bob

LiteLLM custom provider for IBM Bob (bobshell). Replicates the wire protocol of the official bob CLI — including HMAC request signing, key-shape backend selection, and automatic instance/team derivation against /admin/v1/profile.

Install & develop

This project uses uv. All commands assume an active virtualenv:

uv venv                      # creates .venv if not present
source .venv/bin/activate    # activate the venv for the rest of the session
uv sync                      # install runtime + dev deps in editable mode

If a command in this README doesn't start with source .venv/bin/activate, run it inside the activated venv (or prefix with uv run).

Usage from LiteLLM

import litellm
import litellm_ibm_bob  # importing this registers the `ibm-bob` provider

resp = litellm.completion(
    model="ibm-bob/premium",
    messages=[{"role": "user", "content": "hi"}],
    api_key="<BOBSHELL_API_KEY>",
    # instance_id / team_id are optional — derived automatically via
    # /admin/v1/profile, the same way `bob` does. Pass them explicitly only
    # when you need to override (mirrors bob's --instance-id / --team-id flags).
)
print(resp.choices[0].message.content)

Environment / kwarg surface (matches bob itself — no BOBSHELL_INSTANCE_ID or BOBSHELL_TEAM_ID are read, since bob doesn't read them either):

Source	Purpose
BOBSHELL_API_KEY env, or api_key= kwarg	API key. Auto-classified (sk-/pk- legacy, JWT, opaque).
CUSTOM_BASE_URL env, or api_base= kwarg	Override backend host (mirrors bob's $CUSTOM_BASE_URL).
instance_id= kwarg	Override auto-derived instance (mirrors bob --instance-id).
team_id= kwarg	Override auto-derived team (mirrors bob --team-id).

Backend routing

Key shape	Backend	Auth header	HMAC signed
sk-…, pk-… (legacy)	prod.ibm-bob-staging.cloud.ibm.com	Bearer …	no
Parses as a JWT	api.us-east.bob.ibm.com	Bearer …	yes
Anything else (e.g. bob_prod_…)	api.us-east.bob.ibm.com	Apikey …	yes

Testing

The full suite reads exactly one env var, BOBSHELL_API_KEY. Tests for code paths incompatible with the supplied key auto-skip.

source .venv/bin/activate

# Unit tests — no network, runs anywhere
pytest tests/unit -q

# Integration tests — live calls. Skips automatically when BOBSHELL_API_KEY
# is unset. Provides:
#   - legacy-key tests pass when BOBSHELL_API_KEY is sk-/pk-
#   - new-key tests pass when BOBSHELL_API_KEY is JWT/opaque
BOBSHELL_API_KEY=bob_prod_... pytest tests/integration -v

Maintainer helper scripts

These live in scripts/ at the repo root. They are not packaged in the distributable wheel — installing the package does not give the user these helpers. Run them directly with python scripts/<name>.py … inside the venv.

scripts/extract_secrets.py

When to run: Whenever you bump the pinned upstream bob CLI version. The runtime package reads only src/litellm_ibm_bob/_secrets.json; this script is the only supported way to regenerate it.

What it does: Reads a bob Node.js bundle (the bob symlink at /opt/homebrew/bin/bob, the underlying bundle/bob.js, or the pretty-printed source). Locates the embedded BUILD_TIME_HMAC_SECRET marker, derives the AES-256-CBC key via sha256(content-with-marker-stripped), decrypts the secret, and writes a minimal JSON file containing only the two fields the runtime needs: version and build_time_hmac_secret.

source .venv/bin/activate

# Regenerate the committed _secrets.json from your locally installed bob
python scripts/extract_secrets.py /opt/homebrew/bin/bob

# Preview without writing
python scripts/extract_secrets.py /opt/homebrew/bin/bob --stdout

# Write to a custom path
python scripts/extract_secrets.py /opt/homebrew/bin/bob \
    --output /tmp/new-secrets.json

After regenerating, commit the updated src/litellm_ibm_bob/_secrets.json and ship a new release. The wheel build (uv build) embeds the JSON file automatically; verify with unzip -l dist/*.whl.

scripts/http_cli.py

When to run: For manual troubleshooting only — verifying API-key behaviour, exploring undocumented endpoints, checking why a signature is rejected, or capturing a response body for a unit-test fixture. Not invoked by any other part of this codebase, and not needed by end users of the provider.

What it does: Makes a single signed HTTP request against the Bob backend using the same auth/signing pipeline as the runtime provider. Auto-detects backend host and auth scheme from the BOBSHELL_API_KEY shape. Override flags exactly mirror bob's CLI flags (--instance-id, --team-id, --base-url).

source .venv/bin/activate
export BOBSHELL_API_KEY=...

# Smoke-test the modern signed backend
python scripts/http_cli.py GET /inference/v1/model/info

# Smoke-test the legacy backend (auto-selected when key starts with sk-/pk-)
python scripts/http_cli.py GET /v1/model/info

# POST a chat completion from a file
python scripts/http_cli.py POST /inference/v1/chat/completions \
    --body @payload.json

# Override anything `bob` lets you override
python scripts/http_cli.py \
    --instance-id <uuid> --team-id <uuid> \
    --base-url https://api.dev.bob.ibm.com \
    --show-headers \
    GET /admin/v1/profile

# Disable signing (useful for staging hosts that don't validate signatures)
python scripts/http_cli.py --no-sign GET /v1/model/info

Code quality

source .venv/bin/activate
ruff format src tests scripts
ruff check src tests scripts

Ruff is configured for lint + format. The ANN ruleset enforces that public APIs carry type annotations (Ruff itself does not perform type checking).

Build

source .venv/bin/activate
uv build           # produces dist/*.whl and dist/*.tar.gz
unzip -l dist/litellm_ibm_bob-*.whl    # verify _secrets.json is bundled