keel-llm-adapter-bedrock

A reference keel-llm-protocol adapter for AWS Bedrock — the enterprise gateway. Use Claude / Llama / Titan / Mistral / Cohere models hosted on Bedrock with Keel's vendor-neutral protocol + reliability + observability stack.

Try it in 5 minutes: the agentic reference demo — docker compose up brings up Jaeger + an agent that exercises the full failover stack. (Swap in BedrockAdapter for the enterprise version.)

Is this for you?

Adopt when — you're on AWS and your model access is via Bedrock (Anthropic Claude, Meta Llama, Mistral, Cohere, Amazon Titan, or other Bedrock-hosted models). You want one stack across direct providers + Bedrock — same ResilientClient, same typed errors, same observability — without writing AWS-specific glue throughout your codebase. Skip when — you call Bedrock directly with boto3 in a simple single-model app and don't need failover / observability / vendor-neutral wiring. The official AWS SDKs have a wider surface (image generation, RAG knowledge bases, agents-for-bedrock features).

Install

pip install keel-llm-adapter-bedrock     # pulls in keel-llm-protocol + httpx + botocore (for SigV4)

Note on dependencies: we use botocore.auth.SigV4Auth for request signing but send via httpx, not boto3. This keeps the [full] OTel story working — opentelemetry-instrumentation-httpx (from keel-llm-otel[full]) sees each Bedrock call as a real distributed-trace span with traceparent propagation.

Use

import os
import asyncio
from keel_llm_adapter_bedrock import BedrockAdapter
from keel_llm_protocol import user
from keel_llm_protocol.errors import RateLimitError, AdapterError

adapter = BedrockAdapter(
    model="anthropic.claude-3-5-sonnet-20241022-v2:0",
    region="us-east-1",
    # Credentials: explicit kwargs OR env vars (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN).
    aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)

async def main() -> None:
    try:
        resp = await adapter.generate([user("Explain Bedrock briefly.")])
        print(resp.text, resp.usage.total_tokens, resp.finish_reason)
    except RateLimitError as e:
        # Bedrock throttling — healthy but rate-limited; the keel-llm-reliability
        # ResilientClient defers to the rate-limiter instead of tripping the breaker.
        ...
    except AdapterError as e:
        if e.retryable:
            ...      # transient / timeout — retry or fail over
        else:
            raise    # auth / bad-request / context / content — fail fast

asyncio.run(main())

In a multi-provider ResilientClient (the enterprise on-ramp)

The reason most enterprise teams reach for this package — mix Bedrock with direct providers in one ResilientClient:

from keel_llm_reliability import ResilientClient, Request
from keel_llm_adapter_bedrock import BedrockAdapter
from keel_llm_adapter_openai import OpenAIAdapter
from keel_llm_protocol import user

client = ResilientClient([
    BedrockAdapter(model="anthropic.claude-3-5-sonnet-20241022-v2:0", region="us-east-1"),
    OpenAIAdapter(model="gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"], provider="openai"),
])

result = await client.failover(Request(messages=[user("Hi")]))
# Bedrock throttles → fails over to OpenAI. Same agent code; same Attempt trail.

Tools

BedrockAdapter implements ToolCallingModelAdapter. The Converse API's toolConfig + toolUse/toolResult blocks are mapped to / from ToolSpec + ToolCall:

from keel_llm_protocol import ToolSpec
weather = ToolSpec(
    name="get_weather",
    description="Get weather for a city.",
    parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
)
resp = await adapter.generate_with_tools([user("Weather in SF?")], [weather])
for call in resp.tool_calls:
    print(call.name, call.arguments)

How Bedrock diverges — and where it's handled

Error mapping

Known limitation

Streaming via converse-stream is not in v0.1.0. Bedrock streams use AWS event-stream binary framing (not SSE), which needs its own parser. Planned for 0.2; capabilities correctly excludes "streaming" so consumers can route around it via capabilities introspection without surprise.

Status

0.1.0 — 0.x while the API stabilizes through year one (breaking changes possible at minor bumps, documented in the CHANGELOG; pin exact versions).

The Keel toolkit

Composable, vendor-neutral LLM reliability libraries on PyPI: keel-llm-reliability · keel-llm-protocol · keel-llm-adapter-openai · keel-llm-adapter-anthropic · keel-llm-adapter-google · keel-llm-adapter-bedrock · keel-circuit-breaker · keel-llm-otel

MIT licensed.