Deepgram SageMaker Transport

SageMaker transport for the Deepgram Python SDK. Uses AWS SageMaker's HTTP/2 bidirectional streaming API as an alternative to WebSocket, allowing transparent switching between Deepgram Cloud and Deepgram on SageMaker.

Requires Python 3.12+ (due to AWS SDK dependencies).

Installation

pip install "deepgram-sdk>=7.8.1,<8.0.0" deepgram-sagemaker

deepgram-sagemaker installs aws-sdk-sagemaker-runtime-http2 0.11.x with its awscrt extra, plus boto3. awscrt uses a compiled extension; supported platforms install a wheel, while source builds require a C toolchain.

Usage

The SageMaker transport is async-only and must be used with AsyncDeepgramClient:

import asyncio
from deepgram import AsyncDeepgramClient
from deepgram.core.events import EventType
from deepgram_sagemaker import SageMakerTransportFactory

factory = SageMakerTransportFactory(
    endpoint_name="my-deepgram-endpoint",
    region="us-west-2",
)

# SageMaker uses AWS credentials (not Deepgram API keys)
client = AsyncDeepgramClient(api_key="unused", transport_factory=factory)

async def main():
    async with client.listen.v1.connect(model="nova-3") as connection:
        connection.on(EventType.MESSAGE, lambda msg: print(msg))
        await connection.start_listening()

asyncio.run(main())

Configuration

For burst-tuned timeouts and retry behavior, build a SageMakerConfig and pass it via config=:

from deepgram_sagemaker import SageMakerConfig, SageMakerTransportFactory

config = SageMakerConfig(
    endpoint_name="my-deepgram-endpoint",
    region="us-east-2",
    connection_timeout=5.0,
    connection_acquire_timeout=15.0,
)
factory = SageMakerTransportFactory(config=config)

All time-based fields are float seconds (matching asyncio convention).

High-concurrency notes

The transport's defaults are tuned for high-burst workloads (large numbers of streams opened in a tight loop against an endpoint that may need to scale up). If you're opening 200–500 streams simultaneously against a cold endpoint, the AWS SDK's general-purpose defaults (~2 s connect, ~10 s acquire) will fire before the load balancer has accepted all of the inbound TLS handshakes — you will see a wave of acquire / connect timeouts that look like server-side problems but are really client-side fail-fast tripping early.

This transport ships with more lenient defaults (30 s / 60 s) so the common high-concurrency path works out of the box. Tighten them if you need fail-fast behavior in low-latency pipelines:

config = SageMakerConfig(
    endpoint_name="my-deepgram-endpoint",
    region="us-east-2",
    connection_timeout=5.0,
    connection_acquire_timeout=15.0,
)

Retry & storm absorption

Transient AWS-side failures (ThrottlingException, connection-pool exhaustion, transient connect/timeout failures) are absorbed by the transport itself: classified as retryable, retried with jittered exponential backoff up to max_retries and retry_budget, with messages buffered during the reset window replayed onto the new stream so audio isn't dropped. Only terminal errors (auth, validation, resource-not-found) and budget-exhausted retryable errors propagate to the application.

config = SageMakerConfig(
    endpoint_name="my-deepgram-endpoint",
    max_retries=10,
    initial_backoff=0.2,
    max_backoff=10.0,
    retry_budget=60.0,
)

Set max_retries=0 to disable internal retry entirely (every transient AWS error then surfaces immediately to the application).

AWS Credentials

The transport resolves AWS credentials using boto3's credential chain:

• Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• Shared credentials file (~/.aws/credentials)
• IAM role (EC2, ECS, Lambda)

Links

• Deepgram Python SDK
• SageMaker example
• Deepgram documentation