OpenInference Bedrock Instrumentation
Project description
OpenInference AWS Bedrock Instrumentation
Python autoinstrumentation library for AWS Bedrock calls made using boto3
.
This package implements OpenInference tracing for invoke_model
calls made using a boto3
bedrock-runtime
client. These traces are fully OpenTelemetry compatible and can be sent to an OpenTelemetry collector for viewing, such as Arize phoenix
.
Installation
pip install openinference-instrumentation-bedrock
Quickstart
In a notebook environment (jupyter
, colab
, etc.) install openinference-instrumentation-bedrock
, arize-phoenix
and boto3
.
You can test out this quickstart guide in Google Colab!
pip install openinference-instrumentation-bedrock arize-phoenix boto3
Ensure that boto3
is configured with AWS credentials.
First, import dependencies required to autoinstrument AWS Bedrock and set up phoenix
as an collector for OpenInference traces.
from urllib.parse import urljoin
import boto3
import phoenix as px
from openinference.instrumentation.bedrock import BedrockInstrumentor
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
Next, we'll start a phoenix
server and set it as a collector.
px.launch_app()
session_url = px.active_session().url
phoenix_otlp_endpoint = urljoin(session_url, "v1/traces")
phoenix_exporter = OTLPSpanExporter(endpoint=phoenix_otlp_endpoint)
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter=phoenix_exporter))
trace_api.set_tracer_provider(tracer_provider=tracer_provider)
Instrumenting boto3
is simple:
BedrockInstrumentor().instrument()
Now, all calls to invoke_model
are instrumented and can be viewed in the phoenix
UI.
session = boto3.session.Session()
client = session.client("bedrock-runtime")
prompt = b'{"prompt": "Human: Hello there, how are you? Assistant:", "max_tokens_to_sample": 1024}'
response = client.invoke_model(modelId="anthropic.claude-v2", body=prompt)
response_body = json.loads(response.get("body").read())
print(response_body["completion"])
More Info
More details about tracing with OpenInference and phoenix
can be found in the phoenix
documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openinference_instrumentation_bedrock-0.1.5.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97e67c58ee0f6ef883c308c395204d354a3312d547e03b8792c92fdb5f34cba8 |
|
MD5 | 8b38e15b0699175307fdeb770a2491ae |
|
BLAKE2b-256 | 7b282a43f0a4974b9effc0ec335aa4aa26ce67cf7156c2330bfd0ee041426d1a |
Hashes for openinference_instrumentation_bedrock-0.1.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a08eabbb5bf979dc1a44a38243360f47b433c47d2ae86135e39dd53fca4e8d24 |
|
MD5 | 8d0aef06569d33768c84acf4d118befc |
|
BLAKE2b-256 | 28d0f312e722ea89e041546c3cad11600a546829e518bd5d42b6274c65f853bd |