OpenInference LlamaIndex Instrumentation
Project description
OpenInference LlamaIndex Instrumentation
Python auto-instrumentation library for LlamaIndex.
These traces are fully OpenTelemetry compatible and can be sent to an OpenTelemetry collector for viewing, such as arize-phoenix
.
Installation
pip install openinference-instrumentation-llama-index
Compatibility
llama-index version | openinference-instrumentation-llama-index version |
---|---|
>=0.10.43 | >=2.0.0 |
>=0.10.0, <0.10.43 | >=1.0.0, <0.2 |
>=0.9.14, <0.10.0 | 0.1.3 |
Quickstart
Install packages needed for this demonstration.
python -m pip install --upgrade \
openinference-instrumentation-llama-index \
opentelemetry-sdk \
opentelemetry-exporter-otlp \
"opentelemetry-proto>=1.12.0" \
arize-phoenix
Start the Phoenix app in the background as a collector. By default, it listens on http://localhost:6006
. You can visit the app via a browser at the same address.
The Phoenix app does not send data over the internet. It only operates locally on your machine.
python -m phoenix.server.main serve
The following Python code sets up the LlamaIndexInstrumentor
to trace llama-index
and send the traces to Phoenix at the endpoint shown below.
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)
To demonstrate tracing, we'll use LlamaIndex below to query a document.
First, download a text file.
import tempfile
from urllib.request import urlretrieve
from llama_index.core import SimpleDirectoryReader
url = "https://raw.githubusercontent.com/Arize-ai/phoenix-assets/main/data/paul_graham/paul_graham_essay.txt"
with tempfile.NamedTemporaryFile() as tf:
urlretrieve(url, tf.name)
documents = SimpleDirectoryReader(input_files=[tf.name]).load_data()
Next, we'll query using OpenAI. To do that you need to set up your OpenAI API key in an environment variable.
import os
os.environ["OPENAI_API_KEY"] = "<your openai key>"
Now we can query the indexed documents.
from llama_index.core import VectorStoreIndex
query_engine = VectorStoreIndex.from_documents(documents).as_query_engine()
print(query_engine.query("What did the author do growing up?"))
Visit the Phoenix app at http://localhost:6006
to see the traces.
More Info
More details about tracing with OpenInference and Phoenix can be found in the Phoenix documentation.
For AI/ML observability solutions in production, including a cloud-based trace collector, visit Arize.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openinference_instrumentation_llama_index-2.2.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | dda948e6d7b0a43dcf9f308e412e6df8e8a32e7a7f4303daf12078fe9117992d |
|
MD5 | bf835075d5a0ca3c939609c59bfa1bad |
|
BLAKE2b-256 | cf04f4de46fc1b2ccf687daacebcc8561e20be32d0b7aa726dab5630be298c63 |
Hashes for openinference_instrumentation_llama_index-2.2.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | feb9bb1f968f1084f94d71c86ac65a88f6c3861c1d2558a2aff8a604b1a235b7 |
|
MD5 | 71ab7a0746a314068d35f7f562a7e8d3 |
|
BLAKE2b-256 | 0ec277f7f271a55cc156d2e499cd49bb25f1ecdd8c3bdd3cdb4e28f0e2999836 |