aops

Python SDK for AOps — prompt version management and agent observability platform.

• Pull prompts from the AOps backend at runtime (any LLM SDK)
• Trace runs — record which chains were called, in what order, and with what latency
• Capture LLM I/O — log inputs and outputs per chain for full observability
• Live updates — background polling reflects prompt edits without redeployment

Installation

pip install aops

With LangChain integration:

pip install "aops[langchain]" langchain-openai

Quick Start

1. Get an API key

In the AOps UI: Agent detail page → API Keys → New API Key

The key embeds the server host — no separate base_url needed.

2. Initialize and pull a prompt

import aops

aops.init(api_key="aops_...", agent="my-agent")

prompt = aops.pull("my-chain")   # returns str

3. Trace a run

Wrap your agent logic in aops.run() to record chain call order and latency:

with aops.run():
    classify_prompt = aops.pull("classify")       # traced
    category = call_llm(classify_prompt, user_input)

    respond_prompt = aops.pull(f"respond-{category}")   # traced
    return call_llm(respond_prompt, user_input)
# → posted to backend on exit, visible in the Flow tab

Capturing LLM Input / Output

Input is captured at pull() time by passing variables. Output is captured after the LLM responds via your chosen integration.

Step 1 — Pass variables to pull()

with aops.run():
    prompt = aops.pull("classify", variables={"inquiry": user_input})
    # ↑ input = rendered prompt (chain instructions + substituted user_input)

Step 2 — Capture output

Choose the integration that fits your stack.

Option A — LangChain / LCEL (AopsCallbackHandler)

from aops.langchain import AopsCallbackHandler
from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage

handler = AopsCallbackHandler()
llm = ChatOpenAI(model="gpt-4o-mini", callbacks=[handler])

with aops.run():
    prompt = aops.pull("classify", variables={"inquiry": user_input})
    result = llm.invoke([SystemMessage(content=prompt), HumanMessage(content=user_input)])
    # output recorded automatically by handler

Option B — OpenAI SDK (wrap())

import openai
from aops import wrap

client = wrap(openai.OpenAI())

with aops.run():
    prompt = aops.pull("classify", variables={"inquiry": user_input})
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "system", "content": prompt}, {"role": "user", "content": user_input}],
    )
    # output recorded automatically by proxy

wrap() supports openai.OpenAI (sync) only. For async, use AopsCallbackHandler.

Option C — Any framework (@aops.trace decorator)

Captures the function's first argument as input and return value as output. Works with any LLM library.

@aops.trace("classify")
def classify(user_input: str) -> str:
    prompt = aops.pull("classify", variables={"inquiry": user_input})
    return call_any_llm(prompt, user_input)

with aops.run():
    result = classify(user_input)

Supports async def and class methods transparently.

Supported LLM SDKs

aops.pull() returns a plain str — works with any LLM SDK out of the box.

OpenAI

from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "system", "content": aops.pull("my-chain")}, ...],
)

Anthropic

from anthropic import Anthropic
client = Anthropic()
message = client.messages.create(
    model="claude-haiku-4-5-20251001",
    max_tokens=1024,
    system=aops.pull("my-chain"),
    messages=[{"role": "user", "content": "Hello!"}],
)

LangChain (prompt as SystemMessagePromptTemplate)

from aops.langchain import pull   # returns SystemMessagePromptTemplate
from langchain_core.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain_openai import ChatOpenAI

prompt = pull("my-chain")
chain = (
    ChatPromptTemplate.from_messages([
        prompt,
        HumanMessagePromptTemplate.from_template("{user_input}"),
    ])
    | ChatOpenAI(model="gpt-4o-mini")
)
result = chain.invoke({"user_input": "Hello!"})

Requirements

• Python 3.12+
• AOps backend running (self-hosted) — see github.com/cow-coding/aops
• API key from the AOps UI

Examples

examples/
  openai_example.py       pull() + wrap() + OpenAI SDK
  anthropic_example.py    pull() + Anthropic SDK
  langchain_example.py    AopsCallbackHandler + @chain_prompt
  live_updates.py         background polling / live update detection

Documentation

Integration Quick References

License

MIT