Skip to main content

exemplar-harness-sdk

Python SDK for Exemplar: session ingest, long-term memory, skills, and prompts.

Quick start covers Agno, OpenAI SDK, Google ADK, and Claude Agent SDK. More frameworks — see Framework extras.

Related package: terminal CLI — exemplar-cli (docs, examples).

Contents

Install

pip install exemplar-harness-sdk

# One agent framework (pick what you use)
pip install "exemplar-harness-sdk[agno]"
pip install "exemplar-harness-sdk[openai]"
pip install "exemplar-harness-sdk[google-adk]"
pip install "exemplar-harness-sdk[claude-agent]"

# Everything
pip install "exemplar-harness-sdk[all]"
export EXEMPLAR_API_KEY="eis_your_org_api_key"

Licensed for non-commercial use only. Commercial use requires a separate license from Exemplar Dev LLC. See LICENSE.


Quick start

Pick your agent framework below. Each section shows session ingest, memory, skills, and prompts for that stack.

Reuse the same session_id for every turn in a conversation. Session helpers call Harness.ingest() for you (Google ADK: call ingest_adk_session after the run). Skills and prompts are shared platform APIs — wire them into your agent with the patterns below.

flowchart LR
  Agent[Your agent] --> Integration[SDK integration]
  Integration --> Harness[Harness.ingest]
  Harness --> API[Exemplar platform API]
  API --> Eval[Harness eval]

Agno

pip install "exemplar-harness-sdk[agno]"

Session ingest

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
from exemplar_harness.integrations.agno import harness_agno_post_hook

harness = Harness.from_env()
agent = Agent(
    name="support-bot",
    model=OpenAIChat(id="gpt-4o"),
    post_hooks=[
        harness_agno_post_hook(
            harness,
            session_id="sess-abc",
            agent_name="support-bot",
            source_app="my-app",
        )
    ],
)
agent.run("Summarize our refund policy.")

Memory

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
from exemplar_harness.integrations.agno import harness_agno_post_hook
from exemplar_harness.integrations.memory.agno import (
    make_agno_memory_helper,
    harness_agno_memory_hooks,
)

harness = Harness.from_env()
mem = make_agno_memory_helper(
    harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)
pre, post = harness_agno_memory_hooks(mem)

agent = Agent(
    name="support-bot",
    model=OpenAIChat(id="gpt-4o"),
    pre_hooks=[pre],
    post_hooks=[
        post,
        harness_agno_post_hook(harness, session_id="sess-abc", agent_name="support-bot"),
    ],
)
agent.run("What do you know about my formatting preferences?")

Live example: examples/live/agno_memory_demo.py

Skills

Install the skill folder for Agent Skills–compatible runtimes (SKILL.md + files). Prompt-injection frameworks can also pass skill.instructions (markdown body only).

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness

harness = Harness.from_env()
skills = harness.skills()

# Folder-first: dest/<name>/SKILL.md (+ references/, scripts/, assets/)
skills.install(".agents/skills", names=["refund-policy"])

skill = skills.get("refund-policy")
agent = Agent(
    name="support-bot",
    model=OpenAIChat(id="gpt-4o"),
    instructions=[skill.instructions],
)
agent.run("Can a customer return an item after 20 days?")

Live example: examples/live/agno_skills_demo.py

Prompts

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness

harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})

system = next((m["content"] for m in built.messages if m["role"] == "system"), "")
user = next((m["content"] for m in built.messages if m["role"] == "user"), "")

agent = Agent(
    name="support-bot",
    model=OpenAIChat(id="gpt-4o"),
    instructions=[system] if system else None,
)
agent.run(user)

OpenAI SDK

pip install "exemplar-harness-sdk[openai]"

Session ingest

from openai import OpenAI
from exemplar_harness import Harness
from exemplar_harness.integrations.openai import HarnessOpenAICallback, sdk_chat_completion

harness = Harness.from_env()
callback = HarnessOpenAICallback(
    harness,
    session_id="sess-abc",
    agent_name="support-bot",
    source_app="my-app",
)
client = OpenAI()
sdk_chat_completion(
    harness,
    callback,
    client,
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize our refund policy."}],
)

Memory

from openai import OpenAI
from exemplar_harness import Harness
from exemplar_harness.integrations.memory.openai import (
    make_openai_memory_helper,
    sdk_chat_completion_with_memory,
)

harness = Harness.from_env()
helper = make_openai_memory_helper(
    harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)
client = OpenAI()
sdk_chat_completion_with_memory(
    helper,
    client,
    model="gpt-4o",
    messages=[{"role": "user", "content": "What do you know about my formatting preferences?"}],
)

Live example: examples/live/openai_memory_demo.py

Skills

from openai import OpenAI
from exemplar_harness import Harness

harness = Harness.from_env()
skills = harness.skills()
skills.install(".agents/skills", names=["refund-policy"])

skill = skills.get("refund-policy")
client = OpenAI()
client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": skill.instructions},
        {"role": "user", "content": "Can a customer return an item after 20 days?"},
    ],
)

Live example: examples/live/openai_skills_demo.py

Prompts

from openai import OpenAI
from exemplar_harness import Harness
from exemplar_harness.integrations.openai import HarnessOpenAICallback, sdk_chat_completion

harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})
callback = HarnessOpenAICallback(harness, session_id="sess-abc", agent_name="support-bot")
client = OpenAI()
sdk_chat_completion(
    harness,
    callback,
    client,
    model=built.model or "gpt-4o",
    messages=built.messages,
)

Google ADK

pip install "exemplar-harness-sdk[google-adk]"

Session ingest

import asyncio

from exemplar_harness import Harness
from exemplar_harness.integrations.google_adk import ingest_adk_session
from google.adk.agents import LlmAgent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types

harness = Harness.from_env()
session_id, app_name, user_id = "sess-abc", "my_app", "user-1"

agent = LlmAgent(
    model="gemini-2.5-flash",
    name="support_bot",  # must be a valid Python identifier
    instruction="Answer concisely.",
)
sessions = InMemorySessionService()


async def run_and_ingest() -> None:
    await sessions.create_session(app_name=app_name, user_id=user_id, session_id=session_id)
    runner = Runner(agent=agent, app_name=app_name, session_service=sessions)
    message = types.Content(role="user", parts=[types.Part(text="Summarize our refund policy.")])
    async for _ in runner.run_async(user_id=user_id, session_id=session_id, new_message=message):
        pass
    session = await sessions.get_session(app_name=app_name, user_id=user_id, session_id=session_id)
    ingest_adk_session(
        harness,
        session.model_dump(mode="json", exclude_none=True),
        session_id=session_id,
        agent_name="support-bot",
        source_app="my-app",
    )


asyncio.run(run_and_ingest())

Memory

from exemplar_harness import Harness
from exemplar_harness.integrations.memory.google_adk import (
    make_google_adk_memory_helper,
    prepare_user_message,
    record_session_turn_with_memory,
)

harness = Harness.from_env()
mem = make_google_adk_memory_helper(
    harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)

question = "What do you know about my formatting preferences?"
user_text = prepare_user_message(mem, question)
# ... run your ADK agent with user_text, capture model_response ...
# record_session_turn_with_memory(mem, user_message=question, model_response=model_response)

Live example: examples/live/google_adk_memory_demo.py

Skills

from exemplar_harness import Harness
from google.adk.agents import LlmAgent

harness = Harness.from_env()
skills = harness.skills()
skills.install(".agents/skills", names=["refund-policy"])

skill = skills.get("refund-policy")
agent = LlmAgent(
    model="gemini-2.5-flash",
    name="support_bot",
    instruction=skill.instructions,
)

Live example: examples/live/google_adk_skills_demo.py

Prompts

from exemplar_harness import Harness
from google.adk.agents import LlmAgent
from google.genai import types

harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})

system = next((m["content"] for m in built.messages if m["role"] == "system"), "Answer concisely.")
user = next((m["content"] for m in built.messages if m["role"] == "user"), "")

agent = LlmAgent(model="gemini-2.5-flash", name="support_bot", instruction=system)
message = types.Content(role="user", parts=[types.Part(text=user)])
# pass `message` into Runner.run_async(...), then ingest_adk_session as above

Claude Agent SDK

pip install "exemplar-harness-sdk[claude-agent]"

Session ingest

import asyncio

from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import HarnessClaudeAgentHandler

harness = Harness.from_env()
handler = HarnessClaudeAgentHandler(
    harness,
    session_id="sess-abc",
    agent_name="support-bot",
    source_app="my-app",
)
asyncio.run(handler.run_query("Summarize our refund policy."))

Memory

import asyncio

from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import HarnessClaudeAgentHandler
from exemplar_harness.integrations.memory.claude_agent import (
    make_claude_agent_memory_helper,
    prepare_prompt,
    record_agent_result_with_memory,
)

harness = Harness.from_env()
handler = HarnessClaudeAgentHandler(harness, session_id="sess-abc", agent_name="support-bot")
mem = make_claude_agent_memory_helper(
    harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)


async def run() -> None:
    question = "What do you know about my formatting preferences?"
    prompt = prepare_prompt(mem, question)
    result, _ = await handler.run_query(prompt)
    record_agent_result_with_memory(mem, prompt=question, result=result)


asyncio.run(run())

Live example: examples/live/claude_agent_memory_demo.py

Skills

import asyncio

from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import (
    HarnessClaudeAgentHandler,
    merge_claude_agent_options,
)

harness = Harness.from_env()
skills = harness.skills()
# Claude Code also discovers folders under .claude/skills via CLI install/pull
skills.install(".claude/skills", names=["refund-policy"])

skill = skills.get("refund-policy")
handler = HarnessClaudeAgentHandler(harness, session_id="sess-abc", agent_name="support-bot")


async def run() -> None:
    from claude_agent_sdk import ClaudeAgentOptions

    options = merge_claude_agent_options(
        handler,
        ClaudeAgentOptions(system_prompt=skill.instructions),
    )
    await handler.run_query("Can a customer return an item after 20 days?", options=options)


asyncio.run(run())

Live example: examples/live/claude_agent_skills_demo.py

Prompts

import asyncio

from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import (
    HarnessClaudeAgentHandler,
    merge_claude_agent_options,
)

harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})
system = next((m["content"] for m in built.messages if m["role"] == "system"), "")
user = next((m["content"] for m in built.messages if m["role"] == "user"), "")
handler = HarnessClaudeAgentHandler(harness, session_id="sess-abc", agent_name="support-bot")


async def run() -> None:
    from claude_agent_sdk import ClaudeAgentOptions

    options = merge_claude_agent_options(
        handler,
        ClaudeAgentOptions(system_prompt=system or None),
    )
    await handler.run_query(user, options=options)


asyncio.run(run())

Session ingest

Base helper: Harness.ingest() / framework helpers above.

Example 1 — Framework helper (Agno)

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
from exemplar_harness.integrations.agno import harness_agno_post_hook

harness = Harness.from_env()
agent = Agent(
    name="support-bot",
    model=OpenAIChat(id="gpt-4o"),
    post_hooks=[harness_agno_post_hook(harness, session_id="sess-abc", agent_name="support-bot")],
)
agent.run("What is the return window?")

Example 2 — Direct ingest (no framework)

from exemplar_harness import Harness

harness = Harness.from_env()
harness.ingest(
    "generic",
    session_id="sess-abc",
    event="turns",
    data={
        "turns": [
            {
                "input": "What is harness eval?",
                "output": "Automated judge over agent sessions.",
                "model": "gpt-4o",
            }
        ]
    },
    agent_name="my-agent",
    source_app="my-app",
)
Parameter Purpose
session_id Groups turns into one eval session
agent_name Agent identifier on the ingest body
source_app Your application name
auto_judge_run Run harness judge after ingest (True / False / omit)

Memory

Base helper: harness.memory(...).

Example 1 — Add and recall

from exemplar_harness import Harness

harness = Harness.from_env()
memory = harness.memory(user_id="user-123", session_id="chat-abc", app_id="my-app")

memory.add("User prefers bullet-point answers.", memory_type="preference")
context = memory.recall("how should I format answers?")  # inject into system prompt

Example 2 — Search and update

results = memory.search("formatting preferences", top_k=5)
listed = memory.list(limit=20)
record = memory.get(listed[0].memory_id)
memory.update(record.memory_id, content="User prefers numbered lists.")
memory.delete(record.memory_id)

Skills

Base helper: harness.skills().

Example 1 — Create and list

from exemplar_harness import Harness

harness = Harness.from_env()
skills = harness.skills()

record = skills.create(
    name="refund-policy",
    instructions="# Refund policy\n\nReturns within 30 days.\n\nSee [references/policy.md](references/policy.md).",
    description="Refund workflow",
    tags=["support"],
    files={"references/policy.md": "# Policy\n\n30-day returns.\n"},
)
items = skills.list(limit=20)

Example 2 — Get, search, and install

fetched = skills.get("refund-policy")  # by ID or unique name
hits = skills.search("refund", top_k=5)

# Materialize SKILL.md + supporting files for agent runtimes
skills.install(".agents/skills", names=["refund-policy"])

Prompts

Base helper: harness.prompts().

Example 1 — Create and run

from exemplar_harness import Harness

harness = Harness.from_env()
prompts = harness.prompts()

record = prompts.create(
    name="support-summary",
    title="Support summary",
    messages=[
        {"role": "system", "content": "Be concise."},
        {"role": "user", "content": "Summarize topic: {{topic}}."},
    ],
    variables=["topic"],
)
result = prompts.run("support-summary", variables={"topic": "returns"})
print(result["content"])

Example 2 — Build for your own agent framework

Fetch a stored prompt, substitute {{variables}} locally, and hand messages to LangGraph, OpenAI, Haystack, etc. Does not call Exemplar’s model runner.

built = prompts.build("support-summary", variables={"topic": "returns"})
# built.messages -> [{"role": "system", ...}, {"role": "user", ...}]
# built.model    -> default model from the registry (if set)
# pass built.messages into your framework / LLM client

Or reuse an already-fetched record (no second network call):

record = prompts.get("support-summary")
built = record.build(variables={"topic": "returns"})

Example 3 — List and get

listed = prompts.list(limit=20)
fetched = prompts.get("support-summary")  # by ID or unique name
hits = prompts.search("support", top_k=5)

CLI (separate package)

Skills, prompts, and memory are also available as a standalone terminal package — not bundled with this SDK:

PyPI exemplar-cli
Docs exemplar-cli/README.md
Examples examples/cli/README.md
pip install exemplar-cli
export EXEMPLAR_API_KEY="eis_your_org_api_key"
exemplar doctor

Advanced usage

One advanced pattern per offering. For full framework wiring, see examples/live/.

Session ingest — Session helper with auto judge

from exemplar_harness import Harness

harness = Harness.from_env()
session = harness.session(
    "sess-abc",
    agent_name="support-bot",
    source_app="my-app",
    auto_judge_run=True,  # run judge after each ingest through this session
)
session.ingest(
    "generic",
    event="turns",
    data={"turns": [{"input": "Hello", "output": "Hi!", "model": "gpt-4o"}]},
)

Memory — Generic hook in any agent loop

from exemplar_harness import Harness
from exemplar_harness.integrations.memory import MemoryHook

harness = Harness.from_env()
memory = harness.memory(user_id="user-123", session_id="chat-abc", app_id="my-app")
hook = MemoryHook(memory, recall_top_k=5, auto_add=False)

user_input = "What do you know about me?"
recall = hook.before_turn(user_input)  # prepend to system prompt
# ... run LLM ...
hook.after_turn(user_input, assistant_output)  # no-op unless auto_add=True

Skills — Install folders for agent runtimes

A skill is a directory rooted at SKILL.md (plus optional references/, scripts/, assets/). Install the whole folder for Cursor, Claude Code, ADK, Agno, and similar agents — do not treat record.instructions as the full skill (that field is the markdown body only, useful for quick editor/MCP use).

from exemplar_harness import Harness

skills = Harness.from_env().skills()

# Materialize all active skills: dest/<name>/SKILL.md + supporting files
paths = skills.install(".agents/skills")

# Or one skill into Cursor project skills
skills.install(".cursor/skills", names=["refund-policy"])

CLI equivalent: exemplar skills install --all --dest .agents/skills (alias of pull).

Prompts — Publish, build for frameworks, or run inline

from exemplar_harness import Harness

prompts = Harness.from_env().prompts()

prompts.publish_version(
    "support-summary",
    messages=[
        {"role": "system", "content": "Be concise."},
        {"role": "user", "content": "Bullet summary for: {{topic}}."},
    ],
    change_notes="Use bullets",
)

# Hand off to your agent framework (local {{var}} substitution)
built = prompts.build("support-summary", variables={"topic": "returns"})

# Or execute via Exemplar without a stored prompt
inline = prompts.run_inline(
    messages=[
        {"role": "system", "content": "Be concise."},
        {"role": "user", "content": "Say hello."},
    ],
    model="openai/gpt-4o-mini",
)

Framework extras

Framework Extra Wire this
LangChain langchain make_langchain_callback_handlercallbacks=[handler]
LangGraph langgraph HarnessLangGraphHandler.make_graph_callback_handler()
LiteLLM litellm register_litellm_handler + metadata={"session_id": ...}
OpenAI SDK openai HarnessOpenAICallback.on_completion
Anthropic SDK anthropic make_anthropic_middleware
Claude Agent SDK claude-agent HarnessClaudeAgentHandler.run_query
Portkey portkey HarnessPortkeyCallback.on_completion
Agno agno harness_agno_post_hookAgent.post_hooks
Haystack haystack HarnessHaystackHandler.run_and_record
LlamaIndex llamaindex register_llamaindex_handler
AutoGen autogen HarnessAutoGenHandler.on_agent_run_complete
CrewAI crewai make_crewai_listener
Google GenAI SDK google-genai instrument_google_genai_client
Google ADK google-adk ingest_adk_session
Pydantic AI pydantic-ai instrument_pydantic_ai_agent
Semantic Kernel semantic-kernel register_semantic_kernel_filter
smolagents smolagents harness_smolagents_step_callback

Matching memory helpers live under exemplar_harness.integrations.memory.*.

# List / run live demos
python -m examples.live.run_all --list
python -m examples.live.run_memory_demos --list
python -m examples.live.run_platform_demos --list
HARNESS_EXAMPLES_SIMULATED=1 python -m examples.live.run_memory_demos --only memory_agno
HARNESS_EXAMPLES_SIMULATED=1 python -m examples.live.run_platform_demos --only skills_openai

Reference

Optional install extras

Extra Module
langchain exemplar_harness.integrations.langchain
langgraph exemplar_harness.integrations.langgraph
litellm exemplar_harness.integrations.litellm
openai exemplar_harness.integrations.openai
anthropic exemplar_harness.integrations.anthropic
claude-agent exemplar_harness.integrations.claude_agent
portkey exemplar_harness.integrations.portkey
agno exemplar_harness.integrations.agno
haystack exemplar_harness.integrations.haystack
llamaindex exemplar_harness.integrations.llamaindex
autogen exemplar_harness.integrations.autogen
crewai exemplar_harness.integrations.crewai
google-adk exemplar_harness.integrations.google_adk
google-genai exemplar_harness.integrations.google_genai
pydantic-ai exemplar_harness.integrations.pydantic_ai
semantic-kernel exemplar_harness.integrations.semantic_kernel
smolagents exemplar_harness.integrations.smolagents

Vercel AI SDK is not included in the Python package (TypeScript SDK path).

Envelope v1

Each ingest POSTs schemaVersion: 1 to POST /api/harness-ingest/v1/sessions with a sourceType and framework-native data payload. The Exemplar platform API maps envelopes to eval session turns. Harness judge runs use POST /api/harness-judge/v1/runs; per-turn session eval uses POST /api/harness-session-eval/v1/sessions/{sessionId}.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exemplar_harness_sdk-0.2.14.tar.gz (48.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

exemplar_harness_sdk-0.2.14-py3-none-any.whl (79.6 kB view details)

Uploaded Python 3

File details

Details for the file exemplar_harness_sdk-0.2.14.tar.gz.

File metadata

  • Download URL: exemplar_harness_sdk-0.2.14.tar.gz
  • Upload date:
  • Size: 48.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.6

File hashes

Hashes for exemplar_harness_sdk-0.2.14.tar.gz
Algorithm Hash digest
SHA256 df14962c8532f4fe695d20db8507906cf73810e21a93a64d190b9423431e9bd1
MD5 e3518a6dc5ba31387d1c0026dcd28cd1
BLAKE2b-256 18762460459fcdaa1a3fd9a23f03084d061f032db875a5a7b4ff8ebe11f3b4df

See more details on using hashes here.

File details

Details for the file exemplar_harness_sdk-0.2.14-py3-none-any.whl.

File metadata

File hashes

Hashes for exemplar_harness_sdk-0.2.14-py3-none-any.whl
Algorithm Hash digest
SHA256 10dc3a4026bde4a247c00cbb3951cb3c5c91f8ed58aae3f076523175bb4c733a
MD5 a460986d8027538aba760e4565a975ab
BLAKE2b-256 5d3bad71eb87b361e7aa5191eca3b939a36c3ce4b09faf80adf8d06594c1d252

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page