exemplar-harness-sdk
Python SDK for Exemplar: session ingest, long-term memory, skills, and prompts.
Quick start covers Agno, OpenAI SDK, Google ADK, and Claude Agent SDK. More frameworks — see Framework extras.
Related package: terminal CLI — exemplar-cli (docs, examples).
Contents
- Install
- Quick start
- Session ingest
- Memory
- Skills
- Prompts
- CLI (separate package)
- Advanced usage
- Framework extras
- Reference
Install
pip install exemplar-harness-sdk
# One agent framework (pick what you use)
pip install "exemplar-harness-sdk[agno]"
pip install "exemplar-harness-sdk[openai]"
pip install "exemplar-harness-sdk[google-adk]"
pip install "exemplar-harness-sdk[claude-agent]"
# Everything
pip install "exemplar-harness-sdk[all]"
export EXEMPLAR_API_KEY="eis_your_org_api_key"
Licensed for non-commercial use only. Commercial use requires a separate license from Exemplar Dev LLC. See LICENSE.
Quick start
Pick your agent framework below. Each section shows session ingest, memory, skills, and prompts for that stack.
Reuse the same session_id for every turn in a conversation. Session helpers call Harness.ingest() for you (Google ADK: call ingest_adk_session after the run). Skills and prompts are shared platform APIs — wire them into your agent with the patterns below.
flowchart LR
Agent[Your agent] --> Integration[SDK integration]
Integration --> Harness[Harness.ingest]
Harness --> API[Exemplar platform API]
API --> Eval[Harness eval]
Agno
pip install "exemplar-harness-sdk[agno]"
Session ingest
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
from exemplar_harness.integrations.agno import harness_agno_post_hook
harness = Harness.from_env()
agent = Agent(
name="support-bot",
model=OpenAIChat(id="gpt-4o"),
post_hooks=[
harness_agno_post_hook(
harness,
session_id="sess-abc",
agent_id="support-bot",
source_app="my-app",
)
],
)
agent.run("Summarize our refund policy.")
Memory
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
from exemplar_harness.integrations.agno import harness_agno_post_hook
from exemplar_harness.integrations.memory.agno import (
make_agno_memory_helper,
harness_agno_memory_hooks,
)
harness = Harness.from_env()
mem = make_agno_memory_helper(
harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)
pre, post = harness_agno_memory_hooks(mem)
agent = Agent(
name="support-bot",
model=OpenAIChat(id="gpt-4o"),
pre_hooks=[pre],
post_hooks=[
post,
harness_agno_post_hook(harness, session_id="sess-abc", agent_id="support-bot"),
],
)
agent.run("What do you know about my formatting preferences?")
Live example: examples/live/agno_memory_demo.py
Skills
Install the skill folder for Agent Skills–compatible runtimes (SKILL.md + files).
Prompt-injection frameworks can also pass skill.instructions (markdown body only).
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
harness = Harness.from_env()
skills = harness.skills()
# Folder-first: dest/<name>/SKILL.md (+ references/, scripts/, assets/)
skills.install(".agents/skills", names=["refund-policy"])
skill = skills.get("refund-policy")
agent = Agent(
name="support-bot",
model=OpenAIChat(id="gpt-4o"),
instructions=[skill.instructions],
)
agent.run("Can a customer return an item after 20 days?")
Live example: examples/live/agno_skills_demo.py
Prompts
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})
system = next((m["content"] for m in built.messages if m["role"] == "system"), "")
user = next((m["content"] for m in built.messages if m["role"] == "user"), "")
agent = Agent(
name="support-bot",
model=OpenAIChat(id="gpt-4o"),
instructions=[system] if system else None,
)
agent.run(user)
OpenAI SDK
pip install "exemplar-harness-sdk[openai]"
Session ingest
from openai import OpenAI
from exemplar_harness import Harness
from exemplar_harness.integrations.openai import HarnessOpenAICallback, sdk_chat_completion
harness = Harness.from_env()
callback = HarnessOpenAICallback(
harness,
session_id="sess-abc",
agent_id="support-bot",
source_app="my-app",
)
client = OpenAI()
sdk_chat_completion(
harness,
callback,
client,
model="gpt-4o",
messages=[{"role": "user", "content": "Summarize our refund policy."}],
)
Memory
from openai import OpenAI
from exemplar_harness import Harness
from exemplar_harness.integrations.memory.openai import (
make_openai_memory_helper,
sdk_chat_completion_with_memory,
)
harness = Harness.from_env()
helper = make_openai_memory_helper(
harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)
client = OpenAI()
sdk_chat_completion_with_memory(
helper,
client,
model="gpt-4o",
messages=[{"role": "user", "content": "What do you know about my formatting preferences?"}],
)
Live example: examples/live/openai_memory_demo.py
Skills
from openai import OpenAI
from exemplar_harness import Harness
harness = Harness.from_env()
skills = harness.skills()
skills.install(".agents/skills", names=["refund-policy"])
skill = skills.get("refund-policy")
client = OpenAI()
client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": skill.instructions},
{"role": "user", "content": "Can a customer return an item after 20 days?"},
],
)
Live example: examples/live/openai_skills_demo.py
Prompts
from openai import OpenAI
from exemplar_harness import Harness
from exemplar_harness.integrations.openai import HarnessOpenAICallback, sdk_chat_completion
harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})
callback = HarnessOpenAICallback(harness, session_id="sess-abc", agent_id="support-bot")
client = OpenAI()
sdk_chat_completion(
harness,
callback,
client,
model=built.model or "gpt-4o",
messages=built.messages,
)
Google ADK
pip install "exemplar-harness-sdk[google-adk]"
Session ingest
import asyncio
from exemplar_harness import Harness
from exemplar_harness.integrations.google_adk import ingest_adk_session
from google.adk.agents import LlmAgent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
harness = Harness.from_env()
session_id, app_name, user_id = "sess-abc", "my_app", "user-1"
agent = LlmAgent(
model="gemini-2.5-flash",
name="support_bot", # must be a valid Python identifier
instruction="Answer concisely.",
)
sessions = InMemorySessionService()
async def run_and_ingest() -> None:
await sessions.create_session(app_name=app_name, user_id=user_id, session_id=session_id)
runner = Runner(agent=agent, app_name=app_name, session_service=sessions)
message = types.Content(role="user", parts=[types.Part(text="Summarize our refund policy.")])
async for _ in runner.run_async(user_id=user_id, session_id=session_id, new_message=message):
pass
session = await sessions.get_session(app_name=app_name, user_id=user_id, session_id=session_id)
ingest_adk_session(
harness,
session.model_dump(mode="json", exclude_none=True),
session_id=session_id,
agent_id="support-bot",
source_app="my-app",
)
asyncio.run(run_and_ingest())
Memory
from exemplar_harness import Harness
from exemplar_harness.integrations.memory.google_adk import (
make_google_adk_memory_helper,
prepare_user_message,
record_session_turn_with_memory,
)
harness = Harness.from_env()
mem = make_google_adk_memory_helper(
harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)
question = "What do you know about my formatting preferences?"
user_text = prepare_user_message(mem, question)
# ... run your ADK agent with user_text, capture model_response ...
# record_session_turn_with_memory(mem, user_message=question, model_response=model_response)
Live example: examples/live/google_adk_memory_demo.py
Skills
from exemplar_harness import Harness
from google.adk.agents import LlmAgent
harness = Harness.from_env()
skills = harness.skills()
skills.install(".agents/skills", names=["refund-policy"])
skill = skills.get("refund-policy")
agent = LlmAgent(
model="gemini-2.5-flash",
name="support_bot",
instruction=skill.instructions,
)
Live example: examples/live/google_adk_skills_demo.py
Prompts
from exemplar_harness import Harness
from google.adk.agents import LlmAgent
from google.genai import types
harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})
system = next((m["content"] for m in built.messages if m["role"] == "system"), "Answer concisely.")
user = next((m["content"] for m in built.messages if m["role"] == "user"), "")
agent = LlmAgent(model="gemini-2.5-flash", name="support_bot", instruction=system)
message = types.Content(role="user", parts=[types.Part(text=user)])
# pass `message` into Runner.run_async(...), then ingest_adk_session as above
Claude Agent SDK
pip install "exemplar-harness-sdk[claude-agent]"
Session ingest
import asyncio
from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import HarnessClaudeAgentHandler
harness = Harness.from_env()
handler = HarnessClaudeAgentHandler(
harness,
session_id="sess-abc",
agent_id="support-bot",
source_app="my-app",
)
asyncio.run(handler.run_query("Summarize our refund policy."))
Memory
import asyncio
from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import HarnessClaudeAgentHandler
from exemplar_harness.integrations.memory.claude_agent import (
make_claude_agent_memory_helper,
prepare_prompt,
record_agent_result_with_memory,
)
harness = Harness.from_env()
handler = HarnessClaudeAgentHandler(harness, session_id="sess-abc", agent_id="support-bot")
mem = make_claude_agent_memory_helper(
harness, user_id="user-123", session_id="sess-abc", app_id="my-app"
)
async def run() -> None:
question = "What do you know about my formatting preferences?"
prompt = prepare_prompt(mem, question)
result, _ = await handler.run_query(prompt)
record_agent_result_with_memory(mem, prompt=question, result=result)
asyncio.run(run())
Live example: examples/live/claude_agent_memory_demo.py
Skills
import asyncio
from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import (
HarnessClaudeAgentHandler,
merge_claude_agent_options,
)
harness = Harness.from_env()
skills = harness.skills()
# Claude Code also discovers folders under .claude/skills via CLI install/pull
skills.install(".claude/skills", names=["refund-policy"])
skill = skills.get("refund-policy")
handler = HarnessClaudeAgentHandler(harness, session_id="sess-abc", agent_id="support-bot")
async def run() -> None:
from claude_agent_sdk import ClaudeAgentOptions
options = merge_claude_agent_options(
handler,
ClaudeAgentOptions(system_prompt=skill.instructions),
)
await handler.run_query("Can a customer return an item after 20 days?", options=options)
asyncio.run(run())
Live example: examples/live/claude_agent_skills_demo.py
Prompts
import asyncio
from exemplar_harness import Harness
from exemplar_harness.integrations.claude_agent import (
HarnessClaudeAgentHandler,
merge_claude_agent_options,
)
harness = Harness.from_env()
built = harness.prompts().build("support-summary", variables={"topic": "returns"})
system = next((m["content"] for m in built.messages if m["role"] == "system"), "")
user = next((m["content"] for m in built.messages if m["role"] == "user"), "")
handler = HarnessClaudeAgentHandler(harness, session_id="sess-abc", agent_id="support-bot")
async def run() -> None:
from claude_agent_sdk import ClaudeAgentOptions
options = merge_claude_agent_options(
handler,
ClaudeAgentOptions(system_prompt=system or None),
)
await handler.run_query(user, options=options)
asyncio.run(run())
Session ingest
Base helper: Harness.ingest() / framework helpers above.
Example 1 — Framework helper (Agno)
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from exemplar_harness import Harness
from exemplar_harness.integrations.agno import harness_agno_post_hook
harness = Harness.from_env()
agent = Agent(
name="support-bot",
model=OpenAIChat(id="gpt-4o"),
post_hooks=[harness_agno_post_hook(harness, session_id="sess-abc", agent_id="support-bot")],
)
agent.run("What is the return window?")
Example 2 — Direct ingest (no framework)
from exemplar_harness import Harness
harness = Harness.from_env()
harness.ingest(
"generic",
session_id="sess-abc",
event="turns",
data={
"turns": [
{
"input": "What is harness eval?",
"output": "Automated judge over agent sessions.",
"model": "gpt-4o",
}
]
},
agent_id="my-agent",
source_app="my-app",
)
| Parameter | Purpose |
|---|---|
session_id |
Groups turns into one eval session |
agent_id |
Agent id on the ingest body (agentId); set on Harness(..., agent_id=...) to also send X-Harness-Agent-Id on MCP tool calls |
source_app |
Your application name |
auto_judge_run |
Run harness judge after ingest (True / False / omit) |
Memory
Base helper: harness.memory(...).
Example 1 — Add and recall
from exemplar_harness import Harness
harness = Harness.from_env()
memory = harness.memory(user_id="user-123", session_id="chat-abc", app_id="my-app")
memory.add("User prefers bullet-point answers.", memory_type="preference")
context = memory.recall("how should I format answers?") # inject into system prompt
Example 2 — Search and update
results = memory.search("formatting preferences", top_k=5)
listed = memory.list(limit=20)
record = memory.get(listed[0].memory_id)
memory.update(record.memory_id, content="User prefers numbered lists.")
memory.delete(record.memory_id)
Skills
Base helper: harness.skills().
Example 1 — Create and list
from exemplar_harness import Harness
harness = Harness.from_env()
skills = harness.skills()
record = skills.create(
name="refund-policy",
instructions="# Refund policy\n\nReturns within 30 days.\n\nSee [references/policy.md](references/policy.md).",
description="Refund workflow",
tags=["support"],
files={"references/policy.md": "# Policy\n\n30-day returns.\n"},
)
items = skills.list(limit=20)
Example 2 — Get, search, and install
fetched = skills.get("refund-policy") # by ID or unique name
hits = skills.search("refund", top_k=5)
# Materialize SKILL.md + supporting files for agent runtimes
skills.install(".agents/skills", names=["refund-policy"])
Prompts
Base helper: harness.prompts().
Example 1 — Create and run
from exemplar_harness import Harness
harness = Harness.from_env()
prompts = harness.prompts()
record = prompts.create(
name="support-summary",
title="Support summary",
messages=[
{"role": "system", "content": "Be concise."},
{"role": "user", "content": "Summarize topic: {{topic}}."},
],
variables=["topic"],
)
result = prompts.run("support-summary", variables={"topic": "returns"})
print(result["content"])
Example 2 — Build for your own agent framework
Fetch a stored prompt, substitute {{variables}} locally, and hand messages to LangGraph, OpenAI, Haystack, etc. Does not call Exemplar’s model runner.
built = prompts.build("support-summary", variables={"topic": "returns"})
# built.messages -> [{"role": "system", ...}, {"role": "user", ...}]
# built.model -> default model from the registry (if set)
# pass built.messages into your framework / LLM client
Or reuse an already-fetched record (no second network call):
record = prompts.get("support-summary")
built = record.build(variables={"topic": "returns"})
Example 3 — List and get
listed = prompts.list(limit=20)
fetched = prompts.get("support-summary") # by ID or unique name
hits = prompts.search("support", top_k=5)
CLI (separate package)
Skills, prompts, and memory are also available as a standalone terminal package — not bundled with this SDK:
| PyPI | exemplar-cli |
| Docs | exemplar-cli/README.md |
| Examples | examples/cli/README.md |
pip install exemplar-cli
export EXEMPLAR_API_KEY="eis_your_org_api_key"
exemplar doctor
Advanced usage
One advanced pattern per offering. For full framework wiring, see examples/live/.
Session ingest — Session helper with auto judge
from exemplar_harness import Harness
harness = Harness.from_env()
session = harness.session(
"sess-abc",
agent_id="support-bot",
source_app="my-app",
auto_judge_run=True, # run judge after each ingest through this session
)
session.ingest(
"generic",
event="turns",
data={"turns": [{"input": "Hello", "output": "Hi!", "model": "gpt-4o"}]},
)
Memory — Generic hook in any agent loop
from exemplar_harness import Harness
from exemplar_harness.integrations.memory import MemoryHook
harness = Harness.from_env()
memory = harness.memory(user_id="user-123", session_id="chat-abc", app_id="my-app")
hook = MemoryHook(memory, recall_top_k=5, auto_add=False)
user_input = "What do you know about me?"
recall = hook.before_turn(user_input) # prepend to system prompt
# ... run LLM ...
hook.after_turn(user_input, assistant_output) # no-op unless auto_add=True
Skills — Install folders for agent runtimes
A skill is a directory rooted at SKILL.md (plus optional references/, scripts/, assets/).
Install the whole folder for Cursor, Claude Code, ADK, Agno, Deep Agents, AG2, CrewAI, and similar agents —
do not treat record.instructions as the full skill (that field is the markdown body only,
useful for quick editor/MCP use).
Live skills demos also cover framework-native loaders:
- Deep Agents:
examples/live/deepagents_skills_demo.py(create_deep_agent(..., skills=[...])) - AG2:
examples/live/ag2_skills_demo.py(SkillPlugin) - CrewAI:
examples/live/crewai_skills_demo.py(Agent(..., skills=[...])) - Agno / OpenAI / Google ADK / Claude Agent: see Quick start sections above
from exemplar_harness import Harness
skills = Harness.from_env().skills()
# Materialize all active skills: dest/<name>/SKILL.md + supporting files
paths = skills.install(".agents/skills")
# Or one skill into Cursor project skills
skills.install(".cursor/skills", names=["refund-policy"])
CLI equivalent: exemplar skills install --all --dest .agents/skills (alias of pull).
Prompts — Publish, build for frameworks, or run inline
from exemplar_harness import Harness
prompts = Harness.from_env().prompts()
prompts.publish_version(
"support-summary",
messages=[
{"role": "system", "content": "Be concise."},
{"role": "user", "content": "Bullet summary for: {{topic}}."},
],
change_notes="Use bullets",
)
# Hand off to your agent framework (local {{var}} substitution)
built = prompts.build("support-summary", variables={"topic": "returns"})
# Or execute via Exemplar without a stored prompt
inline = prompts.run_inline(
messages=[
{"role": "system", "content": "Be concise."},
{"role": "user", "content": "Say hello."},
],
model="openai/gpt-4o-mini",
)
Framework extras
| Framework | Extra | Wire this |
|---|---|---|
| LangChain | langchain |
make_langchain_callback_handler → callbacks=[handler] |
| LangGraph | langgraph |
HarnessLangGraphHandler.make_graph_callback_handler() |
| Deep Agents | deepagents |
HarnessDeepAgentsHandler.make_graph_callback_handler() / record_run |
| LiteLLM | litellm |
register_litellm_handler + metadata={"session_id": ...} |
| OpenAI SDK | openai |
HarnessOpenAICallback.on_completion |
| Anthropic SDK | anthropic |
make_anthropic_middleware |
| Claude Agent SDK | claude-agent |
HarnessClaudeAgentHandler.run_query |
| Portkey | portkey |
HarnessPortkeyCallback.on_completion |
| Agno | agno |
harness_agno_post_hook → Agent.post_hooks |
| Haystack | haystack |
HarnessHaystackHandler.run_and_record |
| LlamaIndex | llamaindex |
register_llamaindex_handler |
| AutoGen | autogen |
HarnessAutoGenHandler.on_agent_run_complete |
| AG2 | ag2 |
HarnessAG2Handler.record_ask / record_chat_turn |
| CrewAI | crewai |
make_crewai_listener |
| Google GenAI SDK | google-genai |
instrument_google_genai_client |
| Google ADK | google-adk |
ingest_adk_session |
| Pydantic AI | pydantic-ai |
instrument_pydantic_ai_agent |
| Semantic Kernel | semantic-kernel |
register_semantic_kernel_filter |
| smolagents | smolagents |
harness_smolagents_step_callback |
Matching memory helpers live under exemplar_harness.integrations.memory.*.
# List / run live demos
python -m examples.live.run_all --list
python -m examples.live.run_memory_demos --list
python -m examples.live.run_platform_demos --list
HARNESS_EXAMPLES_SIMULATED=1 python -m examples.live.run_memory_demos --only memory_agno
HARNESS_EXAMPLES_SIMULATED=1 python -m examples.live.run_platform_demos --only skills_openai
Reference
Optional install extras
| Extra | Module |
|---|---|
langchain |
exemplar_harness.integrations.langchain |
langgraph |
exemplar_harness.integrations.langgraph |
deepagents |
exemplar_harness.integrations.deepagents |
litellm |
exemplar_harness.integrations.litellm |
openai |
exemplar_harness.integrations.openai |
anthropic |
exemplar_harness.integrations.anthropic |
claude-agent |
exemplar_harness.integrations.claude_agent |
portkey |
exemplar_harness.integrations.portkey |
agno |
exemplar_harness.integrations.agno |
haystack |
exemplar_harness.integrations.haystack |
llamaindex |
exemplar_harness.integrations.llamaindex |
autogen |
exemplar_harness.integrations.autogen |
ag2 |
exemplar_harness.integrations.ag2 |
crewai |
exemplar_harness.integrations.crewai |
google-adk |
exemplar_harness.integrations.google_adk |
google-genai |
exemplar_harness.integrations.google_genai |
pydantic-ai |
exemplar_harness.integrations.pydantic_ai |
semantic-kernel |
exemplar_harness.integrations.semantic_kernel |
smolagents |
exemplar_harness.integrations.smolagents |
Vercel AI SDK is not included in the Python package (TypeScript SDK path).
Envelope v1
Each ingest POSTs schemaVersion: 1 to POST /api/harness-ingest/v1/sessions with a sourceType and framework-native data payload. The Exemplar platform API maps envelopes to eval session turns. Harness judge runs use POST /api/harness-judge/v1/runs; per-turn session eval uses POST /api/harness-session-eval/v1/sessions/{sessionId}.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file exemplar_harness_sdk-0.2.15.tar.gz.
File metadata
- Download URL: exemplar_harness_sdk-0.2.15.tar.gz
- Upload date:
- Size: 50.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.8.2 CPython/3.12.4 Darwin/24.5.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
169197c102ef07cc2eb9ef77c9b64a3320c34aac344426b11daa2392109ff24b
|
|
| MD5 |
8a5dae06aa4d28af7e285ed43bb2957a
|
|
| BLAKE2b-256 |
fa2d6d53c503162406d049db0e01e01b1ba43de3b7923f468f42af9259cffb09
|
File details
Details for the file exemplar_harness_sdk-0.2.15-py3-none-any.whl.
File metadata
- Download URL: exemplar_harness_sdk-0.2.15-py3-none-any.whl
- Upload date:
- Size: 83.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.8.2 CPython/3.12.4 Darwin/24.5.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e02ea48687cc10e83941482bce71ebce28ea5d6e42d8c59d98eb892d8a2cbd47
|
|
| MD5 |
1f8bf66cef77eb918d58b879bc3c43f1
|
|
| BLAKE2b-256 |
4c5ec3ace6a6c73c0e8cb07c3723cb56f87e6f0704d94ef0cdc583051ba2790e
|