Rootmark SDK

Agent instrumentation for Rootmark — observe, verify and govern agent runs.

The SDK records what your agents actually did: the span tree of a run, the model calls inside it, token counts, and the payloads you choose to keep. It sends them to a Rootmark collector, which you can self-host.

pip install rootmark

Requires Python 3.12 or newer.

Upgrading from 0.1.0 needs a collector upgrade too. Since 0.1.1 a span is sent when it starts as well as when it ends, so that work in progress is visible at all. A collector older than this release stores the first row it sees for a span and discards the rest — which is the end row, the one carrying the result and the duration. Runs would appear and never finish. Upgrade the collector first (it needs ClickHouse migrations 007–009), or stay on 0.1.0.

0.1.1 is broken; use 0.1.2 or later. In 0.1.1 a span that the SDK opened and then discarded was left looking like it was still running, which held its whole run open permanently. Any Bedrock call made through boto3 inside a Strands agent hits it, on every call.

What changed in 0.1.1

• Runs are visible while they are still going. A span is recorded when it starts, not only when it finishes.
• rootmark.mark_waiting(span, reason) distinguishes a step paused on a human from one wedged against a model that stopped answering.
• heartbeat= on rootmark.span(), so a long step keeps saying it is alive.
• Steps: step_key and step_kind name which logical step a span is, so the collector can aggregate the same step across runs.
• Bedrock guardrail traces no longer carry modelOutput, and the size cap rose from 4 KB to 16 KB. A long answer used to overrun the cap and take the policy assessment with it. Read the model's response from the span's outputs instead.

See CHANGELOG.md for the full list.

Quickstart

import rootmark

rootmark.init(
    api_key="pk_...",                 # project API key
    project_id="...",                 # UUID
    api_key_id="...",                 # UUID
    api_url="https://rootmark.example.com",
    workflow_name="my-service",       # not "default" -- see below
)

rootmark.patch()                      # instrument supported frameworks in place

That is the whole integration for the common case. patch() installs instrumentation process-wide, so call it once at startup and before the first agent invocation.

Grouping and labelling runs

with rootmark.workflow_override("contract-review"):
    with rootmark.bind_context(tenant_id="acme", session_id=session, tags=["surface:chat"]):
        result = agent(prompt)

bind_context applies to everything that runs inside it, including tasks the agent loop spawns, because the binding follows the async context.

workflow_name should not be "default": that value lets framework instrumentation override the name you bind per run with its own agent id.

Naming the steps of a run

with rootmark.span("fetch_clauses", run_type=RunType.TOOL,
                   step_key="fetch_clauses", step_kind="retrieval"):
    ...

step_key is a stable identity for which logical step this is, constant across runs; step_kind is what it was for (retrieval, generation, routing, review, tool, memory). Together they let the collector answer "which step is slow" without anyone reading individual traces.

Both are optional, and a kind the SDK does not recognise is kept rather than dropped — your vocabulary is allowed to be ahead of ours. Derive step_key from something stable: a key built from a timestamp or a request id fragments its own aggregate, and nothing can detect that for you.

Long-running steps

with rootmark.span("await_approval", run_type=RunType.TOOL, heartbeat=30) as span:
    rootmark.mark_waiting(span, "human_review")
    decision = wait_for_approval()      # minutes, or hours

mark_waiting says the step is deliberately not working, and what it is waiting on. The reason is free text in your own vocabulary — human_review, rate_limit, upstream — and is shown back verbatim. It is the difference between a run somebody has to investigate and one they can leave alone.

heartbeat re-announces the span every N seconds while it runs. Leave it unset for anything that finishes quickly: a span is already visible from the moment it starts, and a heartbeat for a 200 ms call is bandwidth spent to say nothing. Set it on the long ones, where the useful question is not "is it running" but "is anyone still on the other end" — without it, a step whose process was killed looks exactly like one that is simply taking its time. Intervals below five seconds are raised to five rather than refused.

This roughly doubles the rows a run produces, since each span is now sent at least twice. The collector collapses the pair when it merges, so stored size settles back; what doubles is what crosses the network.

Framework support

The base install instruments nothing on its own beyond direct use. Install the extra for what you actually run:

pip install "rootmark[openai]"    # OpenAI 1.x and 2.x
pip install "rootmark[langchain]" # LangChain callback handler
pip install "rootmark[crewai]"    # CrewAI event listeners
pip install "rootmark[strands]"   # Strands agents / AWS Bedrock
pip install "rootmark[boto3]"     # Bedrock via botocore, no code change

For boto3, a session built by hand rather than the default one needs rootmark.patch_session(session) — the botocore hooks only reach the default session otherwise.

Delivery

Spans are batched on a background thread and flushed on a timer, so nothing blocks a request. A failed export is written to a spill file and retried, keeping its batch identity so the collector can recognise a replay rather than double-count it. Ordering is preserved: a batch that cannot be delivered holds back the ones behind it.

Point spill_file somewhere writable. The default is relative to the working directory, which in a container is frequently owned by root while the process runs as someone else — and a spill file that cannot be written is buffering you silently do not have.

Call rootmark.flush() before a process exits if you want outstanding spans delivered, and rootmark.shutdown() to stop the processor.

Keeping payloads out

rootmark.init(..., suppress_payloads=True)

Spans are still recorded — timings, token counts, the shape of the run — but prompt and completion text is not sent. Use this when the collector sits outside the trust boundary that the data does not leave.

Links

• Documentation: https://rootmark.dev/docs/
• Source: https://github.com/IvanAdactic/rootmark