NeMo Relay

nemo-relay is the NeMo Relay package for Python applications. It gives Python code access to a portable agent runtime for execution scopes, middleware, plugins, lifecycle events, adaptive behavior, and observability around tool and LLM calls.

The package wraps the shared Rust runtime, so Python applications use the same runtime semantics as the Rust and Node.js surfaces.

Why Use It?

Use the Python binding for the following tasks:

• Own execution context in Python: Group agent, tool, and LLM work into one scope tree from Python application code.
• Package policy around callbacks: Use guardrails and intercepts to block work, sanitize observability payloads, rewrite requests, or wrap execution.
• Emit one lifecycle stream: Send runtime events to in-process subscribers, Agent Trajectory Interchange Format (ATIF), or typed OpenTelemetry workflows.
• Integrate without a framework migration: Wrap framework or provider callbacks while preserving the application’s orchestration model.

What You Get

The Python package provides the following capabilities:

• Scope, tool, and LLM helpers: Managed boundaries that emit lifecycle events and run middleware in a consistent order.
• Middleware APIs: Guardrails and intercepts for tool and LLM requests, responses, and execution, plus mark and scope event sanitizers for data, category_profile, and metadata.
• Subscribers and exporters: OpenTelemetrySubscriber exports traces; OpenTelemetryLogSubscriber and OpenTelemetryMetricSubscriber export severity-tagged marks and typed metric measurements. Bare OTLP/HTTP origins resolve to /v1/traces, /v1/logs, or /v1/metrics for the selected signal. Each direct OTLP subscriber exposes runtime_diagnostics() for bounded exporter and event-processing failure summaries. Trace export failures use otel.traces_export_failed; after a trace export fails, force_flush() continues to return an error until a later trace batch exports successfully. The nemo_relay.observability helpers configure plugin-owned endpoint fan-out.
• Plugin and typed helpers: Public modules for plugins, codecs, typed wrappers, adaptive runtime behavior, and observability plugin configuration.
• Shared Rust runtime semantics: Python behavior aligned with the Rust and Node.js surfaces.

Installation

Install the published package with uv:

uv add nemo-relay

If you are not using uv, install it with pip:

pip install nemo-relay

Optional Dependencies

LangChain Integration

LangChain integration is available with the langchain extra:

# With uv
uv add "nemo-relay[langchain]"

# With pip
pip install "nemo-relay[langchain]"

LangGraph Integration

LangGraph integration is available with the langgraph extra, this builds upon and includes the langchain extra as well.

# With uv
uv add "nemo-relay[langgraph]"

# With pip
pip install "nemo-relay[langgraph]"

Deep Agents Integration

Deep Agents integration is available with the deepagents extra. This extra builds upon and includes the langgraph and langchain extras.

# With uv
uv add "nemo-relay[deepagents]"

# With pip
pip install "nemo-relay[deepagents]"

LangChain NVIDIA Integration

The LangChain NVIDIA extra builds upon the langchain extra adding a compatible version of the langchain-nvidia-ai-endpoints package.

# With uv
uv add "nemo-relay[langchain-nvidia]"

# With pip
pip install "nemo-relay[langchain-nvidia]"

To install this along with the langgraph extra, use:

# With uv
uv add "nemo-relay[langgraph,langchain-nvidia]"
# With pip
pip install "nemo-relay[langgraph,langchain-nvidia]"

Getting Started

Register a subscriber, create a scope, and emit a mark event:

import nemo_relay


def on_event(event) -> None:
    print(f"{event.kind} {event.name}")


nemo_relay.subscribers.register("printer", on_event)

with nemo_relay.scope.scope("demo-agent", nemo_relay.ScopeType.Agent) as handle:
    nemo_relay.scope.event("initialized", handle=handle, data={"binding": "python"})

nemo_relay.subscribers.flush()
nemo_relay.subscribers.deregister("printer")

Use nemo_relay.scope.event(..., severity=nemo_relay.LogSeverity.Info) for a mark intended for log export. Use nemo_relay.scope.metric() with nemo_relay.MetricMeasurement objects for metrics; Relay validates the complete measurement group before publishing it.

OTLP Logs and Metrics

For plugin-managed export, configure version 4 and enable logs and metrics. Omitting their endpoint lists derives /v1/logs and /v1/metrics from the trace endpoint:

from nemo_relay.observability import (
    ComponentSpec,
    ObservabilityConfig,
    OpenTelemetryEndpointConfig,
    OpenTelemetryLogSectionConfig,
    OpenTelemetryMetricSectionConfig,
    OpenTelemetrySectionConfig,
)

component = ComponentSpec(
    ObservabilityConfig(
        opentelemetry=OpenTelemetrySectionConfig(
            enabled=True,
            endpoints=[
                OpenTelemetryEndpointConfig(
                    type="gen_ai", endpoint="http://localhost:4318/v1/traces"
                )
            ],
            logs=OpenTelemetryLogSectionConfig(enabled=True),
            metrics=OpenTelemetryMetricSectionConfig(enabled=True),
        )
    )
)

Emit a typed log mark and an atomically validated metric group with the public scope helpers:

from nemo_relay import DataSchema, LogSeverity, MetricKind, MetricMeasurement, MetricValueType
from nemo_relay import scope

scope.event(
    "cache-nearly-full",
    data={"entries": 900},
    data_schema=DataSchema("example.cache", "1"),
    severity=LogSeverity.Warn,
)
scope.metric(
    "cache-entries",
    [MetricMeasurement("example.cache.entries", MetricKind.Gauge, MetricValueType.U64, 900)],
)

Direct log and metric subscribers are independently managed. Register each before emitting marks, then deregister, force-flush, and shut it down during graceful teardown. Their runtime_diagnostics() snapshots contain bounded code, message, and count entries:

A successful direct log or trace force_flush() also updates this snapshot with any queue drops observed so far; the diagnostic count remains cumulative through later flushes and shutdown.

from nemo_relay import (
    OpenTelemetryLogConfig,
    OpenTelemetryLogSubscriber,
    OpenTelemetryMetricConfig,
    OpenTelemetryMetricSubscriber,
)

# Equivalent explicit OTLP/HTTP paths are /v1/logs and /v1/metrics, respectively.
logs = OpenTelemetryLogSubscriber(OpenTelemetryLogConfig("http://localhost:4318"))
metrics = OpenTelemetryMetricSubscriber(OpenTelemetryMetricConfig("http://localhost:4318"))
logs.register("otlp-logs")
metrics.register("otlp-metrics")
try:
    for diagnostic in logs.runtime_diagnostics().entries:
        print(diagnostic.code, diagnostic.message)
finally:
    logs.deregister("otlp-logs")
    logs.force_flush()
    logs.shutdown()
    metrics.deregister("otlp-metrics")
    metrics.force_flush()
    metrics.shutdown()

Native subscriber delivery is asynchronous, so call nemo_relay.subscribers.flush() before you read subscriber output or exit. From an asyncio task, use await nemo_relay.subscribers.flush_async() so async event sanitizers can continue running on that event loop.

For host integrations that need a serialized event shape, consume the canonical JSON payload from the subscriber event object:

import json
import nemo_relay


def on_event(event) -> None:
    payload = event.to_dict()
    print(payload["kind"], payload["name"])
    assert json.loads(event.to_json()) == payload


nemo_relay.subscribers.register("host-exporter", on_event)
try:
    with nemo_relay.scope.scope("demo-agent", nemo_relay.ScopeType.Agent):
        nemo_relay.scope.event("initialized", data={"binding": "python"})
finally:
    nemo_relay.subscribers.flush()
    nemo_relay.subscribers.deregister("host-exporter")

Package Surface

The public package modules are:

• nemo_relay.scope
• nemo_relay.tools
• nemo_relay.llm
• nemo_relay.guardrails
• nemo_relay.intercepts
• nemo_relay.subscribers
• nemo_relay.plugin
• nemo_relay.adaptive
• nemo_relay.observability
• nemo_relay.typed
• nemo_relay.codecs

Integrations

• nemo_relay.integrations.langchain
• nemo_relay.integrations.langgraph
• nemo_relay.integrations.deepagents

The compiled extension is exposed as nemo_relay._native.

Documentation

NeMo Relay Documentation: https://docs.nvidia.com/nemo/relay