agently-devtools 0.0.1a0

Observation and developer tooling companion package for Agently

agently-devtools

agently-devtools is the local observation, evaluation, and playground companion package for agently.

Install

pip install -U agently agently-devtools

Python >=3.10

Start

agently-devtools start

Default local address:

• Console: http://127.0.0.1:15596/
• Ingest: http://127.0.0.1:15596/observation/ingest

Observation Bridge

import os

from agently import Agently
from agently_devtools import ObservationBridge

bridge = ObservationBridge(
    os.environ["AGENTLY_DEVTOOLS_INGEST_URL"],
    app_id="your_app_id",
    group_id="your_group_id",
)
bridge.register(Agently.event_center)

TriggerFlow executions automatically publish their flow definition, so the execution graph can show the full static structure, including branches that have not been hit yet.

Scenario Evaluations

import os

from agently_devtools import EvaluationBridge, EvaluationCase, EvaluationRunner

bridge = EvaluationBridge(
    base_url=os.environ["AGENTLY_DEVTOOLS_BASE_URL"],
    app_id="your_app_id",
    group_id="your_group_id",
)

binding = bridge.bind_agent(
    agent,
    suite_id="your_suite_id",
    target_name="your_target_name",
)

report = EvaluationRunner(bridge=bridge).run(
    binding,
    cases=[EvaluationCase(case_id="case-1", input="your scenario")],
    rounds=3,
)