health-check-runner 1.0.0

Universal JSON-driven CLI health check library — Robot Framework, pytest, Python, CLI

health-check-runner

Universal JSON-driven CLI health check library. Write your test cases once in a JSON template and run them via Robot Framework, pytest, plain Python, or the CLI — no code changes required.

---

Table of Contents

1. Install
2. Quick Start
3. Template Format
4. Validation Types
5. Variable Resolution
6. Running via CLI
7. Running via pytest
8. Running via Robot Framework
9. Running via Plain Python
10. Session Configuration
11. Environment Files
12. Output Formats
13. Credential Masking
14. Reconnect on Failure

---

Install

# Core (SSH only)
pip install health-check-runner

# With Robot Framework support
pip install "health-check-runner[robot]"

# With pytest support
pip install "health-check-runner[pytest]"

# All optional dependencies
pip install "health-check-runner[all]"

# Development install (editable)
pip install -e .

---

Quick Start

from health_check_runner import Runner

with Runner(
    template="templates/mydevice.json",
    sessions="config/sessions.json",
    env="config/envs/site.json",
) as r:
    result = r.run()
    print(result.summary())
    print(result.to_table())

---

Template Format

{
  "suite_key": "MYDEVICE",
  "shared_rules_key": "SHARED",
  "test_cases": [
    {
      "id": "TC1",
      "name": "Check software version",
      "steps": [
        {
          "action": "execute",
          "session": "mydevice",
          "command": "show version",
          "store_as": "version_output"
        },
        {
          "action": "validate",
          "source": "version_output",
          "type": "assert_contains",
          "expected": "${expected_sw_version}"
        }
      ]
    }
  ]
}

Step Fields

Field	Description
action	execute or validate
session	Session ID from sessions.json
command	Shell command to run
store_as	Variable name to store output in
source	Variable name to validate against
type	Validation type (see below)
expected	Expected value / pattern
continue_on_failure	true = soft failure, keep running
timeout	Per-step SSH timeout override (seconds)

---

Validation Types

String Assertions

Type	Description
assert_contains	Output contains expected string
assert_not_contains	Output does NOT contain expected string
assert_equal	Output (stripped) equals expected
assert_contains_any	Output contains at least one of expected list
assert_line_count_equal	Line count equals expected integer
assert_line_count_gte	Line count >= expected integer
for_each_line	Every non-empty line contains expected
lines_containing	Lines matching filter all contain expected
all_lines_match_any	Every non-empty line matches one of expected list
section_lines_contain_any	Lines in section between start/end contain one of expected

Extraction

Type	Description
regex_extract	Extract named group via pattern, store in store_as
regex_line_match	A line matches pattern
regex_findall	Find all matches, store list in store_as
string_extract	Extract text after after marker, store in store_as
string_extract_between	Extract between start and end, store in store_as
extract_between	Alias for string_extract_between
extract_between_parens	Extract content inside first (…)
split_extract	Split on delimiter, take index field_index, store in store_as
fetch_between	Extract between before / after, store in store_as
split_field_assert	Split on delimiter, assert field at field_index contains expected
split_fields_numeric_gt	Split on delimiter, assert all numeric fields > threshold

Numeric

Type	Description
assert_numeric_lt	Extracted number < expected
assert_numeric_gt	Extracted number > expected
assert_greater_than	Alias for assert_numeric_gt
strip_percent_and_assert_lte	Strip %, assert value <= expected

Network-Specific

Type	Description
unlocked_must_be_enabled	Unlocked cells must also be enabled
verify_ping_ipv4	Parse ping output, assert 0% packet loss
verify_ping_ipv6	Parse IPv6 ping output, assert 0% packet loss
version_in_active_partition	Version string found in active partition listing

Date / Time

Type	Description
date_diff_assert_lte	Parse date from output, assert diff from now <= max_days
last_line_date_diff	Last non-empty line is a date; assert diff <= max_days

XML

Type	Description
xml_xpath	Parse XML body from output; evaluate xpath; assert result contains expected

Example:

{
  "action": "validate",
  "source": "curl_output",
  "type": "xml_xpath",
  "xpath": "//response/status",
  "expected": "SUCCESS"
}

Namespaces are stripped automatically so XPath can use simple element names.

JSON

Type	Description
json_extract	Parse JSON body from output; navigate dot-notation path; assert or store result

Example — assert:

{
  "action": "validate",
  "source": "curl_output",
  "type": "json_extract",
  "path": "data.items.0.status",
  "expected": "active"
}

Example — store:

{
  "action": "validate",
  "source": "curl_output",
  "type": "json_extract",
  "path": "data.version",
  "store_as": "api_version"
}

Path syntax:

• a.b.c — nested keys
• items.0 — array index
• items.*.status — wildcard, returns list of values

Informational

Type	Description
log_output	Log the source variable at INFO level; always passes
store_nf_version	Log NF version info; always passes

---

Variable Resolution

Variables in command, expected, and other string fields are resolved in this order:

1. Template vars block — flat key/value dict at the top of the template
2. Environment file — merged env/sessions data
3. OS environment variables — ${MY_VAR} → os.environ["MY_VAR"]
4. Stored step variables — set via store_as in earlier steps
5. Unresolved — token left as-is (no error)

Example:

{
  "vars": {
    "expected_sw_version": "R22B"
  }
}

---

Running via CLI

healthcheck-run \
  --template  templates/mydevice.json \
  --sessions  config/sessions.json \
  --env       config/envs/site.json \
  [--shared   config/shared_rules.json] \
  [--tc TC1 --tc TC2] \
  [--output   table|json|junit] \
  [--out-file results.xml] \
  [--timeout  60] \
  [--loglevel DEBUG]

Exit codes: 0 = all passed, 1 = failures, 2 = config/connection error.

---

Running via pytest

# conftest.py
from health_check_runner.integrations.pytest_plugin import make_suite_fixture

mydevice = make_suite_fixture(
    template="templates/mydevice.json",
    sessions="config/sessions.json",
    env="config/envs/site.json",
)

# test_bbu.py
import pytest

@pytest.mark.parametrize("tc_id", ["TC1", "TC2", "TC3"])
def test_mydevice(mydevice, tc_id):
    result = bbu.run_tc(tc_id)
    assert result.passed, result.failure_summary()

Auto-parametrize from the template file:

from health_check_runner.integrations.pytest_plugin import parametrize_tcs

@parametrize_tcs("templates/mydevice.json")
def test_mydevice(mydevice, tc_id):
    result = bbu.run_tc(tc_id)
    assert result.passed, result.failure_summary()

---

Running via Robot Framework

# conftest_keywords.py  (or directly in .robot)
from health_check_runner.integrations.robot_keywords import RobotKeywords

*** Settings ***
Library    health_check_runner.integrations.robot_keywords.RobotKeywords
...        template=templates/mydevice.json
...        sessions=config/sessions.json
...        env=config/envs/site.json

Suite Setup     Open Sessions
Suite Teardown  Close Sessions

*** Test Cases ***
TC1 Software Version
    Run Json Test Case    TC1

TC2 Cell Status
    Run Json Test Case    TC2

---

Running via Plain Python

from health_check_runner import Runner

runner = Runner(
    template="templates/mydevice.json",
    sessions="config/sessions.json",
    env="config/envs/site.json",
    shared_rules="config/shared_rules.json",  # optional
    timeout=60,
)

runner.open()

try:
    # Run all TCs
    suite = runner.run()
    print(suite.summary())

    # Or run a single TC
    tc = runner.run_tc("TC1")
    if not tc.passed:
        print(tc.failure_summary())
finally:
    runner.close()

Context manager form:

with Runner(template=..., sessions=..., env=...) as r:
    suite = r.run()
    print(suite.to_table())

---

Session Configuration

config/sessions.json:

{
  "sessions": [
    {
      "id": "jump",
      "type": "direct",
      "host": "${JUMP_HOST}",
      "username": "admin",
      "password": "${JUMP_PASS}",
      "port": 22
    },
    {
      "id": "mydevice",
      "type": "ssh_from",
      "via": "jump",
      "host": "192.168.1.10",
      "username": "admin",
      "password": "secret"
    },
    {
      "id": "mydevice_cli",
      "type": "amos_from",
      "via": "jump",
      "host": "192.168.1.10"
    },
    {
      "id": "root",
      "type": "su_from",
      "via": "bbu",
      "password": "rootpass"
    }
  ]
}

Session Types

Type	Description
direct	SSH directly to host
ssh_from	SSH to host from an existing session (via)
amos_from	Start AMOS on host from an existing session (via)
command_from	Run a custom command from via to open a new shell
su_from	Run su - on via session

---

Environment Files

config/envs/site.json contains site-specific values that override template defaults:

{
  "expected_sw_version": "R22B",
  "bbu_ip": "192.168.1.10"
}

Variables are merged with template vars (env file wins on conflict).

---

Output Formats

Table (default)

Suite: MYDEVICE  |  Passed: 18  |  Failed: 2  |  Total: 20

TC_ID     NAME                          STATUS   DURATION
--------- ----------------------------- -------- --------
TC1       Software version check        PASS     2.3s
TC2       Cell status check             FAIL     1.1s
  FAIL: expected 'active' but got 'inactive'

JSON

suite.to_json()  # pretty-printed JSON string

JUnit XML

suite.to_junit_xml()  # JUnit-compatible XML for CI systems

Write to file via CLI:

healthcheck-run --template ... --output junit --out-file results.xml

---

Credential Masking

Passwords loaded from the env/sessions files are automatically registered as secrets and replaced with *** in all log output.

To register additional secrets programmatically:

runner = Runner(...)
runner.logger.register_secret("my-api-token")

---

Reconnect on Failure

If an SSH connection drops mid-suite (idle timeout, network blip), the SessionManager automatically:

1. Detects the dead channel (paramiko.SSHException, EOFError, socket.error, OSError)
2. Tears down the session and any sub-contexts (e.g., AMOS, su) that depended on it
3. Re-establishes the full connection chain
4. Replays the failed command once

If the replay also fails, the error is raised as a normal step failure — the TC is marked failed and the suite continues.

No configuration needed; reconnect logic is always active.