testzeus-cli 0.0.19

Command-line interface for TestZeus testing platform

TestZeus CLI

A powerful command-line interface for the TestZeus testing platform.

Installation

pip install testzeus-cli

Authentication

Before using the CLI, you need to authenticate with the TestZeus platform:

testzeus login

You will be prompted to enter your email and password. Your credentials are securely stored in your system's keyring.

testzeus login --profile dev

Authentication automatically detects and stores your tenant information, which will be used for all subsequent commands.

Auth Commands

Command	Description
login	Authenticate with TestZeus
logout	Log out and remove stored credentials
whoami	Display current authentication status

Global Options

The following options can be used with any command:

Option	Description
--profile	Configuration profile to use (default: "default")
--api-url	Custom TestZeus API URL
--verbose	Enable verbose output
--format	Output format: json, table, or yaml (default: table)

Managing Tests

List Tests

testzeus tests list

Filter test list with key-value pairs:

testzeus tests list --filters status=draft

Sort and expand related entities:

testzeus tests list --sort created --expand tags,test_data

Get Test Details

testzeus tests get <test-id>
testzeus tests get <test-id> --expand tags,test_data

Create Test

Create a test with text-based features:

testzeus tests create --name "My Test" --feature "Feature: Test something"

Create a test with features from a file:

testzeus tests create --name "My Test" --feature-file ./features.txt

Additional options:

testzeus tests create --name "My Test" --feature-file ./features.txt --status ready --data data_id1 --data data_id2 --tags tag1 --tags tag2 --environment env_id --execution-mode strict

Set default runtime inputs and outputs used by suite orchestration:

testzeus tests create \
  --name "Login Test" \
  --feature-file ./login.feature \
  --test-params '{"base_url":"https://staging.example.com","tenant_slug":"acme"}' \
  --output-schema '{"browser_auth_state":{"type":"file"}}'

Read a test's merged input metadata and list suites that reference it:

testzeus tests input-params <test-id>
testzeus tests dependent-suites <test-id>

Update Test

Update test name:

testzeus tests update <test-id> --name "New Name"

Update test features from text:

testzeus tests update <test-id> --feature "Updated feature content"

Update test features from a file:

testzeus tests update <test-id> --feature-file ./updated_features.txt

Update other properties:

testzeus tests update <test-id> --status ready --data data_id1 --tags tag1 --environment env_id

Update default runtime inputs or outputs:

testzeus tests update <test-id> \
  --test-params-file ./test-params.json \
  --output-schema-file ./output-schema.json

Delete Test

testzeus tests delete <test-id>

Test Runs

List Test Runs

testzeus test-runs list

Filter runs by status:

testzeus test-runs list --filters status=running

Get Test Run Details

testzeus test-runs get <run-id>

Get expanded details including all outputs and steps:

testzeus test-runs get-expanded <run-id>

Watch Test Run Progress

testzeus test-runs watch <run-id>
testzeus test-runs watch <run-id> --interval 10

Get Test Run Status

testzeus test-runs status <run-id>

Download Test Run Attachments

testzeus test-runs download-attachments <run-id>
testzeus test-runs download-attachments <run-id> --output-dir ./my-attachments

Test Suites

List and Get Test Suites

testzeus test-suites list
testzeus test-suites get <suite-id>
testzeus test-suites get <suite-id> --expand tests,environment

Create and Update Test Suites

Create a suite from a workflow definition:

testzeus test-suites create \
  --name "checkout-suite" \
  --workflow-definition '{"version":"1.0","nodes":[{"id":"login","name":"Login","test_id":"test_login"},{"id":"checkout","name":"Checkout","test_id":"test_checkout"}],"edges":[{"id":"login-to-checkout","from_node":"login","to_node":"checkout","dependency_type":"temporal"}],"outputs":[]}' \
  --default-inputs '{"base_url":"https://staging.example.com"}' \
  --input-schema '[{"name":"base_url","type":"string","required":true}]' \
  --execution-mode lenient

Update a suite:

testzeus test-suites update <suite-id> --status ready --notification-channels channel_1,channel_2

Delete a suite:

testzeus test-suites delete <suite-id>

Test Suite Runs

Start a suite run:

testzeus test-suite-runs create \
  --name "checkout-suite-run" \
  --test-suite <suite-id> \
  --input-values '{"base_url":"https://staging.example.com"}'

Inspect and control a suite run:

testzeus test-suite-runs list --filters status=running
testzeus test-suite-runs get <suite-run-id>
testzeus test-suite-runs pause <suite-run-id> --mode graceful --reason "maintenance window"
testzeus test-suite-runs resume <suite-run-id>
testzeus test-suite-runs cancel <suite-run-id>
testzeus test-suite-runs delete <suite-run-id>

Test Suite Node Runs

testzeus test-suite-node-runs list --filters test_suite_run=<suite-run-id>
testzeus test-suite-node-runs get <node-run-id>

Test Suite Schedules

testzeus test-suite-schedules list
testzeus test-suite-schedules create \
  --name "daily-checkout" \
  --test-suite <suite-id> \
  --cron-expression "0 9 * * *" \
  --input-values '{"base_url":"https://staging.example.com"}'
testzeus test-suite-schedules update <schedule-id> --inactive
testzeus test-suite-schedules delete <schedule-id>

Test Data

List Test Data

testzeus test-data list

Filter by type:

testzeus test-data list --filters type=test

Get Test Data Details

testzeus test-data get <data-id>
testzeus test-data get <data-id> --expand related_entities

Create Test Data

Create with inline content:

testzeus test-data create --name "Test Data 1" --data "{\"key\":\"value\"}"

Create with data from a file:

testzeus test-data create --name "Test Data" --data-file ./data.json

Additional options:

testzeus test-data create --name "Test Data" --type test --status ready --data-file ./data.json

Update Test Data

Update name and other properties:

testzeus test-data update <data-id> --name "New Data Name" --type updated --status ready

Update data content from text:

testzeus test-data update <data-id> --data "{\"key\":\"updated\"}"

Update data content from a file:

testzeus test-data update <data-id> --data-file ./updated_data.json

Delete Test Data

testzeus test-data delete <data-id>

File Management for Test Data

Upload a file to test data:

testzeus test-data upload-file <data-id> <file-path>

Delete all files from test data:

testzeus test-data delete-all-files <data-id>

Environments

List Environments

testzeus environments list

Filter environments:

testzeus environments list --filters status=ready

Get Environment Details

testzeus environments get <env-id>
testzeus environments get <env-id> --expand related_entities

Create Environment

Create with inline data:

testzeus environments create --name "Test Environment" --data "{\"key\":\"value\"}"

Create with data from a file:

testzeus environments create --name "Test Environment" --data-file ./env_data.json

Additional options:

testzeus environments create --name "Test Environment" --status ready --data-file ./env_data.json --tags "tag1,tag2"

Update Environment

Update environment properties:

testzeus environments update <env-id> --name "New Name" --status ready

Update environment data:

testzeus environments update <env-id> --data "{\"key\":\"updated\"}"
testzeus environments update <env-id> --data-file ./updated_env_data.json

Delete Environment

testzeus environments delete <env-id>

File Management for Environments

Upload a file to environment:

testzeus environments upload-file <env-id> <file-path>

Remove a file from environment:

testzeus environments remove-file <env-id> <file-path>

Delete all files from environment:

testzeus environments delete-all-files <env-id>

Tags

List Tags

testzeus tags list

Filter tags:

testzeus tags list --filters name=test

Get Tag Details

testzeus tags get <tag-id>

Create Tag

testzeus tags create --name "test-tag" --value "test-value"

Create tag without value:

testzeus tags create --name "simple-tag"

Update Tag

testzeus tags update <tag-id> --name "new-name" --value "new-value"

Delete Tag

testzeus tags delete <tag-name>

Test Report Schedules

Test report schedules allow you to create automated test execution schedules with various filtering options and notification configurations.

List Test Report Schedules

testzeus schedule list

Filter and paginate:

testzeus schedule list --filters is_active=true --page 1 --per-page 20

Get Test Report Schedule Details

testzeus schedule get <schedule-id>

Create Test Report Schedule

Create a basic schedule with cron expression:

testzeus schedule create --name "nightly-reports" --cron-expression "0 0 * * *"

Create a schedule with time intervals (alternative to cron):

testzeus schedule create \
  --name "daily-reports" \
  --filter-time-intervals "2025-01-01 00:00:00,2025-01-01 01:00:00" \
  --is-active true

Create a comprehensive schedule with comma-separated filters:

testzeus schedule create \
  --name "regression-reports" \
  --is-active true \
  --cron-expression "0 2 * * 1-5" \
  --filter-name-pattern "regression_*" \
  --filter-tags "tag1,tag2,tag3" \
  --filter-env "env1,env2" \
  --filter-test-data "data1,data2" \
  --notification-channels "channel1,channel2"

Important validation rules:

• Use either --cron-expression OR --filter-time-intervals (not both)
• Use either --filter-tags OR --filter-tag-pattern (not both)
• Use either --filter-env OR --filter-env-pattern (not both)
• Use either --filter-test-data OR --filter-test-data-pattern (not both)

Update Test Report Schedule

Update schedule with new cron expression:

testzeus schedule update <schedule-id> \
  --name "updated-schedule" \
  --cron-expression "0 3 * * *" \
  --is-active true

Update with comma-separated filters:

testzeus schedule update <schedule-id> \
  --filter-tags "newtag1,newtag2" \
  --notification-channels "newchannel1,newchannel2"

Update with time intervals:

testzeus schedule update <schedule-id> \
  --filter-time-intervals "2025-01-02 10:00:00,2025-01-02 11:00:00"

Delete Test Report Schedule

testzeus schedule delete <schedule-id>

Notification Channels

Notification channels manage how test results and reports are delivered to your team.

List Notification Channels

testzeus notification list

Get Notification Channel Details

testzeus notification get <channel-id>

Create Notification Channel

Create a basic email notification channel:

testzeus notification create \
  --name "qa-team" \
  --emails "[e-mail],[e-mail]" \
  --is-active true

Create a notification channel with webhooks:

testzeus notification create \
  --name "prod-alerts" \
  --emails "[e-mail],[e-mail]" \
  --webhooks "https://hooks.example.com/webhook1,https://hooks.example.com/webhook2" \
  --is-active true \
  --is-default true

Update Notification Channel

Update channel properties:

testzeus notification update <channel-id> --name "updated-channel" --is-active true

Update emails and webhooks:

testzeus notification update <channel-id> \
  --emails "[e-mail],[e-mail]" \
  --webhooks "https://new.webhook.com,https://backup.webhook.com"

Delete Notification Channel

testzeus notification delete <channel-id>

Extensions

Extensions allow you to extend TestZeus functionality with custom data and responses.

List Extensions

testzeus extension list

Get Extension Details

testzeus extension get <extension-id>

Create Extension

Create with inline data:

testzeus extension create \
  --name "custom-validator" \
  --data-content "validation logic here"

Create with data from file:

testzeus extension create \
  --name "custom-script" \
  --data-file ./script.py

Delete Extension

testzeus extension delete <extension-id>

AI Test Generator

AI Test Generator helps create test cases automatically using AI-powered analysis.

List AI Test Generators

testzeus testcase-generator list

Get AI Test Generator Details

testzeus testcase-generator get <generator-id>

Create AI Test Generator

Create with minimal required parameters:

testzeus testcase-generator create \
  --test-id <test-id> \
  --user-prompt "Generate comprehensive test cases for login functionality"

Create with custom parameters:

testzeus testcase-generator create \
  --test-id <test-id> \
  --user-prompt "Generate test cases for payment processing" \
  --reasoning-effort high \
  --num-testcases 10 \
  --test-data data1 --test-data data2 \
  --environment env_id

Create with prompts from files:

testzeus testcase-generator create \
  --test-id <test-id> \
  --test-feature-file ./feature.txt \
  --user-prompt-file ./prompt.txt \
  --reasoning-effort medium \
  --num-testcases 5

Required Parameters:

• --test-id: Test ID to generate for (required)
• --user-prompt: User prompt for AI generation (required, or use --user-prompt-file)

Default Values:

• --reasoning-effort: low (options: low, medium, high)
• --num-testcases: 3 (range: 1-20)
• submit: automatically set to true

Delete AI Test Generator

testzeus testcase-generator delete <generator-id>

Test Run Groups

Test run groups allow you to organize and execute multiple tests together as a cohesive unit. This is useful for running test suites, regression tests, or any collection of tests that should be executed together.

List Test Run Groups

testzeus test-run-group list

Filter and paginate:

testzeus test-run-group list --filters status=completed --page 1 --per-page 20

Sort and expand related entities:

testzeus test-run-group list --sort created --expand tags,test_ids,environment

Get Test Run Group Details

testzeus test-run-group get <group-id>
testzeus test-run-group get <group-id> --expand tags,test_ids,environment

Execute Test Run Group

Execute a test run group with specific test IDs:

testzeus test-run-group execute --name "regression-suite" --test-ids "test1,test2,test3"

Execute a test run group with tags:

testzeus test-run-group execute --name "smoke-suite" --tags "smoke,critical"

Execute a comprehensive test run group:

testzeus test-run-group execute \
  --name "smoke-tests" \
  --execution-mode strict \
  --test-ids "test_id_1,test_id_2,test_id_3" \
  --environment env_id \
  --notification-channels "channel_id_1,channel_id_2"

Required Parameters:

• --name: Test run group name (required)
• Either --test-ids OR --tags (required, but not both)

Validation Rules:

• Use either --test-ids OR --tags (not both)
• At least one test ID or tag must be provided

Default Values:

• --execution-mode: lenient (options: lenient, strict)

Execute and Monitor Test Run Group Status

Execute a test run group and monitor until completion. This is useful for CI/CD pipelines where you want to wait for tests to complete before proceeding. This command does NOT wait for CTRF report generation or download reports.

Execute and monitor with test IDs:

testzeus test-run-group execute-and-monitor-status \
  --name "ci-tests" \
  --test-ids "test1,test2,test3"

Execute and monitor with tags:

testzeus test-run-group execute-and-monitor-status \
  --name "smoke-tests" \
  --tags "smoke,critical"

Execute with custom polling interval:

testzeus test-run-group execute-and-monitor-status \
  --name "regression-tests" \
  --test-ids "test1,test2" \
  --interval 60 \
  --environment env_id \
  --notification-channels "channel1,channel2"

Required Parameters:

• --name: Test run group name (required)
• Either --test-ids OR --tags (required, but not both)

Options:

• --environment: Environment ID (optional)
• --notification-channels: Comma-separated notification channel IDs (optional)
• --interval: Polling interval in seconds (default: 30)

Notes:

• Monitors only the group status (not CTRF report status)
• Does NOT download reports - use download-report command separately if needed
• Returns success when status is completed, failure when cancelled, failed, or error

Delete Test Run Group

testzeus test-run-group delete <group-id>

Cancel Test Run Group

Cancel a running test run group:

testzeus test-run-group cancel <group-id>

Get Test Run Group Status

Get detailed status information:

testzeus test-run-group get-status <group-id>

This returns status information including:

• Overall group status
• CTRF report status
• Individual test run statuses
• Timestamps

Download Test Run Group Report

Download report in PDF format (default):

testzeus test-run-group download-report <group-id>

Download report in specific format:

testzeus test-run-group download-report <group-id> --format ctrf
testzeus test-run-group download-report <group-id> --format csv --output-dir ./reports

Available Formats:

• pdf (default) - PDF report
• ctrf - CTRF JSON format
• csv - CSV data export
• zip - ZIP archive with all formats

Options:

• --output-dir: Output directory (default: downloads)
• --format: Report format (default: pdf)

Download Test Run Group Attachments

Download all attachments for all test runs in a test run group:

testzeus test-run-group download-attachments <group-id>

Download attachments to custom directory:

testzeus test-run-group download-attachments <group-id> --output-dir ./test-artifacts

This command creates a hierarchical directory structure:

downloads/
└── <test-run-group-name>/
    ├── <test-run-1-name>/
    │   ├── attachment1.pdf
    │   ├── attachment2.log
    │   └── ...
    ├── <test-run-2-name>/
    │   ├── attachment3.png
    │   └── ...
    └── ...

Features:

• Downloads attachments from all test runs in the group
• Automatically organizes files by test run
• Shows progress and summary statistics
• Continues processing if individual downloads fail
• Supports verbose output for detailed breakdown

Options:

• --output-dir: Base directory to save attachments (default: downloads)

Test Authoring

Create tests via interactive wizard or quick mode.

Wizard Mode

Interactive multi-step wizard for guided test creation:

testzeus tests-authoring wizard

Quick Mode

Rapid test creation with minimal prompts:

testzeus tests-authoring quick --name my_test --description "Test checkout"

Manual Mode

Create test directly with Gherkin feature:

testzeus tests-authoring manual --name my_test --feature "Feature: Login..."

See testzeus_cli/commands/tests_authoring.py for implementation.

Configuration

The CLI stores configuration and credentials in your user's config directory. Different profiles can be used to manage multiple TestZeus environments.

Default configuration location:

• Linux/Mac: ~/.testzeus/config.yaml
• Windows: %APPDATA%\testzeus\config.yaml

Passwords are securely stored in your system's keyring.

Examples

Complete Workflow

# Login to TestZeus
testzeus login

# Create test data
testzeus test-data create --name "User Data" --data "{\"username\":\"testuser\"}" 

# Create a new test with features from a file
testzeus tests create --name "Login Test" --feature-file ./features/login.feature --data <test_data_id>

# Execute tests using test run groups
testzeus test-run-group execute --name "Login Tests" --test-ids <test_id>

# Watch the test run group progress
testzeus test-run-group get-status <test_run_group_id>

# Download test report
testzeus test-run-group download-report <test_run_group_id> --format pdf

# Download all attachments from the test runs
testzeus test-run-group download-attachments <test_run_group_id> --output-dir ./results

Working with Environments

# Create an environment with data
testzeus environments create --name "Production Environment" --data-file ./prod_config.json --status ready

# Upload additional files to the environment
testzeus environments upload-file <env-id> ./additional_config.yaml

# Create a test that uses the environment
testzeus tests create --name "Production Test" --feature-file ./test.feature --environment <env-id>

Managing Tags

# Create tags for organizing tests
testzeus tags create --name "regression" --value "suite"
testzeus tags create --name "priority" --value "high"

# Create a test with tags
testzeus tests create --name "Critical Test" --feature-file ./critical.feature --tags tag1 --tags tag2

Working with Test Run Groups

# Execute a test run group with specific test IDs
testzeus test-run-group execute \
  --name "nightly-regression" \
  --execution-mode strict \
  --test-ids "test_id_1,test_id_2,test_id_3" \
  --environment env_id

# Check the status
testzeus test-run-group get-status <group-id>

# Download the test report
testzeus test-run-group download-report <group-id> --format pdf

# Download all attachments from test runs
testzeus test-run-group download-attachments <group-id> --output-dir ./artifacts

# Cancel a running group if needed
testzeus test-run-group cancel <group-id>

# Delete the group when done
testzeus test-run-group delete <group-id>

CI/CD Integration with execute-and-monitor-status

For CI/CD pipelines, use execute-and-monitor-status to run tests and wait for completion:

# Run tests and wait for completion (useful in CI/CD)
testzeus test-run-group execute-and-monitor-status \
  --name "ci-pipeline-$(date +%Y%m%d-%H%M%S)" \
  --tags "smoke,regression" \
  --interval 30

# The command exits with success (0) when tests complete
# and failure (1) when tests fail, are cancelled, or error out

Example GitHub Actions workflow:

- name: Run TestZeus Tests
  run: |
    testzeus test-run-group execute-and-monitor-status \
      --name "pr-tests-${{ github.run_id }}" \
      --tags "smoke" \
      --interval 30

Error Handling

When an error occurs, the CLI will display an error message. For more detailed information, run any command with the --verbose flag:

testzeus tests list --verbose

Output Formats

The CLI supports multiple output formats:

• table: Human-readable tabular format (default)
• json: JSON format for programmatic usage
• yaml: YAML format

Example:

testzeus tests list --format json

Development and Contribution

To contribute to the TestZeus CLI, fork the repository and install development dependencies:

pip install -e ".[dev]"

Release Process

The TestZeus CLI uses GitHub Actions for automated releases to PyPI. To create a release:

1. Use the Makefile's release target: make release
2. This will:
   • Prompt for version bump type (patch, minor, major)
   • Update the version in pyproject.toml
   • Commit and create a git tag
   • Push changes and tags to GitHub
3. The tag push will automatically trigger the GitHub Actions publish workflow