Skip to main content

PyPlayKit - Enterprise Test Automation Framework

Tagline: A modular, scalable, and CI/CD-ready Python + Playwright framework for enterprise automation supporting multiple projects and 50+ QE engineers.


Overview

PyPlayKit is a multi-project enterprise test automation framework built with Python, Playwright, and Pytest. It supports multiple independent projects with UI, API, Database, and Integration testing capabilities.

Key Features:

  • 🏢 Multi-Project Support: Separate test organization for multiple applications
  • 👥 Scalable: Designed for 50-100 QE engineers working simultaneously
  • 🎯 Zero Merge Conflicts: Project-first organization eliminates conflicts
  • 📊 Data Comparison: File-to-File, File-to-DB, DB-to-File validation (Excel, CSV, JSON)
  • 📈 Interactive HTML Reports: Filterable validation reports with KPIs ⭐ NEW
  • 📚 Comprehensive Documentation: 8,700+ lines across 15+ guides
  • 🔧 Extensible: Plugin architecture with orchestration support
  • 🔍 Self-Healing: Locator recovery with fallback chains
  • 🔭 Observable: Comprehensive metrics and reporting
  • 📟 Device & Embedded Automation: Test IoT / embedded / PLC / CAN / BLE / Android devices against digital twins or real hardware ⭐ NEW
  • ☁️ Remote & Cloud Browsers: Run the UI suite on BrowserStack, Sauce Labs (Chrome and Edge) or your own Playwright server with one option, --pyplaykit-remote ⭐ NEW
  • 🤖 AI-ready: MCP server with 24 tools, each also a pyplaykit-mcp CLI subcommand

Quick Start

Prerequisites

  • Python 3.11 or higher
  • pip

Setup (5 minutes)

# Windows PowerShell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
playwright install
pip install -r requirements-dev.txt

Set Environment Variables

# Required for tests
$env:PYPLAYKIT_TEST_USERNAME="standard_user"
$env:PYPLAYKIT_TEST_PASSWORD="secret_sauce"

Run Tests

# Run all smoke tests
pytest -m smoke

# Run specific project tests
pytest tests/projects/project_1/
pytest tests/projects/project_2/
pytest tests/projects/project_3/

# Run by marker
pytest -m project_1
pytest -m project_2
pytest -m project_3

Architecture

Multi-Project Structure

PyPlayKit supports multiple independent projects with separate test suites:

tests/
├── projects/                              # Multi-project organization
│   ├── project_1/                      # Project 1: Quote management
│   │   ├── functional/
│   │   │   ├── ui/                       # UI tests
│   │   │   ├── api/                      # API tests
│   │   │   └── database/                 # Database tests
│   │   └── integration/                  # Integration tests
│   │
│   ├── project_2/                        # Project 2: Dispatch & logistics
│   │   ├── functional/
│   │   │   ├── ui/
│   │   │   ├── api/
│   │   │   └── database/
│   │   └── integration/
│   │
│   └── project_3/            # Project 3: Customs management
│       ├── functional/
│       │   ├── ui/
│       │   ├── api/
│       │   └── database/
│       └── integration/
│
└── unit/                                  # Framework unit tests

Layered Architecture

  1. Test Layer (tests/) - Multi-project organization with Pytest
  2. Page Object Layer (pages/) - Encapsulated locators per project
  3. Core Engine Layer (pyplaykit/core/) - Playwright lifecycle and browser management
  4. Utilities Layer (pyplaykit/utils/) - Logging, data loading, assertions, API clients
  5. Configuration Layer (config/) - Framework + project-specific configs
  6. Plugin Layer (plugins/) - Extensible plugin architecture
  7. Orchestration Layer (pyplaykit/orchestration/) - Dependency-aware execution
  8. Resilience Layer (pyplaykit/resilience/) - Self-healing locator resolution
  9. Integration Layer (pyplaykit/integrations/) - Jira and test management adapters

Architecture Diagrams

Architecture Overview — the full layered dependency stack, cross-cutting layers (plugins, orchestration, resilience), the MCP read-only sidecar, and the security & governance rail:

PyPlayKit Architecture Overview

Data Comparison Flow — config-driven Excel / CSV / DB comparison: sources → loader → comparison engine → result + HTML report:

Data Comparison Flow

Security Posture — defense-in-depth: runtime secrets/masking/auth controls plus the bandit / pip-audit / gitleaks CI scanning pipeline:

Security Posture

Diagrams are hosted in the public ShansDocsForPublicUsage repository so they render on the PyPI page. The source SVGs and the full architecture reference (docs/ARCHITECTURE.md — 9 Mermaid views covering layers, runtime lifecycle, config resolution, data engineering, plugins/observability, MCP, and security) live in the project's docs/ tree; regenerate the PNGs after editing an SVG with python scripts/svg_to_png.py.


Project Structure

pyplaykit/
├── config/
│   ├── config.yaml                       # Framework configuration
│   ├── environments.yaml                 # Global environments
│   └── projects/                         # Project-specific configs
│       ├── project_1.yaml
│       ├── project_2.yaml
│       └── project_3.yaml
│
├── core/
│   ├── base_page.py
│   ├── base_test.py
│   ├── browser_factory.py
│   └── playwright_manager.py
│
├── integrations/
│   └── adapters.py
│
├── orchestration/
│   └── planner.py
│
├── plugins/
│   ├── base.py
│   ├── registry.py
│   ├── api_plugin.py
│   ├── data_plugin.py
│   └── security_plugin.py
│
├── resilience/
│   └── locator_recovery.py
│
├── pages/                                # Page objects per project
│   ├── project_1/
│   ├── project_2/
│   └── project_3/
│
├── tests/
│   ├── projects/                         # All project tests
│   │   ├── project_1/
│   │   ├── project_2/
│   │   └── project_3/
│   └── unit/                             # Framework unit tests
│
├── utils/
│   ├── logger.py
│   ├── data_loader.py
│   ├── assertion_helper.py
│   ├── api_client.py
│   ├── response_validator.py
│   ├── data_validator.py
│   ├── config_reader.py
│   ├── observability.py
│   ├── environment_validator.py
│   └── tdm.py
│
├── test_data/                            # Test data per project
│   ├── project_1/
│   ├── project_2/
│   └── project_3/
│
├── reports/                              # Test reports and artifacts
│
├── conftest.py                           # Pytest configuration
├── pytest.ini                            # Pytest settings
├── pyproject.toml                        # Package metadata
├── requirements.txt                      # Dependencies
└── README.md                             # This file

Current Projects

1. project_1

Description: Quote management and generation system
Tests: tests/projects/project_1/
Config: config/projects/project_1.yaml
Marker: @pytest.mark.project_1

Run tests:

pytest tests/projects/project_1/
pytest -m project_1

2. project_2

Description: Dispatch and logistics management system
Tests: tests/projects/project_2/
Config: config/projects/project_2.yaml
Marker: @pytest.mark.project_2

Run tests:

pytest tests/projects/project_2/
pytest -m project_2

3. Customs Modernization

Description: Customs management system modernization
Tests: tests/projects/project_3/
Config: config/projects/project_3.yaml
Marker: @pytest.mark.project_3

Run tests:

pytest tests/projects/project_3/
pytest -m project_3

Running Tests

By Project

# project_1
pytest tests/projects/project_1/
pytest tests/projects/project_1/ -m smoke

# project_2
pytest tests/projects/project_2/
pytest tests/projects/project_2/ -m smoke

# Customs Modernization
pytest tests/projects/project_3/
pytest tests/projects/project_3/ -m smoke

By Test Type

# UI tests only
pytest tests/projects/project_1/functional/ui/

# API tests only
pytest tests/projects/project_2/functional/api/

# Database tests only
pytest tests/projects/project_3/functional/database/

# Integration tests only
pytest tests/projects/project_1/integration/

By Marker

# Project markers
pytest -m project_1
pytest -m project_2
pytest -m project_3

# Test type markers
pytest -m api
pytest -m ui
pytest -m database
pytest -m integration

# Combined markers
pytest -m "project_1 and smoke"
pytest -m "project_2 and api"

All Projects

# Run smoke tests for all projects
pytest tests/projects/ -m smoke

# Run all tests for all projects
pytest tests/projects/

# Run specific test type across all projects
pytest tests/projects/ -m api
pytest tests/projects/ -m ui

With Options

# Different environment
pytest tests/projects/project_1/ --pyplaykit-env qa
pytest tests/projects/project_2/ --pyplaykit-env uat

# Different browser
pytest tests/projects/project_1/ --pyplaykit-browser firefox --pyplaykit-headed

# Parallel execution
pytest tests/projects/project_1/ -n 4

# With readiness checks
pytest tests/projects/project_1/ --pyplaykit-readiness-check

Available pytest CLI Options

Registered in conftest.py:

  • --pyplaykit-env — target environment (dev, qa, uat, prod)
  • --pyplaykit-browser — browser (chromium, firefox, webkit)
  • --pyplaykit-headed — disable headless mode
  • --pyplaykit-base-url — override base URL
  • --pyplaykit-readiness-check — enable environment readiness checks
  • --pyplaykit-device — Playwright device descriptor for mobile emulation (e.g. "iPhone 13")
  • --pyplaykit-remote — run UI tests on BrowserStack (browserstack), Sauce Labs (sauce, Chromium only) or a Playwright server (playwright_server); settings in execution.remote, see docs/REMOTE_BROWSERS.md
  • --pyplaykit-mobile / --pyplaykit-mobile-device — run mobile tests (mobile_driver) on a local Appium 2 server; settings in mobile:, see docs/MOBILE_TESTING.md
  • --pyplaykit-quality-gate — enable quality gate; exits with code 1 on BLOCK verdict
  • --pyplaykit-update-snapshots — overwrite visual regression baselines
  • --pyplaykit-orchestration — enable suite dependency enforcement
  • --pyplaykit-project — active project name override
  • --pyplaykit-config — path to config directory override
  • --pyplaykit-dut-mode — device mode: sim (digital twins, default), emu, hil (real hardware)
  • --pyplaykit-bench / --pyplaykit-dut — device bench in scope / default DUT
  • --pyplaykit-device-record — record device traffic for replay twins and fidelity checks

Test Markers

Defined in pytest.ini:

Suite Markers

  • @pytest.mark.smoke - Critical path tests
  • @pytest.mark.sanity - Quick validation tests
  • @pytest.mark.regression - Full regression suite

Project Markers

  • @pytest.mark.project_1 - project_1 tests
  • @pytest.mark.project_2 - project_2 tests
  • @pytest.mark.project_3 - Customs Modernization tests

Test Type Markers

  • @pytest.mark.ui - UI functional tests
  • @pytest.mark.api - API functional tests
  • @pytest.mark.database - Database validation tests
  • @pytest.mark.integration - Integration tests

Feature Markers

  • @pytest.mark.login - Login functionality
  • @pytest.mark.quotes - Quote management
  • @pytest.mark.logistics - Logistics/dispatch
  • @pytest.mark.customs_mgmt - Customs management

Adding Tests

Where to Add Your Tests

Test Type Location Example
project_1 UI tests/projects/project_1/functional/ui/<feature>/ test_create_quote.py
project_1 API tests/projects/project_1/functional/api/<domain>/ test_quotes_api.py
project_2 UI tests/projects/project_2/functional/ui/<feature>/ test_dispatch_dashboard.py
project_2 API tests/projects/project_2/functional/api/<domain>/ test_orders_api.py
Customs UI tests/projects/project_3/functional/ui/<feature>/ test_declarations.py
Customs API tests/projects/project_3/functional/api/<domain>/ test_customs_api.py
Database tests tests/projects/<project>/functional/database/<category>/ test_data_integrity.py
Integration tests tests/projects/<project>/integration/ test_workflow_e2e.py

Example: Adding a New Test

1. Create page object (for UI tests):

# pages/project_1/quote_creation_page.py
from pyplaykit.core.base_page import BasePage

class QuoteCreationPage(BasePage):
    CUSTOMER_SELECT = "#customer"
    SAVE_BUTTON = "#save"
    
    def create_quote(self, customer: str):
        self.click(self.CUSTOMER_SELECT)
        self.click(f"option:has-text('{customer}')")
        self.click(self.SAVE_BUTTON)

2. Create test:

# tests/projects/project_1/functional/ui/quotes/test_create_quote.py
import pytest
from pages.project_1.quote_creation_page import QuoteCreationPage

@pytest.mark.project_1
@pytest.mark.smoke
@pytest.mark.quotes
def test_create_quote_with_valid_customer(page, runtime_options):
    quote_page = QuoteCreationPage(page, runtime_options["resilience_policy"])
    quote_page.navigate(runtime_options["base_url"] + "/quotes/new")
    quote_page.create_quote("ACME Corp")
    assert quote_page.is_quote_saved()

3. Run your test:

pytest tests/projects/project_1/functional/ui/quotes/test_create_quote.py -v

Device & Embedded Automation

Test devices — IoT, embedded Linux, PLCs, ECUs, BLE wearables, Android — with the same pytest workflow, against a simulated digital twin or real hardware. The test body does not change between modes.

pip install "pyplaykit[devices]"            # all protocols; or e.g. "pyplaykit[devices-serial,devices-mqtt]"
# config/devices.yaml
devices:
  default_mode: sim                          # sim | emu | hil
  duts:
    thermostat:
      capabilities: [serial]
      channels: {console: {type: serial, port: "${THERMOSTAT_PORT}", baudrate: 115200}}
      health: [{channel: console, send: version, expect: "FW=\\S+"}]
      sim: {twin: pyplaykit.devices.sim.twins:ThermostatTwin}
import pytest

@pytest.mark.device
@pytest.mark.requires_dut(capability="serial")
@pytest.mark.fault(kind="latency", delay=0.05)       # optional: degrade the link
def test_reports_firmware(dut):
    console = dut.channel("console")
    console.send("version\r\n")
    assert console.expect(r"FW=(\S+)", 5).group(1) == "2.1.0"
pytest -m device                                   # against the twins
pytest -m device --pyplaykit-dut-mode hil          # against real hardware (THERMOSTAT_PORT=COM7)
pyplaykit devices list | show thermostat | check   # bench inventory and readiness
pyplaykit devices editor --serve                   # edit config/devices.yaml with forms
python demo/device_lab/run_demo.py                 # virtual device lab: thirteen twins, six protocols
pytest demo/reader_lab -o addopts=                 # reader lab: barcode, RFID, fingerprint, badge, card

Transports: serial, MQTT, SSH, Telnet, HTTP, CoAP, Modbus, CAN/CAN FD, BLE, ADB. Also: fault injection (link, sensor, power), record/replay, a shared device pool with quarantine, power control and self-healing recovery. See the Device Automation Guide and the client / executive overviews.


Remote & Cloud Browsers

Run the same UI tests on a cloud grid or your own Playwright server; tests and page objects do not change. Off by default (execution.remote in config/config.yaml).

Provider Option Browsers Credentials
BrowserStack --pyplaykit-remote browserstack Chrome, Edge, Firefox, WebKit BROWSERSTACK_USERNAME, BROWSERSTACK_ACCESS_KEY
Sauce Labs --pyplaykit-remote sauce Chrome, Edge SAUCE_USERNAME, SAUCE_ACCESS_KEY
Playwright server --pyplaykit-remote playwright_server as the server provides none (execution.remote.ws_endpoint)
pytest tests/demo/functional/ui -m smoke --pyplaykit-remote browserstack --pyplaykit-browser webkit
pytest tests/demo/functional/ui -m smoke --pyplaykit-remote sauce        # set execution.remote.region

Each test is its own session by default (session_scope: worker shares one per pytest-xdist worker); its pass/fail appears on the provider's dashboard, and a failed test's session and video links go to the reports. Guide: docs/REMOTE_BROWSERS.md (also pyplaykit docs remote-browsers).

Mobile Testing

Test native and hybrid apps, and websites in a phone's own browser, on Android and iOS through Appium 2. A mobile_driver fixture hands each test a session, screen objects subclass MobilePage (locators as class constants, explicit waits, gestures, hybrid-app contexts, the resilience policy), and a failed test keeps a screenshot, the screen's source, the device log and a screen recording under reports/mobile/<test>/. Off by default (mobile: in config/config.yaml).

pip install "pyplaykit[mobile,devices-adb]"
pyplaykit init . --mobile                                   # sample screen object and test
pytest tests/project1/functional/mobile --pyplaykit-mobile local   # an Appium 2 server and an emulator or phone

The local provider connects to an Appium server you run, checks it before the first session, and can install and reset an Android app over ADB. BrowserStack and Sauce Labs devices are planned. CI runs the mobile layer on an Android emulator (.github/workflows/mobile-emulator.yml), and scripts/run_mobile_docker_demo.bat / .sh run the same demo on an emulator in Docker on your machine (watch it at http://localhost:6080). Guide: docs/MOBILE_TESTING.md (also pyplaykit docs mobile).

Data Comparison Patterns

PyPlayKit provides comprehensive data comparison capabilities using the built-in DataValidator utility.

Prerequisites for Data Comparison

# Option 1: Install with optional dependencies
pip install pyplaykit[data-comparison]

# Option 2: Install dependencies separately
pip install pandas openpyxl

# Option 3: Use the examples requirements file
pip install -r demo/requirements.txt

Note: Data comparison features require pandas and openpyxl. The framework will work without them for UI/API/Database testing.

Supported Comparison Types

Comparison Type Use Case Example
File-to-File Compare Excel, CSV, JSON files Validate data export/import
File-to-Database Verify data loads into database ETL validation
Database-to-File Validate database exports Report generation testing

Quick Example: Excel-to-Excel Comparison

Simple One-Line API (Recommended for QE Engineers):

import pytest
from pyplaykit.utils.data_comparison_utils import compare_excel_files

@pytest.mark.data
def test_compare_excel_files(logger):
    # ONE function call - framework handles everything!
    result = compare_excel_files(
        source_file="report_baseline.xlsx",
        target_file="report_current.xlsx",
        float_tolerance=0.01
    )
    
    # Check results
    logger.info(f"{result.summary}")
    logger.info(f"HTML Report: {result.report_path}")
    
    if not result.passed:
        pytest.fail(f"Validation failed! {result.failed_count} discrepancies found.")

Manual Validation (For custom logic):

import pytest
import pandas as pd
from pyplaykit import DataValidator

@pytest.mark.data
def test_compare_with_custom_logic(logger):
    df1 = pd.read_excel("baseline.xlsx")
    df2 = pd.read_excel("current.xlsx")
    
    records1 = df1.to_dict('records')
    records2 = df2.to_dict('records')
    
    for idx, (r1, r2) in enumerate(zip(records1, records2)):
        DataValidator.assert_records_equal(r1, r2)
    
    logger.info("✓ Files match!")

DataValidator Methods

Method Purpose
assert_records_equal() Compare two dictionaries exactly
assert_floats_equal() Compare numeric values with tolerance
assert_strings_equal_normalized() Compare strings with normalization
assert_collection_contains_record() Check if record exists in collection
assert_datetimes_equal() Compare datetime values

Runnable Examples

See demo/functional/database/data_comparison_examples.py for 8 complete examples:

# Run all data comparison examples
pytest demo/functional/database/data_comparison_examples.py -v -s

# Run specific example
pytest demo/functional/database/data_comparison_examples.py::test_excel_to_excel_basic -v -s

Interactive HTML Reports ⭐ NEW

Option 1: Automatic Report Generation (Recommended):

from pyplaykit.utils.data_comparison_utils import compare_excel_files

# Framework automatically tracks ALL mismatches and generates HTML report
result = compare_excel_files(
    source_file="baseline.xlsx",
    target_file="current.xlsx",
    float_tolerance=0.01
)

# Result includes:
print(f"Passed: {result.passed}")
print(f"Matched: {result.passed_count}")
print(f"Failed: {result.failed_count}")
print(f"Total Mismatches: {result.total_mismatches}")
print(f"Report: {result.report_path}")

Available Functions:

File-to-File:

  • compare_excel_files() - Excel to Excel
  • compare_csv_files() - CSV to CSV

File-to-Database:

  • compare_excel_to_db() - Excel to Database
  • compare_csv_to_db() - CSV to Database

Database-to-File:

  • compare_db_to_excel() - Database to Excel
  • compare_db_to_csv() - Database to CSV

Database-to-Database:

  • compare_db_to_db() - Database to Database

Advanced:

  • compare_dataframes() - DataFrame to DataFrame (custom sources)

Option 2: Manual Report Building (Advanced):

from pyplaykit.utils.data_comparison_report import DataComparisonReport

# For custom comparison logic
report = DataComparisonReport()
report.set_comparison_type("FILE_TO_FILE (Excel)")
report.set_source("baseline.xlsx", 100)
report.set_target("current.xlsx", 100)

# Your custom comparison logic here...
# report.add_mismatch(...) for each discrepancy

report.generate_report("reports/validation.html")

Report Features:

  • 📊 KPI dashboard with pass/fail rates
  • 🔍 Column-level filtering and search
  • 📋 Row-by-row mismatch details
  • 🎨 Color-coded status indicators
  • 📱 Mobile-responsive design
  • ⚡ Zero manual mismatch tracking required!

Examples:

Full Documentation


Unit Testing and Coverage

# Windows
scripts\run_unit_tests_with_coverage.bat

# Linux/macOS
bash scripts/run_unit_tests_with_coverage.sh

# View coverage report
# Open reports/coverage-html/index.html in browser

Coverage target: 95% (configured in pytest.ini)


Security Scanning

# Windows
scripts\run_security_reports.bat

# Linux/macOS
bash scripts/run_security_reports.sh

# View reports
# Open reports/security_reports/html/security_consolidated_report.html

Building Internal Package

# Windows
scripts\build_internal_package.bat

# Linux/macOS
bash scripts/build_internal_package.sh

Configuration

Framework Configuration

  • config/config.yaml - Framework defaults
  • config/environments.yaml - Global environment settings

Project Configuration

  • config/projects/project_1.yaml - project_1 settings
  • config/projects/project_2.yaml - project_2 settings
  • config/projects/project_3.yaml - Customs settings

Each project config includes:

  • Environment-specific URLs
  • API endpoints
  • Test users
  • Feature flags

Documentation for QE Engineers

Quick Start

Implementation

Architecture

Data Comparison & Validation

Project Guides


Key Features

Multi-Project Support

  • Separate test suites per project
  • Independent configurations
  • Project-specific page objects and test data
  • No merge conflicts between projects

Scalability

  • Designed for 50-100 QE engineers
  • Clear ownership boundaries
  • Parallel development across projects
  • Independent CI/CD pipelines

Test Types

  • UI Testing: Playwright-based with page objects
  • API Testing: REST API validation with response validators
  • Database Testing: Data integrity and migration validation
  • Integration Testing: Cross-layer consistency validation

Observability

  • Session-level metrics and KPIs
  • Automatic failure classification
  • Flaky test detection
  • Multi-persona reports (engineering, QA, leadership)

Resilience

  • Self-healing locators with fallback chains
  • DOM retry mechanisms
  • Confidence scoring
  • Optional audit logging

Plugin Architecture

  • Extensible hook system
  • Built-in plugins: API, Data, Security
  • Config-gated activation
  • Session and test-level hooks

Wave Implementation Status

Wave 1 ✅ Complete:

  • Observability tracker
  • Environment readiness checks
  • Session-level reporting

Wave 2 ✅ Complete:

  • Test data management utilities
  • Multi-level persona reports
  • Integration adapters (file and API)

Wave 3 ✅ Complete:

  • Plugin architecture with registry
  • Orchestration planner with dependency graphs

Wave 4 ✅ Complete:

  • Self-healing locators with fallback chains
  • Distribution packaging baseline
  • Multi-project organization structure

Wave 5 ✅ Complete:

  • MCP server for AI interoperability (11 read-only tools, 3 categories; 19 tools in 4 categories as of v1.32.0)
  • CLI layer — every MCP tool as a pyplaykit-mcp <subcommand> shell command
  • CLI version management with PyPI upgrade detection
  • Multi-project init as default layout; upgrade-to-multi migration command
  • Version resolution from installed package metadata
  • Allure environment capture fix for Windows/Python 3.14

Wave 5+ ✅ Complete:

  • Renamed examples/ → demo/ with layered functional/{ui,api,database}/ structure
  • HTML comparison report path logged to console after every comparison run
  • Dockerized SRC→TGT PostgreSQL comparison demo (demo/functional/database/postgres_src_tgt/)
  • CSV / XLS / XLSX file-comparison demo with multi-sheet behavior documentation (demo/functional/database/file_comparison/)

Wave 6 ✅ Complete — Enterprise Readiness Gap Closure (24/30 gaps):

  • GAP-08: Expanded DB assertion library (assert_date_range, assert_aggregate, assert_schema_match, per-row diff output)
  • GAP-10: ApiClient extensions (OAuth2 client-credentials, multipart upload, cookie management, bearer-token helpers)
  • GAP-19: WCAG accessibility testing via axe-core (BasePage.assert_accessible)
  • GAP-20: Mobile/tablet device emulation (--pyplaykit-device, BrowserFactory.build_context_kwargs)
  • GAP-22: Jenkins and Azure DevOps CI pipeline templates (ci/Jenkinsfile, ci/azure-pipelines.yml)
  • GAP-24: Quality Gate Engine (GateVerdict PASS/WARN/BLOCK, --pyplaykit-quality-gate, verdict JSON report)

Device & Embedded Automation ✅ Complete (v1.32.0) — simulation and fault-injection scope; hardware-in-the-loop stories await a bench kit:

  • pyplaykit.devices: device object model, 11 transports (serial, MQTT, SSH, Telnet, HTTP, CoAP, Modbus, CAN, BLE, ADB, fake)
  • Digital twins (Python / YAML) on a simulator host with real protocol endpoints; 11 reference twins, five of them identification and payment readers (barcode, RFID, fingerprint, employee badge, payment card)
  • config/devices.yaml, dut fixtures, requires_dut capability matching, --pyplaykit-dut-mode sim|emu|hil
  • Fault injection (link, sensor, power), record/replay twins and simulator fidelity scoring
  • Device pool (leases, health checks, quarantine), power drivers and recovery escalation
  • pyplaykit devices bench CLI and 6 MCP device tools (19 MCP tools in total)
  • v1.33.0: device configuration editor (pyplaykit devices editor), Virtual Device Lab demo and Device Lab Studio

Wave 7 🚧 In progress — QE agents (change intelligence, coverage and test design, test creation, execution intelligence): see design, plan and ADRs 0006–0016 in docs/adr/.

  • Done so far (Phase 1, disabled by default): versioned agent contract and JSON Schemas, agent discovery with digest pinning, the runner (limits, effective autonomy, typed results), the agents: config block, pyplaykit agents list | show | run, --pyplaykit-agents (local only) and the MCP run_agent tool.
  • GenAI spans for agent runs, LLM and tool calls (OpenTelemetry GenAI conventions), content capture off by default; agent.* events and pyplaykit_agent_* Prometheus metrics (tokens, cost, latency, tool calls, budget breaches, proposals); reports/agents/runs.jsonl and usage_summary.json, agent sections in the persona reports, a dashboard panel, pyplaykit agents usage and the MCP list_agents / get_agent_usage tools
  • v1.37.0: Phase 1 complete — tool layer and Python stack adapter, the test-impact map, and the first built-in agent change-intelligence (impact, risk score, regression plan applied with --pyplaykit-select-from in PR pipelines); user guide: pyplaykit docs agents
  • v1.37.1: first PyPI release of Phase 1 (v1.37.0 was not published), plus built-in agents test-review (test code quality) and test-creation (UI, API, database and device tests from test-case CSVs)
  • v1.37.2: test-creation commits to a feature branch of your sandbox repository and pushes it (never a pull request); hands-on playground with sample CSVs (pyplaykit docs test-creation --save-examples); scaffold demos skip without pandas
  • v1.38.0: built-in agent execution-optimizer: shards balanced by recorded durations (CI matrix and pytest-xdist), likeliest failures first, reruns only for known-flaky tests, orchestration stages; sessions with failure history also record per-test durations and flaky tests
  • v1.39.0: Phase 2 conformance and out-of-process runtime: pyplaykit agents conformance (23 contract, policy and security cases), the stdio transport for agents that are not built-in (JSON-RPC, minimal environment, F12 limits, tree kill), the Python stdio runner and a Node.js reference agent; context references confined (F9); agent authoring guide: pyplaykit docs agent-authoring
  • v1.40.0: agents measured and released: pyplaykit agents eval (golden datasets, replayed traffic, Wilson intervals, versioned baselines and a regression gate in CI), the release registry (agents release draft | evaluate | approve | publish | deprecate | revoke | retire, frozen hash-chained records, approvals by risk tier), assignments and a pre-run release check on every run, release lineage on runs, spans and artifacts, and the pyplaykit_list_agent_releases MCP tool
  • v1.40.1: first PyPI release of 1.40.0's content (1.40.0 was tagged but its publish failed); fixes the publish workflow
  • v1.41.0: agent release gates: pyplaykit agents release checklist (approval needs a complete checklist: owner, support matrix, permissions, scope, risk assessment, a quality, cost and latency baseline, disable and rollback paths, frozen at the first approval), per-file prompt and configuration history (release history, compare), and pyplaykit agents disable
  • v1.42.0: the docs land in your project: pyplaykit init copies every bundled doc to docs/pyplaykit/ (one unversioned folder, INDEX.md stamped with the version), pyplaykit docs --save refreshes it after an upgrade and keeps docs you edited, and pyplaykit docs, pyplaykit --version and the pytest header warn when it is from another version
  • v1.43.0: all framework code under the pyplaykit namespace (pyplaykit.utils, pyplaykit.core, ...), so a project's own utils/ no longer shadows PyPlayKit; the old top-level import names keep working with a deprecation warning until 2.0
  • v1.44.0: dependency version ranges instead of exact pins (tested at both ends of every core range in CI), every range above the last release with a known advisory (scripts/check_dependency_ranges.py, OSV, also weekly), and pyplaykit constraints for the exact tested set
  • v1.44.1: the gitleaks secret scan runs with gitleaks' default rules again (the config had replaced them with none); history rescanned clean, false positives allowlisted by exact value; GITLEAKS_CMD for a gitleaks outside the repo
  • v1.44.2: the CI security jobs fail on findings (gitleaks on history and working tree, pip-audit through scripts/security_gate.py) instead of always passing
  • v1.44.3: the device extras require filelock>=4.2.1; new advisories covered the releases >=3.20.3 admitted
  • v1.45.0: five reader twins (barcode, RFID, fingerprint, employee badge, payment card), the reader lab, and the readers in the Virtual Device Lab and the Studio
  • v1.46.0: the device automation guide in pyplaykit docs devices; pyplaykit init scaffolds the reader twins and an example test file per reader; Studio test list with runs with or without a fault
  • v1.47.0: lost HTTP replies for twins (SimDevice.drop_http_replies); python -m for the deprecated module names; a slow MQTT broker shutdown no longer fails a run; dispensing demo and detailed Studio suites in the repository
  • v1.48.0: scripts/run_dispensing_studio.bat/.sh open the Device Lab Studio on the dispensing stations; the Studio opens on any device (--focus <dut>, #<dut>)
  • v1.48.1: code-review fixes — the agent sandbox refuses patches that write outside its writable paths, redaction masks dictionary keys, the security gate fails on a malformed pip-audit report, regression selection runs the full suite when data or fixture files change, audit-log pruning and simulator shutdown fixed, ignore_row_order matching much faster on large sheets
  • v1.49.0: UI tests on BrowserStack or a Playwright server (--pyplaykit-remote, execution.remote), one session per test or per worker, failure links to BrowserStack's video and logs, remote errors classified as infrastructure, credentials never logged (US-23.1.1, ADR-0020)
  • v1.50.0: UI tests on Sauce Labs' Chrome and Edge (--pyplaykit-remote sauce, US-24.1.1); mobile test automation foundation: mobile_driver (Appium 2), MobilePage, mobile failure artifacts and a local Appium provider (--pyplaykit-mobile local, ADR-0021)
  • v1.51.0: mobile guide (pyplaykit docs mobile) and pyplaykit init --mobile (sample screen object and test); the mobile layer on an Android emulator in CI and in Docker (scripts/run_mobile_docker_demo.*); works on Appium 2 and 3 (US-26.2.2, US-26.3.1)
  • v1.36.0: agent governance and safety — policy through org governance, redaction, audit log, tool tiers, enforcement model
  • v1.35.0: agent observability — GenAI traces, agent.* events and pyplaykit_agent_* metrics, run log and reports
  • v1.34.0: LLM layer — provider-neutral requests, Anthropic / OpenAI-compatible / Ollama adapters, fake and record/replay clients, model tiers with fallback and retries, budgets, cost and a response cache. No built-in agents ship yet.

Reports and Artifacts

Test Reports

  • HTML report: reports/report.html
  • JUnit XML: reports/junit.xml
  • Coverage: reports/coverage-html/index.html

Observability

  • Summary: reports/observability/summary.json
  • KPI summary: reports/observability/kpi_summary.json
  • Engineering report: reports/observability/engineering_report.json
  • QA report: reports/observability/qa_functional_report.json
  • Leadership report: reports/observability/leadership_kpi_report.json

Viewing observability as an HTML report: the reports above are JSON. To turn them into a single, self-contained, styled HTML report with a generation date/time stamp, run the observability-report CLI command after a pytest run:

# Default: reads reports/observability/, writes a timestamped HTML file there
pyplaykit observability-report

# Point at a different reports directory
pyplaykit observability-report --reports-dir path/to/observability

# Choose an explicit output file
pyplaykit observability-report --output reports/observability/latest.html

# Generate and open in the default web browser
pyplaykit observability-report --open

The report includes a brand header with the generation timestamp and an overall PASSED / ATTENTION-NEEDED status pill, KPI cards (total, passed, failed, skipped, pass rate, flaky count, durations, automation coverage), leadership KPI metrics, a failed-tests table with failure classification, and a flaky-tests list. It requires that a pytest run has already written summary.json. See docs/CLI_REFERENCE.md for full option details.

Integration Exports

  • Jira export: reports/integrations/jira_export.json
  • Test management: reports/integrations/test_management_export.json

Resilience

  • Audit log: reports/resilience/recovery_audit.jsonl

Failure Artifacts

  • Screenshots: reports/screenshots/
  • Videos: reports/videos/
  • Logs: reports/framework.log
  • Device transcripts, simulator state, logcat: reports/devices/<test>/

Environment Variables

Framework

export PYPLAYKIT_TEST_USERNAME="standard_user"
export PYPLAYKIT_TEST_PASSWORD="secret_sauce"

project_1

export project_1_USERNAME="user"
export project_1_PASSWORD="pass"
export project_1_ADMIN_USERNAME="admin"
export project_1_ADMIN_PASSWORD="admin_pass"

project_2

export project_2_DISPATCHER_USERNAME="dispatcher"
export project_2_DISPATCHER_PASSWORD="pass"
export project_2_DRIVER_USERNAME="driver"
export project_2_DRIVER_PASSWORD="pass"
export project_2_ADMIN_USERNAME="admin"
export project_2_ADMIN_PASSWORD="admin_pass"

Customs Modernization

export CUSTOMS_USERNAME="customs_user"
export CUSTOMS_PASSWORD="pass"
export CUSTOMS_ADMIN_USERNAME="admin"
export CUSTOMS_ADMIN_PASSWORD="admin_pass"
export CUSTOMS_OFFICER_USERNAME="officer"
export CUSTOMS_OFFICER_PASSWORD="officer_pass"

CI/CD Integration

GitHub Actions Example

name: Multi-Project Tests

on: [push, pull_request]

jobs:
  project_1-smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          playwright install
      - name: Run project_1 Smoke Tests
        run: pytest tests/projects/project_1/ -m smoke -n 4

  project_2-smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          playwright install
      - name: Run project_2 Smoke Tests
        run: pytest tests/projects/project_2/ -m smoke -n 4

  customs-smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          playwright install
      - name: Run Customs Smoke Tests
        run: pytest tests/projects/project_3/ -m smoke -n 4

Jenkins

A ready-to-use declarative pipeline is provided at ci/Jenkinsfile.

Stages: Install -> Unit Tests (with coverage) -> Security Scan -> Build Validation

// Trigger with parameters
PYPLAYKIT_ENV = qa | uat | dev | prod
BROWSER       = chromium | firefox | webkit

Prerequisites: Python 3.12 on the agent PATH, gitleaks on PATH for the Security Scan stage, and two Jenkins credentials (pyplaykit-test-username, pyplaykit-test-password) of type Secret text. See docs/CI_INTEGRATION.md for the full setup checklist.

Azure DevOps

A ready-to-use pipeline definition is provided at ci/azure-pipelines.yml.

Stages run in order: Build -> Unit Tests + Coverage (parallel with Security Scan)

parameters:
  environment: qa | uat | dev | prod
  browser:     chromium | firefox | webkit

Test results publish automatically to the Tests tab; Cobertura coverage publishes to the Code Coverage tab. Secrets are injected from a variable group named pyplaykit-secrets. See docs/CI_INTEGRATION.md for the full setup checklist.


Team Organization

project_1 Team (20 QE)

  • UI Team (10): tests/projects/project_1/functional/ui/
  • API Team (6): tests/projects/project_1/functional/api/
  • Integration Team (4): tests/projects/project_1/integration/

project_2 Team (20 QE)

  • UI Team (10): tests/projects/project_2/functional/ui/
  • API Team (6): tests/projects/project_2/functional/api/
  • Integration Team (4): tests/projects/project_2/integration/

Customs Modernization Team (20 QE)

  • UI Team (8): tests/projects/project_3/functional/ui/
  • API Team (5): tests/projects/project_3/functional/api/
  • Database/Migration Team (5): tests/projects/project_3/functional/database/
  • Integration Team (2): tests/projects/project_3/integration/

Contributing

See CONTRIBUTING.md for comprehensive guidelines on:

  • Test development patterns
  • Page object creation
  • Test data management
  • Code quality standards
  • Common patterns and examples

Troubleshooting

Tests not discovered?

pytest tests/projects/<project>/ --collect-only

Import errors?

# Activate virtual environment
.\.venv\Scripts\Activate.ps1  # Windows
source .venv/bin/activate      # Linux/Mac

Configuration issues?

pytest --markers | grep -E "project_1|project_2|customs"

Support


License

[Your License Here]


Authors

  • Framework Team
  • QA Engineering Teams

PyPlayKit - Enterprise Test Automation at Scale 🚀

Metadata

Release files for pyplaykit 1.51.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pyplaykit 1.51.0
File Size Uploaded Provenance Actions
pyplaykit-1.51.0.tar.gz 1.7 MB With provenance Details for pyplaykit-1.51.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pyplaykit 1.51.0
File Interpreter ABI Platform Actions
pyplaykit-1.51.0-py3-none-any.whl Python 3 none any Details for pyplaykit-1.51.0-py3-none-any.whl

Total release size: 2.8 MB

Release files / pyplaykit-1.51.0.tar.gz

Download URL pyplaykit-1.51.0.tar.gz
Size 1.7 MB
Tags Source
SHA-256 checksum
How to use checksums
32eefd8e19192938203e6ebf6cbc5c60c838451e439eaa05fa01d40bf3733cb8
BLAKE2b-256 checksum
How to use checksums
10908929d24935f4fbc8ee8c697b04d4410bab5e46265ce6d569359d8af1021c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.

Transparency log

Release files / pyplaykit-1.51.0-py3-none-any.whl

Download URL pyplaykit-1.51.0-py3-none-any.whl
Size 1.1 MB
Tags Python 3
SHA-256 checksum
How to use checksums
062b540e4cb8fa3b7ae6eb93872598d036d1ec160bef3d4a111bd1a4bf43b19b
BLAKE2b-256 checksum
How to use checksums
8b12ad206b62095e047c89572d0488dcf000ff1d4c5d9a48a081155d83514560
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.

Transparency log

Security for pyplaykit 1.51.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for pyplaykit-1.51.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.51.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page