📊 TestGraph
Unified Python Testing Analytics & Regression Diagnostics
Local-first test reporting, failure clustering, flakiness heuristics, and AI/ML model evaluation in a single interactive workflow.
🌟 Why TestGraph?
Modern software engineering teams frequently build hybrid architectures: backend APIs, distributed pipelines, and AI/ML inference models. When automated runs fail or degrade, developers are forced to navigate fragmented tools—traditional unit test runners on one hand, and specialized ML monitoring platforms on the other.
TestGraph unifies software test outcomes and ML evaluation metrics under a single, versioned data model:
- 🔒 Local-First & Safe: Zero daemons, zero external accounts, zero network calls. Uses
defusedxmlto neutralize XXE attacks and auto-escapes all untrusted stack traces. - 📉 Regression Diagnostics: Automatically catches newly failing tests, execution slowdowns, and ML metric drops against baseline runs.
- 🔍 Failure Clustering: Normalizes volatile tokens (hex addresses, UUIDs, temporary paths) into deterministic signatures to group related errors.
- 🎲 Historical Flakiness: Tracks outcome transitions across historical runs, flagging flaky tests and warning when sample sizes are insufficient ($N < 5$).
- 🤖 AI/ML Evaluation: Built-in calculation of classification metrics (Accuracy, Macro/Weighted F1, Confusion Matrix) and latency benchmarks.
- 📊 Standalone Interactive HTML: Generates self-contained HTML reports with embedded Plotly charts that work completely offline.
🚀 Quick Start
Installation
pip install testgraph-analytics
For development and additional data science tooling:
pip install "testgraph-analytics[dev]"
💻 Command-Line Interface (CLI)
TestGraph comes with a fast, versatile CLI:
1. Inspect Run Summary
testgraph summary reports/junit.xml
2. Compare Runs & Gate CI on Regressions
# Returns exit code 1 if regressions are detected (ideal for CI gates)
testgraph compare reports/baseline_junit.xml reports/current_junit.xml
3. Generate Standalone Interactive HTML Report
testgraph report reports/junit.xml -o reports/testgraph-report.html --baseline reports/baseline_junit.xml
4. Persist and Query Run History (SQLite)
# Save run to SQLite
testgraph record reports/junit.xml --db testgraph.db
# Inspect recent runs
testgraph history --db testgraph.db
# Detect flaky tests across historical executions
testgraph history --db testgraph.db --flaky
# View history of a specific test case
testgraph history --db testgraph.db --case "tests.test_api::test_health#4a1b8c2d"
🐍 Programmatic Python API
Ingesting JUnit XML & Analyzing Failures
from testgraph.ingestion import JUnitIngester
from testgraph.analytics import compute_run_summary, cluster_failures
# Safe parsing with defusedxml
ingester = JUnitIngester()
run = ingester.parse_file("reports/junit.xml")
# Outcome summary
summary = compute_run_summary(run)
print(
f"Pass Rate: {summary.effective_pass_rate * 100:.1f}% ({summary.passed}/{summary.total_tests})"
)
# Deterministic failure clustering
clusters = cluster_failures(run)
for c in clusters:
print(f"[{c.signature_id}] {c.count}x {c.error_type}: {c.normalized_message}")
Unifying ML Evaluations & Software Tests
from testgraph.models import TestRun
from testgraph.ml import create_evaluation
from testgraph.report import ReportRenderer
# Compute ML classification metrics & latency
ml_eval = create_evaluation(
model_name="fraud_classifier_v2",
y_true=["legit", "fraud", "legit", "fraud"],
y_pred=["legit", "legit", "legit", "fraud"],
latencies_ms=[14.2, 18.5, 12.1, 15.0],
)
# Attach to TestRun
run = TestRun(source="ml_pipeline")
run.ml_evaluations.append(ml_eval)
# Generate interactive standalone HTML report
renderer = ReportRenderer()
renderer.render_to_file(run, "reports/ml_and_tests.html")
🛠️ GitHub Actions CI Integration
Add TestGraph to your GitHub Actions workflow for automated regression gating and HTML report artifacts:
name: Test Suite & Analytics
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install Dependencies
run: |
pip install -e .
pip install pytest
- name: Run Pytest
run: |
pytest -v --junitxml=reports/current.xml
- name: Gate on Regressions
if: always()
run: |
# Compare with baseline if present
if [ -f "reports/baseline.xml" ]; then
testgraph compare reports/baseline.xml reports/current.xml
fi
- name: Generate HTML Report
if: always()
run: |
testgraph report reports/current.xml -o reports/testgraph-report.html
- name: Upload Report Artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: testgraph-html-report
path: reports/testgraph-report.html
📚 Documentation
- Architecture & Design
- Statistical Methods & Formulas
- Programmatic API Reference
- Competitive Landscape Analysis
- Project Roadmap
📄 License
TestGraph is released under the Apache License 2.0.
Metadata
Release files for krixs-test-for-complet-complaition 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| krixs_test_for_complet_complaition-0.1.0.tar.gz | 47.0 kB | Without provenance | Details for krixs_test_for_complet_complaition-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| krixs_test_for_complet_complaition-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for krixs_test_for_complet_complaition-0.1.0-py3-none-any.whl |
Total release size: 96.0 kB
Release files / krixs_test_for_complet_complaition-0.1.0.tar.gz
| Download URL | krixs_test_for_complet_complaition-0.1.0.tar.gz |
|---|---|
| Size | 47.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9f45377eca29a6e242b6dfd7bec23898bec5e6c98c41864eef69b5ad2ba32938
|
|
BLAKE2b-256 checksum How to use checksums |
94bc93b6021fb563d0ab0a7c074c294e2f939097edcea3b3b3801eb422da9244
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.0
|
Release files / krixs_test_for_complet_complaition-0.1.0-py3-none-any.whl
| Download URL | krixs_test_for_complet_complaition-0.1.0-py3-none-any.whl |
|---|---|
| Size | 49.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8d84083941b03e133a0bdc0d2df21446a8d60174222f751739343acbdb070216
|
|
BLAKE2b-256 checksum How to use checksums |
39a2980a76623d05a3d55990ee8b2338327a7267a82bc2aa4278f5f09621dc70
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.0
|
Security for krixs-test-for-complet-complaition 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.