This project has been archived by its maintainers, and is no longer receiving any updates.
testdino
TestDino CLI - Cache test metadata and upload Playwright test reports to TestDino platform
Quick Start
⚠️ Important: The
testdino uploadcommand requires a JSON report to be generated during test execution. Make sure to:
- Install required plugins:
pip install pytest-playwright-json pytest-html- Run your tests with
--playwright-jsonflag:pytest --playwright-json=test-results/report.json- Optionally generate HTML reports:
pytest --html=test-results/index.html --self-contained-html
Cache Command
# Cache test execution metadata after Playwright runs
testdino cache --token="your-api-token"
# With custom directory (recommended when test results are in a specific folder)
testdino cache --working-dir test-results --token="your-api-token"
# With verbose logging
testdino cache --working-dir test-results --verbose --token="your-api-token"
Last Failed Command
# Get last failed test cases for Playwright reruns
testdino last-failed --token="your-api-token"
# Get failed tests for specific shard
testdino last-failed --shard="2/5" --token="your-api-token"
# Run only last failed tests
pytest $(testdino last-failed --token="your-api-token")
Upload Command
# Upload test reports with attachments
testdino upload ./test-results --token="your-api-token"
# Upload with environment tag
testdino upload ./test-results --environment="staging" --token="your-api-token"
# Upload all attachments
testdino upload ./test-results --token="your-api-token" --upload-full-json
Features
- Test Metadata Caching - Store test execution metadata after Playwright runs
- Last Failed Tests - Retrieve and rerun only failed tests for faster CI/CD pipelines
- Shard Support - Get failed tests for specific test shards (e.g., shard 2 of 5)
- Environment Tagging - Tag uploads with environment labels (staging, production, qa)
- Zero Configuration - Auto-discovers Playwright reports and configuration
- Smart Shard Detection - Automatically detects Playwright shard information
- CI/CD Ready - Works seamlessly with GitHub Actions, GitLab CI, Jenkins, Azure DevOps
- Secure Authentication - Token-based API authentication
- Usage Limit Handling - Clear error messages with upgrade guidance
- Last Failed Tests - Retrieve and rerun only failed tests for faster CI/CD pipelines
- Zero Configuration - Auto-discovers Playwright reports and configuration
- Smart Shard Detection - Automatically detects Playwright shard information
- CI/CD Ready - Works seamlessly with GitHub Actions, GitLab CI, Jenkins, Azure DevOps
- Secure Authentication - Token-based API authentication
Commands
Cache Command
Store test execution metadata after Playwright runs.
# Basic usage
testdino cache --token="your-token"
# With custom working directory
testdino cache --working-dir ./test-results --token="your-token"
# With verbose logging
testdino cache --verbose --token="your-token"
Options:
| Option | Description | Default |
|---|---|---|
--working-dir <path> |
Directory to scan for test results | Current directory |
--cache-id <value> |
Custom cache ID override | Auto-detected |
-t, --token <value> |
TestDino API token | Required |
-v, --verbose |
Enable verbose logging | false |
Last Failed Command
Retrieve cached test failures for intelligent reruns.
# Basic usage
testdino last-failed --token="your-token"
# Run only last failed tests
pytest $(testdino last-failed --token="your-token")
# With custom branch and commit
testdino last-failed --branch="main" --commit="abc123" --token="your-token"
Options:
| Option | Description | Default |
|---|---|---|
--cache-id <value> |
Custom cache ID override | Auto-detected |
--branch <value> |
Custom branch name override | Auto-detected |
--commit <value> |
Custom commit hash override | Auto-detected |
-t, --token <value> |
TestDino API token | Required |
-v, --verbose |
Enable verbose logging | false |
Upload Command
Upload test reports with attachments.
⚠️ Important: The
testdino uploadcommand requires a JSON report to function. You must:
- Install required plugins:
pip install pytest-playwright-json pytest-html- Run your tests with the
--playwright-jsonflag:pytest --playwright-json=test-results/report.json- Optionally generate HTML reports:
pytest --html=test-results/index.html --self-contained-html
# Basic upload
testdino upload ./test-results --token="your-token"
# Upload with attachments
testdino upload ./test-results --token="your-token" --upload-images --upload-videos
# Upload all attachments
testdino upload ./test-results --token="your-token" --upload-full-json
Options:
| Option | Description |
|---|---|
<report-directory> |
Directory containing Playwright reports (required) |
-t, --token <value> |
TestDino API token (required) |
--upload-images |
Upload image attachments |
--upload-videos |
Upload video attachments |
--upload-html |
Upload HTML reports |
--upload-traces |
Upload trace files |
--upload-files |
Upload file attachments (.md, .pdf, .txt, .log) |
--upload-full-json |
Upload all attachments |
-v, --verbose |
Enable verbose logging |
Environment Variables
export TESTDINO_TOKEN="your-api-token"
export TESTDINO_TARGET_ENV="staging" # Optional: Set target environment
CI/CD Integration
GitHub Actions
Basic Example:
name: Playwright Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino
playwright install chromium --with-deps
- name: Run Playwright Tests
shell: bash
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
run: |
mkdir -p test-results
# Case 1: Re-run failed jobs → run only failed tests
if [[ "${{ github.run_attempt }}" -gt 1 ]]; then
testdino last-failed --token="$TESTDINO_TOKEN" > last-failed-flags.txt
FAILED_TESTS="$(cat last-failed-flags.txt | tail -1)"
if [[ -z "$FAILED_TESTS" ]]; then
exit 0
fi
# IMPORTANT: Use eval to preserve quotes in the -k expression
eval "pytest $FAILED_TESTS --playwright-json=test-results/report.json --html=test-results/index.html --self-contained-html -p no:selenium -v" || true
exit 0
fi
# Case 2: Normal execution (first run)
pytest \
--playwright-json=test-results/report.json \
--html=test-results/index.html \
--self-contained-html \
-p no:selenium \
-v || true
- name: Cache rerun metadata
if: always()
run: testdino cache --working-dir test-results --token="${{ secrets.TESTDINO_TOKEN }}" -v
- name: Upload test reports
if: always()
run: testdino upload ./test-results --token="${{ secrets.TESTDINO_TOKEN }}"
Advanced Example with Sharding and Intelligent Reruns:
Note: Sharding is optional. The basic example above works perfectly without sharding. Use the advanced example below only if you want to run tests in parallel across multiple shards for faster execution. The CLI automatically detects shard configuration from environment variables (
SHARD_INDEX,SHARD_TOTAL) or Playwright config, and defaults to a single shard (1/1) if none is detected.
name: Playwright Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3, 4, 5]
shardTotal: [5]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest pytest-playwright pytest-playwright-json pytest-html pytest-xdist pytest-shard
pip install testdino
playwright install --with-deps chromium
- name: Run Playwright Tests
shell: bash
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
SHARD_INDEX: ${{ matrix.shardIndex }}
SHARD_TOTAL: ${{ matrix.shardTotal }}
run: |
mkdir -p test-results
# Case 1: Re-run failed jobs → run only failed tests
if [[ "${{ github.run_attempt }}" -gt 1 ]]; then
testdino last-failed --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --token="$TESTDINO_TOKEN" > last-failed-flags.txt
FAILED_TESTS="$(cat last-failed-flags.txt | tail -1)"
if [[ -z "$FAILED_TESTS" ]]; then
exit 0
fi
# IMPORTANT: Use eval to preserve quotes in the -k expression
# pytest-shard uses 0-indexed shard IDs, so subtract 1 from SHARD_INDEX
# Use shard-specific report name
SHARD_ID=$(( $SHARD_INDEX - 1 ))
eval "pytest $FAILED_TESTS --shard-id=$SHARD_ID --num-shards=$SHARD_TOTAL --playwright-json=test-results/report-${{ matrix.shardIndex }}.json --html=test-results/index.html --self-contained-html -p no:selenium -v" || true
exit 0
fi
# Case 2: Normal execution (first run) with sharding
# pytest-shard uses 0-indexed shard IDs, so subtract 1 from SHARD_INDEX
SHARD_ID=$(( $SHARD_INDEX - 1 ))
# Use shard-specific report name to prevent overwriting when artifacts are merged
pytest \
--shard-id=$SHARD_ID \
--num-shards=$SHARD_TOTAL \
--playwright-json=test-results/report-${{ matrix.shardIndex }}.json \
--html=test-results/index.html \
--self-contained-html \
-p no:selenium \
-v || true
- name: Upload HTML report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.shardIndex }}
path: test-results/
retention-days: 1
- name: Cache testdino last failed metadata
if: always()
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
SHARD_INDEX: ${{ matrix.shardIndex }}
SHARD_TOTAL: ${{ matrix.shardTotal }}
run: |
if [ -n "$TESTDINO_TOKEN" ]; then
testdino cache --working-dir test-results --token="$TESTDINO_TOKEN"
fi
upload-to-testdino:
name: Upload Test Results to TestDino
if: ${{ always() }}
needs: test
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install testdino
pip install pytest-playwright-json
- name: Download all test results
uses: actions/download-artifact@v4
with:
pattern: test-results-*
merge-multiple: true
path: combined-test-results
- name: Merge JSON reports from all shards
run: |
python3 -m pytest_playwright_json.merge -d combined-test-results -o combined-test-results/report.json -v
- name: Upload to TestDino
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
run: |
if [ -z "$TESTDINO_TOKEN" ]; then
echo "TESTDINO_TOKEN not set, skipping upload"
exit 0
fi
if [ -d "combined-test-results" ] && [ -n "$(ls -A combined-test-results 2>/dev/null)" ]; then
# Basic upload - uploads merged JSON report
testdino upload ./combined-test-results --token="$TESTDINO_TOKEN"
else
echo "No test results found to upload"
fi
GitLab CI
image: python:3.11
stages:
- test
playwright-tests:
stage: test
script:
- pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino
- playwright install chromium --with-deps
- pytest --playwright-json=test-results/report.json --html=test-results/index.html --self-contained-html
- testdino upload ./test-results --token="$TESTDINO_TOKEN"
when: always
Jenkins
pipeline {
agent any
environment {
TESTDINO_TOKEN = credentials('testdino-token')
}
stages {
stage('Test') {
steps {
sh 'pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino'
sh 'playwright install chromium --with-deps'
sh 'pytest --playwright-json=test-results/report.json --html=test-results/index.html --self-contained-html'
sh 'testdino upload ./test-results --token="$TESTDINO_TOKEN"'
}
}
}
}
Authentication
Getting Your Token
- Sign up at TestDino
- Navigate to Settings > API Tokens
- Generate a new token
- Store it securely in your CI/CD secrets
Token Format:
trx_{environment}_{64-character-hex-string}
Security Best Practices:
- Never commit tokens to version control
- Use environment variables or CI/CD secrets
- Rotate tokens regularly
Examples
Basic Workflow
# Run tests with JSON and HTML reports
pytest \
--playwright-json=test-results/report.json \
--html=test-results/index.html \
--self-contained-html
# Cache metadata (specify directory where test results are located)
testdino cache --working-dir test-results --token="your-token"
Intelligent Test Reruns
# Run tests with JSON and HTML reports, then cache results
pytest \
--playwright-json=test-results/report.json \
--html=test-results/index.html \
--self-contained-html
testdino cache --working-dir test-results --token="your-token"
# On next run, execute only previously failed tests
pytest $(testdino last-failed --token="your-token")
Complete CI/CD Workflow
# Run all tests with JSON and HTML reports
pytest \
--playwright-json=test-results/report.json \
--html=test-results/index.html \
--self-contained-html
# Cache test metadata (specify directory where test results are located)
testdino cache --working-dir test-results --token="$TESTDINO_TOKEN"
# Rerun only failed tests
if [ $? -ne 0 ]; then
FAILED=$(testdino last-failed --token="$TESTDINO_TOKEN")
[ -n "$FAILED" ] && pytest $FAILED
fi
Support
- Documentation: docs.testdino.com
- Issues: GitHub Issues
- Email: support@testdino.com
Made with love by the TestDino team
Metadata
Release files for testdino 1.0.24
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| testdino-1.0.24.tar.gz | 67.0 kB | Without provenance | Details for testdino-1.0.24.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| testdino-1.0.24-py3-none-any.whl | Python 3 | none | any | Details for testdino-1.0.24-py3-none-any.whl |
Total release size: 147.3 kB
Release files / testdino-1.0.24.tar.gz
| Download URL | testdino-1.0.24.tar.gz |
|---|---|
| Size | 67.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
db831bf2462adb0088dfa46bf3fa3f9d8b04bdab67d999d69ad4e551de5153bd
|
|
BLAKE2b-256 checksum How to use checksums |
625d038a6b7e5c0b440d6b40ccc7f71cef42a996a3fcba4479dda5622f713fd6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.2
|
Release files / testdino-1.0.24-py3-none-any.whl
| Download URL | testdino-1.0.24-py3-none-any.whl |
|---|---|
| Size | 80.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
148f3fa488078cd506c69add9962959ba992c750faa18d98734c30fdcfb3de8a
|
|
BLAKE2b-256 checksum How to use checksums |
556c453dc00419dc3816738a0b86824a9dcca400dad4ffb0911f2d4af37ec134
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.2
|
Security for testdino 1.0.24
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.