Skip to main content

TestFn Python SDK

Comprehensive self-hosted testing platform for developers.

Overview

TestFn is a developer-first testing solution that combines test execution, analytics, and visual testing into a single self-hosted platform. This is the Python SDK for TestFn.

Features

  • ✅ Test Execution: Integration with pytest for discovery and execution.
  • ✅ Storage: Persistent storage of test runs and results using SQLAlchemy (SQLite default).
  • ✅ Analytics: Flaky test detection, performance regression analysis, and run metrics.
  • ✅ Reporters: Console and JSON reporters for real-time and post-run output.
  • ✅ Visual Testing: Pixel-by-pixel screenshot comparison and diff generation.

Installation

pip install testfn

Quick Start

Basic Test Execution

import asyncio
from testfn import test_fn

async def main():
    # Initialize runner
    runner = test_fn(framework="pytest")
    
    # Run tests matching a pattern
    run = await runner.run(["tests/**/*.py"])
    
    print(f"Pass rate: {run.summary.passed / run.summary.total * 100}%")

if __name__ == "__main__":
    asyncio.run(main())

Using Analytics

from testfn import Storage, AnalyticsEngine

async def analyze():
    storage = Storage()
    analytics = AnalyticsEngine(storage)
    
    # Detect flaky tests
    flaky = await analytics.detect_flaky("tests/test_ui.py::test_login")
    print(f"Flaky Score: {flaky.flaky_score}")
    print(f"Recommendation: {flaky.recommendation}")
    
    # Find performance regressions
    regressions = await analytics.find_regressions(
        baseline_run_id="run-1", 
        current_run_id="run-2"
    )
    for reg in regressions:
        print(f"Test {reg['test_id']} got {reg['change']:.1%} slower")

Visual Testing

from testfn import VisualTester

tester = VisualTester(baseline_dir="baselines", diff_dir="diffs")

# Compare two screenshots
passed, score, diff_path = tester.compare_screenshots(
    current_path="screenshots/current.png",
    name="homepage",
    threshold=0.1
)

if not passed:
    print(f"Visual regression detected! Diff at: {diff_path}")

Development

# Setup virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e .[dev]

# Run tests
pytest

License

MIT

Metadata

Release files for testfn 0.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for testfn 0.0.1
File Size Uploaded Provenance Actions
testfn-0.0.1.tar.gz 9.8 kB Without provenance Details for testfn-0.0.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for testfn 0.0.1
File Interpreter ABI Platform Actions
testfn-0.0.1-py3-none-any.whl Python 3 none any Details for testfn-0.0.1-py3-none-any.whl

Total release size: 21.0 kB

Release files / testfn-0.0.1.tar.gz

Download URL testfn-0.0.1.tar.gz
Size 9.8 kB
Tags Source
SHA-256 checksum
How to use checksums
4f62db85c9a1c1c4665a9931154b3e2156a6675445ef83a0ecd867ee6ccfdc5d
BLAKE2b-256 checksum
How to use checksums
7a58c99d3a454dd4467c8f41893d986cb3b6735d1aed449a72b49e780e4ac92c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Release files / testfn-0.0.1-py3-none-any.whl

Download URL testfn-0.0.1-py3-none-any.whl
Size 11.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
12c8260aa4503021dbe144ffa5e5537a91d51faeff91a13ef40e902a3b7320f4
BLAKE2b-256 checksum
How to use checksums
fd680e855627ea08a924681ed8f52fc61823674297b4288ef87ab836b45354f3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Security for testfn 0.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.0.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page