TestFn Python SDK
Comprehensive self-hosted testing platform for developers.
Overview
TestFn is a developer-first testing solution that combines test execution, analytics, and visual testing into a single self-hosted platform. This is the Python SDK for TestFn.
Features
- ✅ Test Execution: Integration with
pytestfor discovery and execution. - ✅ Storage: Persistent storage of test runs and results using SQLAlchemy (SQLite default).
- ✅ Analytics: Flaky test detection, performance regression analysis, and run metrics.
- ✅ Reporters: Console and JSON reporters for real-time and post-run output.
- ✅ Visual Testing: Pixel-by-pixel screenshot comparison and diff generation.
Installation
pip install testfn
Quick Start
Basic Test Execution
import asyncio
from testfn import test_fn
async def main():
# Initialize runner
runner = test_fn(framework="pytest")
# Run tests matching a pattern
run = await runner.run(["tests/**/*.py"])
print(f"Pass rate: {run.summary.passed / run.summary.total * 100}%")
if __name__ == "__main__":
asyncio.run(main())
Using Analytics
from testfn import Storage, AnalyticsEngine
async def analyze():
storage = Storage()
analytics = AnalyticsEngine(storage)
# Detect flaky tests
flaky = await analytics.detect_flaky("tests/test_ui.py::test_login")
print(f"Flaky Score: {flaky.flaky_score}")
print(f"Recommendation: {flaky.recommendation}")
# Find performance regressions
regressions = await analytics.find_regressions(
baseline_run_id="run-1",
current_run_id="run-2"
)
for reg in regressions:
print(f"Test {reg['test_id']} got {reg['change']:.1%} slower")
Visual Testing
from testfn import VisualTester
tester = VisualTester(baseline_dir="baselines", diff_dir="diffs")
# Compare two screenshots
passed, score, diff_path = tester.compare_screenshots(
current_path="screenshots/current.png",
name="homepage",
threshold=0.1
)
if not passed:
print(f"Visual regression detected! Diff at: {diff_path}")
Development
# Setup virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -e .[dev]
# Run tests
pytest
License
MIT
Metadata
Release files for testfn 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| testfn-0.0.1.tar.gz | 9.8 kB | Without provenance | Details for testfn-0.0.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| testfn-0.0.1-py3-none-any.whl | Python 3 | none | any | Details for testfn-0.0.1-py3-none-any.whl |
Total release size: 21.0 kB
Release files / testfn-0.0.1.tar.gz
| Download URL | testfn-0.0.1.tar.gz |
|---|---|
| Size | 9.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4f62db85c9a1c1c4665a9931154b3e2156a6675445ef83a0ecd867ee6ccfdc5d
|
|
BLAKE2b-256 checksum How to use checksums |
7a58c99d3a454dd4467c8f41893d986cb3b6735d1aed449a72b49e780e4ac92c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.3
|
Release files / testfn-0.0.1-py3-none-any.whl
| Download URL | testfn-0.0.1-py3-none-any.whl |
|---|---|
| Size | 11.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
12c8260aa4503021dbe144ffa5e5537a91d51faeff91a13ef40e902a3b7320f4
|
|
BLAKE2b-256 checksum How to use checksums |
fd680e855627ea08a924681ed8f52fc61823674297b4288ef87ab836b45354f3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.3
|
Security for testfn 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.