pytest-llm-report
Human-friendly pytest test reports with optional LLM annotations.
Features
- Test inventory with status (pass/fail/skip/xfail) and duration
- Per-test covered files and line ranges (using pytest-cov + coverage contexts)
- Per-file source coverage summary (covered/missed/percentage)
- Optional LLM-generated annotations:
- Scenario: What the test verifies
- Why needed: What regression/bug it prevents
- Key assertions: Critical checks performed
- Smart Context: Automatically fetches model context limits (Gemini, Ollama, LiteLLM) to maximize relevant code included in prompts without hitting token limits.
- Compact Context: Automatically strips docstrings and optimizes whitespace to reduce token consumption (enabled by default).
- Token Usage Reporting: Tracks and displays input, output, and total token usage for each annotation and the entire test run.
- HTML and JSON output formats
- Dark mode support (auto-detects system preference)
- Optional PDF generation
- Detailed Versioning: Reports include exact plugin and repository versions (plus Git SHA/dirty status) for reproducibility
- Aggregation across multiple test runs (see Aggregation)
Installation
pip install pytest-llm-report
Or with uv:
uv add pytest-llm-report
Optional Dependencies
Install additional features as needed:
# Ollama provider
pip install pytest-llm-report[ollama]
# Gemini provider
pip install pytest-llm-report[gemini]
# LiteLLM provider (supports OpenAI, Anthropic, etc.)
pip install pytest-llm-report[litellm]
# PDF generation
pip install pytest-llm-report[pdf]
# All features
pip install pytest-llm-report[all]
With uv:
uv add "pytest-llm-report[ollama]" # or gemini, litellm, pdf, all
Quick Start
Run pytest with coverage contexts enabled:
pytest --cov=your_package --cov-context=test --llm-report=report.html
Configuration
Configure via pyproject.toml:
[tool.pytest_llm_report]
provider = "none" # "none", "ollama", "litellm", or "gemini"
report_json = "reports/pytest_llm_report.json"
Documentation
📖 Full documentation: palakpsheth.github.io/pytest-llm-report
Requirements
- Python 3.11+
- pytest >= 7.0.0
- pytest-cov >= 4.0.0
Contributing
Contributions are welcome! Please see:
- Contributing Guide - Development setup and guidelines
- Code of Conduct - Community guidelines
- Security Policy - Reporting vulnerabilities
- Changelog - Version history
License
MIT
Metadata
Release files for pytest-llm-report 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pytest_llm_report-0.2.1.tar.gz | 134.0 kB | Without provenance | Details for pytest_llm_report-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pytest_llm_report-0.2.1-py3-none-any.whl | Python 3 | none | any | Details for pytest_llm_report-0.2.1-py3-none-any.whl |
Total release size: 216.4 kB
Release files / pytest_llm_report-0.2.1.tar.gz
| Download URL | pytest_llm_report-0.2.1.tar.gz |
|---|---|
| Size | 134.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3f9a5550ccfc63c3fb25aca6df01d73db05728c70cfec3a72eed418ef42f03d7
|
|
BLAKE2b-256 checksum How to use checksums |
73ab304433352c63bb42ef5b3e214ebdc73cba1fe5b8999ed9241f9345791bd0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pytest_llm_report-0.2.1-py3-none-any.whl
| Download URL | pytest_llm_report-0.2.1-py3-none-any.whl |
|---|---|
| Size | 82.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bbb177567aada84b18d9d2d788cc03a71fcec92295252b9f4ba535667b25b709
|
|
BLAKE2b-256 checksum How to use checksums |
473364cecff85ddeb8f3eeb99869f8134e139603dc478e24398fda48086750b4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for pytest-llm-report 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.