pytest-charisma
A pytest plugin that streams test results to the Charisma ingestion API as tests execute.
Results are batched and sent in a background thread so test execution is not blocked by network I/O. The plugin implements retry-once semantics and a circuit breaker (3 consecutive failures disables streaming for the rest of the session).
Installation
pip install pytest-charisma
No token, no index URL — works on any laptop and in any CI. The package is published to public PyPI; the Charisma repo itself stays private (only this client plugin wheel is public).
For local development from a checkout of the repo:
pip install -e packages/pytest-charisma
Publishing (maintainers)
Releases publish to public PyPI automatically from the merge-queue workflow using a PyPI API token.
Trusted publishing (OIDC) is not used because this repo runs on GitHub Enterprise Server, whose OIDC issuer PyPI does not trust. A stored API token is required instead.
One-time setup (done once by a PyPI project owner):
- Sign in to PyPI with the team-owned account (2FA enabled).
- Go to Account settings → API tokens → Add API token. Scope it to the
pytest-charismaproject once the project exists; for the very first publish use an account-scoped token, then re-scope it to the project afterward. - Copy the token (starts with
pypi-). - In the GitHub Enterprise repo, add it as an Actions secret named
PYPI_API_TOKEN(org-level secret recommended so both packages share it). - Bump
versioninpyproject.tomland merge — the workflow builds and publishes on the next release.
Rotate the token periodically and store it in the team secret manager.
Quick start
pytest \
--charisma-url https://api.charisma.example.com \
--charisma-token $CHARISMA_TOKEN \
--charisma-project-alias my-project
The plugin activates automatically when charisma_url is configured. If the URL is absent, the plugin stays silent and does nothing.
Configuration
Options are resolved with priority: CLI flag > pytest.ini / pyproject.toml > environment variable.
| CLI flag | ini key | Env var | Required | Description |
|---|---|---|---|---|
--charisma-url |
charisma_url |
CHARISMA_URL |
Yes | API base URL |
--charisma-token |
charisma_token |
CHARISMA_TOKEN |
Yes | Bearer token for authentication |
--charisma-project-alias |
charisma_project_alias |
CHARISMA_PROJECT_ALIAS |
Yes | Project alias in Charisma |
--charisma-batch-size |
charisma_batch_size |
CHARISMA_BATCH_SIZE |
No | Results per batch, 1–50 (default: 20) |
--charisma-build-id |
charisma_build_id |
CHARISMA_BUILD_ID |
No | CI build number or identifier |
--charisma-commit-sha |
charisma_commit_sha |
CHARISMA_COMMIT_SHA |
No | Git commit SHA |
--charisma-branch |
charisma_branch |
CHARISMA_BRANCH |
No | Git branch name |
--charisma-component |
charisma_component |
CHARISMA_COMPONENT |
No | Component alias within the project |
--charisma-source-url |
charisma_source_url |
CHARISMA_SOURCE_URL |
No | Link back to CI/CD run |
Configuration via pyproject.toml
[tool.pytest.ini_options]
charisma_url = "https://api.charisma.example.com"
charisma_project_alias = "my-project"
charisma_batch_size = "10"
charisma_branch = "main"
# Token should come from env var for security:
# export CHARISMA_TOKEN=your-token
Configuration via pytest.ini
[pytest]
charisma_url = https://api.charisma.example.com
charisma_project_alias = my-project
charisma_batch_size = 10
Environment variables only (CI-friendly)
export CHARISMA_URL=https://api.charisma.example.com
export CHARISMA_TOKEN=your-api-token
export CHARISMA_PROJECT_ALIAS=my-project
export CHARISMA_BUILD_ID=$CI_BUILD_NUMBER
export CHARISMA_COMMIT_SHA=$CI_COMMIT_SHA
export CHARISMA_BRANCH=$CI_BRANCH
export CHARISMA_SOURCE_URL=$CI_BUILD_URL
pytest
How it works
- Collection — After pytest collects tests, the plugin opens a launch session via
POST /api/v1/launches. - Execution — As each test completes, results are buffered. When the buffer reaches
batch_size, the batch is submitted to a background worker thread. - Flush — At session end, remaining results are flushed and the worker drains its queue (up to 30s timeout).
- Resilience — Each batch gets one retry on failure. After 3 consecutive batch failures, the circuit breaker trips and streaming is disabled for the rest of the session. Test execution is never affected.
Stable test-case ids
Annotate a test with @pytest.mark.test_case("<id>") to give it an explicit,
rename-stable Charisma test-case id:
import pytest
@pytest.mark.test_case("TPS-T1234")
def test_login():
...
The plugin streams that id as the testCaseId label, so Charisma groups the
result under TPS-T1234 — the same identity an Allure TMS/test-case label
establishes. Without the marker, results are grouped by the pytest nodeid, so a
rename or move starts a fresh case; with it, the case stays the same across
renames and across the pytest and Allure upload paths. The marker is registered
by the plugin, so it never triggers a PytestUnknownMarkWarning. The closest
marker wins if several apply; a blank id is ignored.
Rerun only planned tests
Set CHARISMA_TESTPLAN_PATH to a plan written by charismactl testplan get and the
plugin deselects everything else, down to the parametrized variant. It works with
streaming off. @allure.title variants are matched by parameter values; a planned test
that no longer matches exactly reruns every variant of its function (logged as a
warning). A missing or invalid plan file fails the run (usage error) instead of
running the whole suite.
Outcome mapping
| pytest outcome | Charisma status |
|---|---|
passed (call phase) |
passed |
failed (call phase) |
failed |
skipped |
skipped |
failed (setup/teardown) |
broken |
Requirements
- Python ≥ 3.10
- pytest ≥ 7.0, < 9.0
- httpx ≥ 0.24, < 1.0
Development
cd packages/pytest-charisma
pip install -e ".[dev]"
pytest
License
MIT
Metadata
Release files for pytest-charisma 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pytest_charisma-0.6.0.tar.gz | 51.2 kB | Without provenance | Details for pytest_charisma-0.6.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pytest_charisma-0.6.0-py3-none-any.whl | Python 3 | none | any | Details for pytest_charisma-0.6.0-py3-none-any.whl |
Total release size: 73.4 kB
Release files / pytest_charisma-0.6.0.tar.gz
| Download URL | pytest_charisma-0.6.0.tar.gz |
|---|---|
| Size | 51.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e98767ff3073186a04636987f524008bc9f2686102daf3296997b083e821fe0a
|
|
BLAKE2b-256 checksum How to use checksums |
b3ed0f1f7e3eb81c93b597703adc1efe180e9e6acc1835560b974eb4c602ad67
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.18
|
Release files / pytest_charisma-0.6.0-py3-none-any.whl
| Download URL | pytest_charisma-0.6.0-py3-none-any.whl |
|---|---|
| Size | 22.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ec3b7aeab37d8b808b688d5a49ab13181f89a7715cab4e4237c07f359c6dd4e9
|
|
BLAKE2b-256 checksum How to use checksums |
2c52171bc6b4f267cdf67253fb7c1fb8b5f028c2658a1d64205316afaa311f1c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.18
|
Security for pytest-charisma 0.6.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.