Report PlasEval evaluations
Installation
With uv (recommended)
See also uv.
uv tool install plaseval-report
With virtualenv
python3.13 -m virtualenv .venv
source .venv/bin/activate
python3.13 -m pip install plaseval-report
Usage
# If installed with uv tool
uvx plaseval-report --help
# If installed in virual environment
plaseval-report --help
Input data
The main file is a TSV file with the following base content (independent of comp or eval commands):
| Column ID | Type | Description |
|---|---|---|
species_id |
str |
Species code |
sample_uid |
str |
Sample UID |
method_code |
str |
Method code |
The following columns are specific to comp command:
| Column ID | Type | Description | Measure code |
|---|---|---|---|
Cuts |
float |
Normalized cut cost | cuts |
Joins |
float |
Normalized join cost | joins |
Extra_ctgs |
float |
Extra contigs cost | extra |
Missing_ctgs |
float |
Missing contigs cost | miss |
Dissimilarity |
float |
Dissimilarity | diss |
The following columns are specific to eval command:
| Column ID | Type | Description | Measure code |
|---|---|---|---|
unw_precision |
float |
Unweighted precision | unw_prec |
unw_recall |
float |
Unweighted recall | unw_recall |
unw_f1 |
float |
Unweighted F1 | unw_f1 |
w_precision |
float |
Weighted precision | w_prec |
w_recall |
float |
Weighted recall | w_recall |
w_f1 |
float |
Weighted F1 | w_f1 |
The configuration of the figures/stats is detailed in the config.yaml file (optional):
#
# (Optional) Method codes options
#
methods:
#
# (Optional) List of method codes to consider.
# If not set, all the method codes are considered, and the order is given by the TSV file.
# If the key to_show is not set, the method order is given by the `methods` list.
#
to_consider:
- <method_code>
- ...
#
# (Optional) List of method codes to show among the ones in `to_consider`.
# If the key to_show is set, the method order is given by the `to_show` list.
#
to_show:
- <method_code>
- ...
#
# (Optional) Map method code to labels
#
labels:
#
# One line labels.
# If not set, take the wrap labels otherwise the method codes.
#
one_line:
<method_code>: <str>
...
#
# Labels potentially on several lines.
# If not set, take the one_line labels otherwise the method codes.
#
wrap:
<method_code>: <str>
...
#
# (Optional) List of pairs of methods to annotate with stats
#
statannotate:
- - <method_code>
- <method_code>
- ...
#
# Method figure aesthetics
#
fig_aes:
palette: <str> # default: Set3, see https://matplotlib.org/stable/users/explain/colors/colormaps.html#qualitative
#
# Map method to palette index
# By default follow the order of the methods to show.
# If one method is missing in the map, automatically set the index to unused ones, then cycle.
#
color_indices:
<method_code>: <int> # The index of the color in the palette
...
#
# (Optional) Highlight specific methods in the figures (e.g. your own tool).
#
label_highlight:
#
# How to render the highlighted labels/marks.
# - bold: fontweight="bold" on the label/tick text
# - color: recolor the label/tick text and/or the bar/point using `color`
# - outline: add a colored edge/border around the bar or marker using `color`
# Several modes can be combined in a list, e.g. [bold, color].
#
mode: bold | color | outline | [bold, color, ...] # default: bold
#
# (Optional) Color used when mode includes `color` or `outline`.
# Any matplotlib-compatible color string (name, hex, etc.).
# If not set, the color is taken from the palette.
#
color: <str> # default: null
#
# List of method codes to highlight.
#
items:
- <method_code>
- ...
#
# (Optional) Species options
# It follows the same structure as for `methods`
#
species:
...
#
# (Optional) Measures options
#
measures:
#
# (Optional) List of measures to consider.
# If not set, all the measures are considered, and the order is given by the TSV file.
#
to_consider:
- <measure_code>
- ...
#
# (Optional) List of measures to show among the ones in `to_consider`.
#
to_show:
- <measure_code>
- ...
#
# (Optional) Samples removal strategy (for the methods listed in `methods`).
# The option is valid for al but result-presence figures and stats.
#
remove_samples: fails | nothing # default: fails
#
# Figure aesthetics (Optional, everything is optional)
#
fig_aes:
context: notebook | paper | talk | poster # default: notebook
focus: true | false # default: false
PlasEval comp/eval command results
The next section illustrates how to generate the figures. Generating the statistics tables is following the same process.
plaseval-report fig --help
plaseval-report stats --help
In the next sections, we must tell which methods we want to consider.
Result presence figures
Know for each tool how many samples have been evaluated by PlasEval:
plaseval-report fig res-presence "$merge_evals_tsv" "$figs_dir/res-presence" --config "$config_yaml"
Distribution figures
plaseval-report fig distribution "$merge_evals_tsv" "$figs_dir/distribution" --config "$config_yaml"
Versus figures
Generate a versus figure:
x_axis="pbhmf_rfpl"
y_axis="gpcc_rfpl"
plaseval-report comp fig versus "$merge_evals_tsv" "$figs_dir/versus" "$x_axis" "$y_axis" --config "$config_yaml"
Repeat stats figures
The above figures use an additional TSV file, repeat_stats.tsv.
For the ground truth, it contains the following columns:
| Column ID | Description |
|---|---|
sample_uid |
Sample ID |
species_id |
Species ID |
num_contigs |
Sum over the bins of the number of contigs |
num_unique_contigs |
Size of set of contigs present in at least one bin |
repeat_ratio |
Repeat ratio. Defined as num_unique_contigs / num_contigs. If defined (i.e. num_contigs > 0), it is a positive float $> 1$. If not defined (i.e. $0/0$), the cell is empty. |
For the predictions, it contains the following columns:
| Column ID | Description |
|---|---|
sample_uid |
Sample ID |
species_id |
Species ID |
method_code |
Method code |
num_contigs |
Sum over the bins of the number of contigs. None if there is no prediction. |
num_unique_contigs |
Size of set of contigs present in at least one bin. None if there is no prediction. |
repeat_ratio |
Repeat ratio. Defined as num_unique_contigs / num_contigs. If defined (i.e. num_contigs > 0), it is a positive float $> 1$. If not defined (i.e. $0/0$) or if there is no prediction, the cell is empty. |
An overview:
# To get a count of defined and undefined repeat ratios
plaseval-report fig repeat-stats overview count gt "$repeat_stats_tsv" "$figs_dir/repeat-stats/overview/count/gt" --config "$config_yaml"
# To get a distribution of the (defined) repeat ratios
plaseval-report fig repeat-stats overview distribution gt "$repeat_stats_tsv" "$figs_dir/repeat-stats/overview/distribution/gt" --config "$config_yaml"
Evaluation measures according to the repeat ratio:
# To get the evolution of evaluation measures according to (defined) repeat ratio
plaseval-report fig repeat-stats evals evolution "$merge_evals_tsv" "$repeat_stats_tsv" "$figs_dir/repeat_stats/evals/evolution" --config "$config_yaml"
# To get the distribution of evaluation measures for the undefined repeat ratios
plaseval-report fig repeat-stats evals distribution "$merge_evals_tsv" "$repeat_stats_tsv" "$figs_dir/repeat_stats/evals/distribution" --config "$config_yaml"
Joining the PlasEval comp and eval evaluations
In order to have the comp and eval figures on the same samples, we can join the two TSV files:
plaseval-report utils join-measures --help
join_tsv=comp_eval_merge_evals.tsv
plaseval-report utils join-measures "$merge_comp_tsv" "$merge_eval_tsv" "$join_tsv"
Joining the two set of measures is relevant when remove_samples option is set to fails because filtering on the joined TSV ensures all the samples have a comp and a eval evaluation.
Metadata
Release files for plaseval-report 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| plaseval_report-0.3.0.tar.gz | 62.1 kB | Without provenance | Details for plaseval_report-0.3.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| plaseval_report-0.3.0-py3-none-any.whl | Python 3 | none | any | Details for plaseval_report-0.3.0-py3-none-any.whl |
Total release size: 185.8 kB
Release files / plaseval_report-0.3.0.tar.gz
| Download URL | plaseval_report-0.3.0.tar.gz |
|---|---|
| Size | 62.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
15fd3d6aaf09831adedbc024a79b9180e09fd4632e852da4e53902677fc56de3
|
|
BLAKE2b-256 checksum How to use checksums |
eb68ff87eccfc262f415d6954caa15ca6ce82b0a2a78e6416afbf615c3d717e3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.9.30 {"installer":{"name":"uv","version":"0.9.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"12","id":"bookworm","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / plaseval_report-0.3.0-py3-none-any.whl
| Download URL | plaseval_report-0.3.0-py3-none-any.whl |
|---|---|
| Size | 123.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5d80fd2c3e3dd55d98a8bc07b2ac688ab4dfa500a6232cf896a5086491b9b60c
|
|
BLAKE2b-256 checksum How to use checksums |
a6e46565ba8e63ddaa6be724130af0236771e8c7d995c2530c397be299bb1832
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.9.30 {"installer":{"name":"uv","version":"0.9.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"12","id":"bookworm","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for plaseval-report 0.3.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.