Skip to main content

Report PlasEval evaluations

PyPI ty Ruff uv Pipeline status License

Installation

See also uv.

uv tool install plaseval-report

With virtualenv

python3.13 -m virtualenv .venv
source .venv/bin/activate
python3.13 -m pip install plaseval-report

Usage

# If installed with uv tool
uvx plaseval-report --help
# If installed in virual environment
plaseval-report --help

Input data

The main file is a TSV file with the following base content (independent of comp or eval commands):

Column ID Type Description
species_id str Species code
sample_uid str Sample UID
method_code str Method code

The following columns are specific to comp command:

Column ID Type Description Measure code
Cuts float Normalized cut cost cuts
Joins float Normalized join cost joins
Extra_ctgs float Extra contigs cost extra
Missing_ctgs float Missing contigs cost miss
Dissimilarity float Dissimilarity diss

The following columns are specific to eval command:

Column ID Type Description Measure code
unw_precision float Unweighted precision unw_prec
unw_recall float Unweighted recall unw_recall
unw_f1 float Unweighted F1 unw_f1
w_precision float Weighted precision w_prec
w_recall float Weighted recall w_recall
w_f1 float Weighted F1 w_f1

The configuration of the figures/stats is detailed in the config.yaml file (optional):

#
# (Optional) Method codes options
#
methods:
  #
  # (Optional) List of method codes to consider.
  # If not set, all the method codes are considered, and the order is given by the TSV file.
  # If the key to_show is not set, the method order is given by the `methods` list.
  #
  to_consider:
    - <method_code>
    - ...
  #
  # (Optional) List of method codes to show among the ones in `to_consider`.
  # If the key to_show is set, the method order is given by the `to_show` list.
  #
  to_show:
    - <method_code>
    - ...
  #
  # (Optional) Map method code to labels
  #
  labels:
    #
    # One line labels.
    # If not set, take the wrap labels otherwise the method codes.
    #
    one_line:
      <method_code>: <str>
      ...
    #
    # Labels potentially on several lines.
    # If not set, take the one_line labels otherwise the method codes.
    #
    wrap:
      <method_code>: <str>
      ...
  #
  # (Optional) List of pairs of methods to annotate with stats
  #
  statannotate:
    - - <method_code>
      - <method_code>
    - ...
  #
  # Method figure aesthetics
  #
  fig_aes:
    palette: <str> # default: Set3, see https://matplotlib.org/stable/users/explain/colors/colormaps.html#qualitative
    #
    # Map method to palette index
    # By default follow the order of the methods to show.
    # If one method is missing in the map, automatically set the index to unused ones, then cycle.
    #
    color_indices:
      <method_code>: <int> # The index of the color in the palette
      ...
    #
    # (Optional) Highlight specific methods in the figures (e.g. your own tool).
    #
    label_highlight:
      #
      # How to render the highlighted labels/marks.
      # - bold:    fontweight="bold" on the label/tick text
      # - color:   recolor the label/tick text and/or the bar/point using `color`
      # - outline: add a colored edge/border around the bar or marker using `color`
      # Several modes can be combined in a list, e.g. [bold, color].
      #
      mode: bold | color | outline | [bold, color, ...] # default: bold
      #
      # (Optional) Color used when mode includes `color` or `outline`.
      # Any matplotlib-compatible color string (name, hex, etc.).
      # If not set, the color is taken from the palette.
      #
      color: <str> # default: null
      #
      # List of method codes to highlight.
      #
      items:
        - <method_code>
        - ...
#
# (Optional) Species options
# It follows the same structure as for `methods`
#
species:
  ...

#
# (Optional) Measures options
#
measures:
  #
  # (Optional) List of measures to consider.
  # If not set, all the measures are considered, and the order is given by the TSV file.
  #
  to_consider:
    - <measure_code>
    - ...
  #
  # (Optional) List of measures to show among the ones in `to_consider`.
  #
  to_show:
    - <measure_code>
    - ...
#
# (Optional) Samples removal strategy (for the methods listed in `methods`).
# The option is valid for al but result-presence figures and stats.
#
remove_samples: fails | nothing # default: fails
#
# Figure aesthetics (Optional, everything is optional)
#
fig_aes:
  context: notebook | paper | talk | poster # default: notebook
  focus: true | false                       # default: false

PlasEval comp/eval command results

The next section illustrates how to generate the figures. Generating the statistics tables is following the same process.

plaseval-report fig --help
plaseval-report stats --help

In the next sections, we must tell which methods we want to consider.

Result presence figures

Know for each tool how many samples have been evaluated by PlasEval:

plaseval-report fig res-presence "$merge_evals_tsv" "$figs_dir/res-presence" --config "$config_yaml"

Distribution figures

plaseval-report fig distribution "$merge_evals_tsv" "$figs_dir/distribution" --config "$config_yaml"

Versus figures

Generate a versus figure:

x_axis="pbhmf_rfpl"
y_axis="gpcc_rfpl"

plaseval-report comp fig versus "$merge_evals_tsv" "$figs_dir/versus" "$x_axis" "$y_axis" --config "$config_yaml"

Repeat stats figures

The above figures use an additional TSV file, repeat_stats.tsv. For the ground truth, it contains the following columns:

Column ID Description
sample_uid Sample ID
species_id Species ID
num_contigs Sum over the bins of the number of contigs
num_unique_contigs Size of set of contigs present in at least one bin
repeat_ratio Repeat ratio. Defined as num_unique_contigs / num_contigs. If defined (i.e. num_contigs > 0), it is a positive float $> 1$. If not defined (i.e. $0/0$), the cell is empty.

For the predictions, it contains the following columns:

Column ID Description
sample_uid Sample ID
species_id Species ID
method_code Method code
num_contigs Sum over the bins of the number of contigs. None if there is no prediction.
num_unique_contigs Size of set of contigs present in at least one bin. None if there is no prediction.
repeat_ratio Repeat ratio. Defined as num_unique_contigs / num_contigs. If defined (i.e. num_contigs > 0), it is a positive float $> 1$. If not defined (i.e. $0/0$) or if there is no prediction, the cell is empty.

An overview:

# To get a count of defined and undefined repeat ratios
plaseval-report fig repeat-stats overview count gt "$repeat_stats_tsv" "$figs_dir/repeat-stats/overview/count/gt" --config "$config_yaml"
# To get a distribution of the (defined) repeat ratios
plaseval-report fig repeat-stats overview distribution gt "$repeat_stats_tsv" "$figs_dir/repeat-stats/overview/distribution/gt" --config "$config_yaml"

Evaluation measures according to the repeat ratio:

# To get the evolution of evaluation measures according to (defined) repeat ratio
plaseval-report fig repeat-stats evals evolution "$merge_evals_tsv" "$repeat_stats_tsv" "$figs_dir/repeat_stats/evals/evolution" --config "$config_yaml"
# To get the distribution of evaluation measures for the undefined repeat ratios
plaseval-report fig repeat-stats evals distribution "$merge_evals_tsv" "$repeat_stats_tsv" "$figs_dir/repeat_stats/evals/distribution" --config "$config_yaml"

Joining the PlasEval comp and eval evaluations

In order to have the comp and eval figures on the same samples, we can join the two TSV files:

plaseval-report utils join-measures --help

join_tsv=comp_eval_merge_evals.tsv
plaseval-report utils join-measures "$merge_comp_tsv" "$merge_eval_tsv" "$join_tsv"

Joining the two set of measures is relevant when remove_samples option is set to fails because filtering on the joined TSV ensures all the samples have a comp and a eval evaluation.

Metadata

Release files for plaseval-report 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for plaseval-report 0.3.0
File Size Uploaded Provenance Actions
plaseval_report-0.3.0.tar.gz 62.1 kB Without provenance Details for plaseval_report-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for plaseval-report 0.3.0
File Interpreter ABI Platform Actions
plaseval_report-0.3.0-py3-none-any.whl Python 3 none any Details for plaseval_report-0.3.0-py3-none-any.whl

Total release size: 185.8 kB

Release files / plaseval_report-0.3.0.tar.gz

Download URL plaseval_report-0.3.0.tar.gz
Size 62.1 kB
Tags Source
SHA-256 checksum
How to use checksums
15fd3d6aaf09831adedbc024a79b9180e09fd4632e852da4e53902677fc56de3
BLAKE2b-256 checksum
How to use checksums
eb68ff87eccfc262f415d6954caa15ca6ce82b0a2a78e6416afbf615c3d717e3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.9.30 {"installer":{"name":"uv","version":"0.9.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"12","id":"bookworm","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / plaseval_report-0.3.0-py3-none-any.whl

Download URL plaseval_report-0.3.0-py3-none-any.whl
Size 123.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5d80fd2c3e3dd55d98a8bc07b2ac688ab4dfa500a6232cf896a5086491b9b60c
BLAKE2b-256 checksum
How to use checksums
a6e46565ba8e63ddaa6be724130af0236771e8c7d995c2530c397be299bb1832
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.9.30 {"installer":{"name":"uv","version":"0.9.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"12","id":"bookworm","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for plaseval-report 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page