regression-testing-utils 0.1.2

Exported Console Scripts

The following exported console scripts are available:

• regression-testing-runner

Usage

Install the package:

pip install regression-testing-utils

Prepare your regression configuration file.

Contents of example.yaml:

---
# Acceptable values for file_format are csv or tsv.
# Meaning, this software can only compare files
# where both files are tab-delimited (tsv) or both
# files are comma-separated (csv).
file_format: tsv

# The line number that the column headers occur on.
# Column headers are necessary for reporting which
# which columns the differences were detected in.
header_line_number: 1

# The line number that the records start on.
# Beginning on this line, records will be compared
# between the two files.
records_start_line_number: 2

# The maximum number of columns.  If specified, this
# will inform the software up to which line should
# be consider. Otherwise, all lines beginning at-
# and including records_start_line_number (above).
max_column_count:

# If the following is True, then the evaluator software will ignore
# the columns listed in the "ignore_columns_str".
ignore_columns: false

# The following will list the columns to be ignored by the evaluator
# software.  Note: if the evaluator software should ignore columns
# specified in this list, then the "ignore_columns" boolean above
# should be set to true.
ignore_columns_str: null

workflow:
  # If running in test mode, the actual commands will not be executed.
  test_mode: false
  stages:
    # Each workflow stage is optional.
    # The stage_name and commands are all required.
    # The desc (description) is optional.
    - stage_name: evaluation
      desc: "Evaluate the results of the software being tested/evaluated"
      commands:
        - evaluator:
          # Set-up for testing:
          # cp regression-testing-utils/examples/example_actual.tsv /tmp/demo-regression-testing-utils/actual.tsv
          # cp regression-testing-utils/examples/example_expected.tsv /tmp/demo-regression-testing-utils/expected.tsv
          args:
            # file1 is required.  It must indicate the first file that will be compared against file2.
            file1: /tmp/demo-regression-testing-utils/actual.tsv
            # file2 is required.  It must indicate the first file that will be compared against file1.
            file2: /tmp/demo-regression-testing-utils/expected.tsv
            # file_format is optional.  It indicates what the file formats are.
            file_format: tsv
            # outfile is required.  It indicates where the comparison report will be written to.
            outfile: /tmp/demo-regression-testing-utils/comparison_report.txt
# If the following is set to true,
# the software will use the Rich Table
# API for displaying which stage is
# being executed.
# If set to false, will use a simple
# banner to display the same information.
use_rich_table: true

Contents of actual.tsv

cat -n /tmp/demo-regression-testing-utils/actual.tsv
1  #CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO
2  1       12345   rs567   A       G       50      PASS    DP=30;AF=0.2;AN=1000;CSQ=missense_variant|HIGH|GeneA|ENSG00000112345|transcriptA|ENST00000234567|protein_coding|1/10|c.123C>T|p.Arg41Trp|123/1000|ensembl
3  2       56789   rs555   T       C       44      PASS    DP=25;AF=0.1;AN=1200;CSQ=synonymous_variant|MEDIUM|GeneB|ENSG00000123456|transcriptB|ENST00000345678|protein_coding|5/20|c.567A>G|p.Ala189Ala|567/1200|ensembl
4  3       98765   rs123   G       T       61      PASS    DP=40;AF=0.3;AN=800;CSQ=splice_acceptor_variant|HIGH|GeneC|ENSG00000134567|transcriptC|ENST00000456789|protein_coding|2/15|c.987+1G>T|p.?|987/800|ensembl
5  1       34567   rs456   C       A       55      FAIL    DP=35;AF=0.15;AN=900;CSQ=frameshift_variant|HIGH|GeneX|ENSG00000145678|transcriptX|ENST00000567890|protein_coding|8/25|c.345_346insT|p.Leu116Phefs*12|345/900|ensembl

Contents of expected.tsv

cat -n /tmp/demo-regression-testing-utils/expected.tsv
1  #CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO
2  1       12345   rs567   A       G       50      PASS    DP=30;AF=0.2;AN=1000;CSQ=missense_variant|HIGH|GeneA|ENSG00000112345|transcriptA|ENST00000234567|protein_coding|1/10|c.123C>T|p.Arg41Trp|123/1000|ensembl
3  2       56789   rs890   T       C       44      PASS    DP=25;AF=0.1;AN=1200;CSQ=synonymous_variant|MEDIUM|GeneB|ENSG00000123456|transcriptB|ENST00000345678|protein_coding|5/20|c.567A>G|p.Ala189Ala|567/1200|ensembl
4  3       98765   rs123   G       T       60      PASS    DP=40;AF=0.3;AN=800;CSQ=splice_acceptor_variant|HIGH|GeneC|ENSG00000134567|transcriptC|ENST00000456789|protein_coding|2/15|c.987+1G>T|p.?|987/800|ensembl
5  1       34567   rs456   C       A       55      PASS    DP=35;AF=0.15;AN=900;CSQ=frameshift_variant|HIGH|GeneX|ENSG00000145678|transcriptX|ENST00000567890|protein_coding|8/25|c.345_346insT|p.Leu116Phefs*12|345/900|ensembl

Execution of exported console script:

regression-testing-runner --config_file /tmp/demo-regression-testing-utils/simple.yaml --outdir /tmp/demo-regression-testing-utils
--logfile was not specified and therefore was set to '/tmp/demo-regression-testing-utils/runner.log'
Setting file format to 'tsv'
3 differences found

Contents of comparison report:

cat -n /tmp/demo-regression-testing-utils/comparison_report.txt
 1  ## method-created: /tmp/regression-testing-utils/venv/lib/python3.10/site-packages/regression_testing_utils/evaluator.py
 2  ## date-created: 2025-02-15-121022
 3  ## created-by: sundaram
 4  ## file 1: /tmp/demo-regression-testing-utils/actual.tsv
 5  ## file 2: /tmp/demo-regression-testing-utils/expected.tsv
 6  ## file-format: tsv
 7  ## logfile: /tmp/demo-regression-testing-utils/runner.log
 8  ## Number of differences: 3
 9  Line #  Column Name     Column #        Column Letter   Value in File 1 Value in File 2
10  1       rs567   3       C       rs555   rs890
11  2       50      6       F       61      60
12  3       PASS    7       G       FAIL    PASS