multiqc-pivot

A MultiQC plugin that folds related samples into one General Statistics row per group.

Installation

pip install multiqc-pivot

Introduction

MultiQC gives every sample its own row. When one subject yields several samples that are measured by different methods, say a tumour and a normal, or two tissues and a paired-genotype check, the General Statistics table ends up with a block of half-empty rows per subject. MultiQC's own sample grouping only fills the group's row for the handful of modules that know how to merge their metrics.

This plugin runs after every module has reported and rebuilds the General Statistics table:

1. Rows for one group fold into a single row, and every folded column is prefixed by which method it came from.
2. The original rows stay beneath the group row, so they still can be viewed, unless a level asks for them to be dropped.
3. Rows for a level that do not belong in the table, such as per-library read QC, move out into their own table under General Statistics with whatever grouping they already had.
4. Hover text, color scales, formats and hidden-by-default state carry over from the module that produced each column.

Usage

Add a sample_pivot block to any MultiQC config, for example with --config my_config.yml.

So, with these sample and library filename prefixes for a patient:

101.subject
101.tissueA
101.tissueB
101.tissueB (filtered data, though)
101.tissueA.library.L1
101.tissueA.library.L2

The following configuration produces one row named 101 carrying Concordance, TissueA Median, TissueB Median, TissueB (filtered) % Aligned and so on, and moves any per-library rows into a separate table:

sample_pivot:
  group: '^(?P<group>[^. ]+)\.'
  levels:
    - match: '\.subject$'
    - match: '\.(?P<analyte>tissueA|tissueB)$'
      label: '{analyte}'
    - match: '\.(?P<analyte>tissueA|tissueB) \(filtered\)$'
      label: '{analyte} (filtered)'
    - match: '\.library\.'
      table: Library statistics
  label_order: [tissueA, tissueB, tissueB (filtered)]
  tables:
    Library Statistics:
      description: Per-library read QC; read pairs nest under their library.

And your report will look like:

YAML Configuration Reference

Dropping Labelled Rows

When the group row already says everything you need, set keep_rows: false on a labelled level. Its columns still fold onto the group row, renamed and titled as before, but the original rows no longer sit beneath it. They are removed from General Statistics only, so each module's own section still shows them.

sample_pivot:
  group: '^(?P<group>[^. ]+)\.'
  levels:
    - match: '\.subject$'
    - match: '\.(?P<analyte>tissueA|tissueB)$'
      label: '{analyte}'
      keep_rows: false

Limitations

1. Sample names are matched after MultiQC has cleaned them, so you must write patterns against the names you see in an un-pivoted report.
2. Only one level of nesting exists in a MultiQC table. Rows moved into a secondary table keep the nesting they already had; rows folded into a group row become its children, and cannot nest further.
3. Two rows in the same group that resolve to the same label collide. The first value is kept and a warning is logged, so make labels specific enough to tell such rows apart.

Development and Testing

See the contributing guide for more information.