pycmplot

                    #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
                    |  MULTI-TRACK CIRCULAR AND LINEAR MANHATTAN PLOTTING  |
                    |                      in Python                       |
                    |                    Kevin Esoh, 2026                  |
                    |                    kesohku1@jh.edu                   |
                    #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#

Content

1. Overview
2. Key features
   • Column auto-detection
   • Density-aware sub-sampling
   • Trim insignificant variants for faster plotting
   • Genome build conversion (liftover)
   • Nearest-gene annotation for GWAS lead SNPs
   • Caching & warm resume
   • User-editable hits overlay
   • Per-locus highlight colours & custom legend
   • Multi-panel canvas
   • Browser-based hits overlay editor
3. Application
4. Tip
5. Installation
   • From PyPI
   • From GitHub
   • Python virtual environment
6. Test the installation
7. Dependencies
8. Command-line usage
   • Linear Manhattan (default)
   • Circular Manhattan
   • Key options
9. Python API
10. Contributing
11. Contributors

Overview

This package will take any number of per SNP/variant summary statistics, be it GWAS, selection scans (e.g. iHS, EHH, FST), etc and generate Manhattan plots. If given a single file, a single one-track Manhattan plot will be generated. Multiple files will result in the generation of a multi-track stacked Manhattan plot.

In the process, the package will generate a hits summary table for variants with p-value (or whatever statistic for significance is used) below the user-specified significance threshold. This hits summary table will contain annotated gene names, in addition to other annotations, that would then be used to annotate the plots.

Importantly, the package allows for conversion of hg19 genomic coordinates to hg38 coordinates. This ensures that summary stats obtained using different imputation panels, for instance, can be processed in the same run. That is, users can simply concatenate multiple summary stats files together, such as those for the same trait but analysed using different imputation panels. Users only need to add a new column specifying the genome build (hg19 or hg38) of the variants. Then the --build_column option of the package should be used to indicate the column and then the package will liftover all postions in hg19 to hg38 ensuring that hits table generation and plotting are done with one unified corrdinate system.

Key features

Column auto-detection

A key functionality of the package is its ability to auto-detect certain columns if ommited on the command-line or python API:

• Chromosome column: -chr, --chrom_column or ommited
• Basepair position column: -pos, --pos_column or ommited
• SNP or Marker ID column: -snp, --snp_column or ommited
• P-value (or whatever value) column: -p, --pval_column or ommited
• Build version column: -b, --build_column or ommited

Candidate names for each of the columns is shown below.

# Resolve column names
chr_candidates = [chrom, 'CHR', 'CHROM', 'Chromosome', '#CHROM', '#CHR', 'Chrom', 'chrom', 'chr', 'chromosome', '#chr', '#chrom']
pos_candidates = [pos, 'BP', 'POS', 'bp', 'pos', 'Basepair']
snp_candidates = [snp, 'SNP', 'RSID', 'rsID', 'MarkerName', 'MarkerID', 'Predictor', 'Marker', 'SNPID', 'ID']
pvl_candidates = [pcol, 'P', 'P-value', 'Wald_P', 'pvalue', 'p_val', 'pval']
bld_candidates = [build, 'BUILD', 'Genome', 'Genome_Build', 'Genome-build']

NB: Upper and lower cases of the candidates are also considered, making each candidate expanded 3 times.

Density-aware sub-sampling

Another key feature is density-aware sub-sampling for Manhattan-style scatter plots. This was inspired by gwaslab's default behaviour (https://cloufield.github.io/gwaslab/).

Every variant whose "interestingness" signal is at or above keep_threshold is preserved (so peaks, suggestive hits, genome-wide-significant hits, and extreme selection-scan values are kept verbatim). It uniformly sub-samples the dense bulk below the threshold down to at most max_below rows in total. For a 10 M-variant scan with the defaults below, this typically cuts the plotted point count from 10 M to ~200 K + a few hundred peaks — visually indistinguishable above the suggestive band, but two orders of magnitude faster to render.

Trim insignificant variants for faster plotting

An optional parameter -tp, --trim_pval is provided to increase speed even further. Set with a value to exclude variants with p-value above a certain threshold, e.g. 0.01 (1e-2) or 0.001 (1e-3). Performed on top of the default auto-thin feature above, it siginificant increases speed and reduces peak memory usage. See benchmark figure (manuscript in preparation).

Genome build conversion (liftover)

Conversion of a both hg18 and hg19 positions to their hg38 equivalent is included through pyliftover.LiftOver.

This means you can concatenate multiple summary stats into one file and include a BUILD column to specify the genome build of each position ('hg18', 'hg19', or 'hg38') and all 'hg18' and 'hg19' positions will be converted to 'hg38' so that all positions are plotted using one coordinate system. If only 'hg18' or 'hg19' positions are present, no liftover be necessary. Hence, liftover is only performed in cases of mixed genome builds.

Nearest-gene annotation for GWAS lead SNPs

The package bundles GFF3 files in hg19 and hg38 coordinates processed to reduce size for gene annotation. Also included are UCSC chain files for coordinate conversion (liftover).

• chain_hg19_hg38 -- UCSC LiftOver chain file for hg19 to hg38 conversion. Resolved from PYCMPLOT_CHAIN_HG19_HG38 or the bundled hg19ToHg38.over.chain.gz.
• chain_hg18_hg38 -- UCSC LiftOver chain file for hg18 to hg38 conversion. Resolved from PYCMPLOT_CHAIN_HG18_HG38 or the bundled hg18ToHg38.over.chain.gz. Only required when any input summary statistics file carries a hg18 build label.
• geneinfo_hg38 -- Ensembl gene-info TSV for GRCh38, used for nearest-gene annotation. Resolved from PYCMPLOT_GENEINFO_HG38 or the bundled Homo_sapiens.GRCh38.geneinfo.tsv.gz.
• geneinfo_hg19 -- Ensembl gene-info TSV for GRCh37, used when input data carry a hg19 build label. Resolved from PYCMPLOT_GENEINFO_HG19 or the bundled Homo_sapiens.GRCh37.geneinfo.tsv.gz.

Caching & warm resume

Loading is the expensive step (I/O + trim + liftover + lead extraction), so pycmplot ships a per-track cache keyed on SHA-256(raw_file_sha256 + version + Stage-1 params). Warm re-runs of the same (files, parameters) combination complete in milliseconds; changing any parameter transparently invalidates only the affected tracks and regenerates them. Enable with --cache (CLI) or cache=True (Python API); use --clear_cache to wipe the tree.

pycmplot --sum_stats hb.tsv,mcv.tsv --labels Hb,MCV \
         --logp --highlight --cache --cache_dir ./.pycmplot

User-editable hits overlay

When caching is on, the auto-generated hits table is written to a group-scoped TSV at <cache_dir>/annotations/hits.<group_key>.tsv that you're expected to hand-edit. Rows added or changed there feed straight back into the next plot — no re-plumbing required. Each row carries source (auto vs user), highlight_color, and category columns; user edits are inherited across cache regenerations by (CHR, POS) lookup, so re-running with a new parameter never loses your annotations.

Per-locus highlight colours & custom legend

Set the highlight_color column on any row of the hits overlay to a matplotlib-parseable colour (name, #rrggbb, or an RGB tuple) to give that locus its own highlight colour; leave auto to fall back to the plot-wide --highlight_color. Set the category column (e.g. novel, replicated, MHC) to group loci in a "Highlighted Categories" legend that both the linear and circular plotters render automatically. If nothing is edited, no legend is added — the pre-feature layout is preserved.

Multi-panel canvas

Place multiple groups of sumstats on the same figure by passing an explicit matplotlib Axes or SubFigure via ax= to the plotter. Cache files and hits overlays are group-scoped, so two panels with different sumstats never clobber each other's artefacts.

Browser-based hits overlay editor

For visual editing of the hits overlay (colour picker, category autocomplete, live plot preview), install the optional editor extra and launch a local Streamlit app against your cache directory:

pip install "pycmplot[editor]"
pycmplot edit --cache_dir ./.pycmplot_cache

For cluster sessions where no local browser is available (SSH-only compute nodes, secured HPC), a terminal-UI backend built on Textual renders a spreadsheet directly in the terminal — arrow keys to navigate, F2/Enter to edit, Ctrl+S to save. No port forwarding required:

pip install "pycmplot[editor-tui]"
pycmplot edit --cache_dir ./.pycmplot_cache --tui

Both backends write through the same write_hits_overlay path the loader uses, so atomic-write and inheritance-across-regeneration guarantees hold identically to a text-editor workflow. Extras are independent — install just the one you need. Headless / CI pipelines that install plain pycmplot pay neither install cost.

For overlays with hundreds of loci, both TUI backends support filter + multi-select + batch-edit — press / to filter with a pandas query() expression, Space to select rows, Ctrl+A to select every filtered row, Ctrl+E to apply one colour/category value to the whole selection.

For pipelines and reproducible analyses, a batch pycmplot hits CLI exposes the same filter grammar without any GUI:

# Preview what would change
pycmplot hits set --cache_dir ./.pycmplot_cache --where 'P < 5e-8' \
    --color '#00cc44' --category "genome-wide" --dry-run

# Apply
pycmplot hits set --cache_dir ./.pycmplot_cache --where 'P < 5e-8' \
    --color '#00cc44' --category "genome-wide"

# Inspect (piping-friendly with --tsv)
pycmplot hits list --cache_dir ./.pycmplot_cache --where 'category == "novel"'

The CLI needs no extras — perfect for CI, Makefiles, and headless compute jobs.

Application

A potential useful application is comparative visualization of results from multiple imputation panels, multiple populations, or multiple traits to observe shared genetic architecture.

Tip

For signed non-pvalue statistics such as iHS, use --signif_threshold and --suggest_threshold to specify lines to draw for iHS cutoff above and below zero, that is in the positive and negative axes.

Read more in the package documentation page: https://pycmplot.readthedocs.io/en/latest/

Installation

From PyPI

pip install pycmplot

From GitHub

git clone https://github.com/esohkevin/pycmplot.git

# or with most recent updates from development branch
# git clone -b dev https://github.com/esohkevin/pycmplot.git

cd pycmplot

pip install -e .

# or

pip install -e . --break-system-packages

Python virtual environment

Use python virtual environment if local installation is not possible

python -m venv ~/envs/pycmplot
source ~/envs/pycmplot/bin/activate

pip install --upgrade pip setuptools wheel
pip install pycmplot

Test the installation

pycmplot -h

Dependencies

Command-line usage

Linear Manhattan (default)

pycmplot \
  --sum_stats HbF.tsv.gz,MCV.txt.gz,MCH.tsv.gz \
  --labels HbF,MCV,MCH \
  --logp \
  --signif_line \
  --highlight \
  --annotate GENE \
  --output_dir ./results \
  --output_format png \
  --dpi 300

Circular Manhattan

pycmplot \
  --sum_stats HbF.tsv.gz,MCV.tsv.gz \
  --labels HbF,MCV \
  --mode cm \
  --trim_pval 0.01 \
  --logp \
  --signif_threshold \
  --plot_title "RBC Traits" \
  --output_dir ./results

Key options

QQ note (0.4.0+): the loader no longer materialises the full sorted p-value array by default. -qq/--qq_plot sets compute_pvals=True automatically on the CLI; Python-API callers must pass it explicitly to feed bundle['pvals'] into a QQ plotter.

Run pycmplot -h for the full option list.

Python API

A demonstration of how to use the python API is provided in this notebook: https://github.com/esohkevin/pycmplot/blob/main/pycmplot_python_api.ipynb

For an end-to-end walkthrough of every feature (caching, hits overlay, per-locus colours & categories, multi-panel canvas, mixed-build liftover, QQ opt-in), see the Tutorial page in the docs.

Contributing

See how to contribute here https://github.com/esohkevin/pycmplot?tab=contributing-ov-file

Contributors

• Kevin Esoh