vepyr

vepyr (/ˈvaɪpər/) — VEP Yielding Performant Results — is a blazing-fast Rust reimplementation of Ensembl's Variant Effect Predictor, exposed as a Python library and a vepyr command. It builds and uses Ensembl VEP caches locally, annotates VCF input through a native DataFusion engine, and returns results as a polars.LazyFrame or a VCF with CSQ in the INFO column.

📚 Documentation

https://biodatageeks.org/vepyr/

Install

pip install vepyr

or from bioconda (linux-64, osx-64, osx-arm64):

conda install -c conda-forge -c bioconda vepyr

Either installs both the Python package and a vepyr executable:

vepyr annotate \
    -i input.vcf.gz \
    -o annotated.vcf.gz \
    --dir_cache ~/vepyr_cache/116_GRCh38_ensembl \
    --fasta GRCh38.fa \
    --fork 8

See Developers for building from source and running the test suite.

Quick sanity check with chr22

Check Ensembl VEP 116.0 parity on the 50,861 normalized HG002 GRCh38 chr22 records using the existing run_comparison.py CLI. Docker provides vepyr 0.9.2, the Hugging Face client, Git LFS and the VCF tools.

From the repository root, download the merged chr22 cache and run its md5 check:

docker build -t vepyr-e2e e2e-testing/docker
mkdir -p e2e-testing/results/sanity-chr22
docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp \
  -v "$PWD:/repo" \
  -v "$PWD/e2e-testing/results/sanity-chr22:/work" \
  vepyr-e2e bash -ec '
    python e2e-testing/scripts/download_chr22.py --work-dir /work --profiles merged
    DATA_VEPYR_DIR=/work python e2e-testing/scripts/run_comparison.py \
      --release 116 --profile merged --chroms 22 \
      --vcf /work/input/input_chr22.vcf.gz --fasta /work/input/chr22.fa.gz \
      --output-dir /work --comparison-mode md5 --md5-mode both --bgzf --no-normalize
  '

The download helper fetches only chr22 shards, manifests and the cache-root policy files from Hugging Face at pinned revisions, plus the BGZF golden VCF and FASTA through Git LFS. The comparison CLI prints the strict and canonical record-body md5 verdicts and exits nonzero on a mismatch. Downloads and results persist in the mounted folder. The input is already normalized with bcftools norm -m -both.

See Quick sanity check with chr22 for the command that runs all 10 profiles, offline reuse and golden-data provenance. The three caches total ~1.5 GB and the ten BGZF goldens total ~178 MB; allow about 6 GB of disk space for the complete run and Docker image.

Continuous integration

Four tiers gate every change; tiers 2-4 test the wheel built from the code under review.

Admins run tiers 2-3 for a PR by commenting exactly /parity on it, or with gh workflow run parity-tests.yml -f pr=<n> (or Actions -> Parity tests). They report parity/vep-parity and parity/integration on the PR head; a new push needs a new run. A /parity comment gets a rocket reaction and a reply with the run link, then a final reply with both statuses; a non-admin's /parity is refused with a reply. The comment route always runs master's copy of the workflow, so it works for fork PRs and a PR cannot alter its own gate. Releases (publish_to_pypi.yml) run all tiers and publish only if all pass; dry_run=true runs everything without tagging or publishing.

The parity/* statuses are posted with GITHUB_TOKEN, so branch protection can only pin them to the GitHub Actions app. A collaborator with write access could post the same contexts from a workflow on their own branch. Setting Settings -> Actions -> General -> Workflow permissions to read-only narrows this (such a workflow must then request statuses: write explicitly); only a dedicated status-posting GitHub App, required by branch protection, closes it.

Whole-genome parity (/parity-wgs)

An admin comment that is exactly /parity-wgs (or gh workflow run parity-wgs.yml -f pr=<n>) runs the HG002 GRCh38 whole-genome md5 parity gate against the Ensembl VEP 116.0 golden VCFs for the ensembl, merged and refseq profiles on the self-hosted runner. It reports parity/wgs (and parity/wgs/<profile>) on the PR head. Runs run one at a time on the self-hosted runner, take about 1.5 hours (wheel build, then 20–30 minutes per profile), and are not a required check unless added to branch protection. On a fork PR, pin the head commit you reviewed: /parity-wgs <sha> (7+ hex characters, or -f sha=<sha>); the run is refused if the head has moved. PR code runs only inside a fresh, locked-down container with the staged data mounted read-only. Runner setup and the accepted residual risk are in ci/wgs-runner/README.md.

License

Apache-2.0