Skip to main content

vepyr

vepyr (/ˈvaɪpər/) — VEP Yielding Performant Results — a blazing-fast Rust reimplementation of Ensembl's Variant Effect Predictor.

logo.png

Setup with uv

  1. Install uv.
curl -LsSf https://astral.sh/uv/install.sh | sh
  1. Clone the repository and enter it.
git clone git@github.com:biodatageeks/vepyr.git
cd vepyr
  1. Sync dependencies and build the package in place.
RUSTFLAGS="-C target-cpu=native" uv sync --reinstall-package vepyr
  1. Run Python commands inside the managed environment.
uv run python -c "import vepyr; print(vepyr.__all__)"
  1. Run the test suite.
uv run pytest

Quick start

The repository ships with small test fixtures so you can verify the full pipeline — build, annotate with indexed Parquet, and write VCF output — without downloading any external data.

1. Build a cache from a local Ensembl VEP cache directory

tests/data/ensembl_cache contains a tiny slice of the Ensembl VEP 115 offline cache (chr22). Convert it to the default indexed Parquet cache:

import vepyr

results = vepyr.build_cache(
    release=115,
    cache_dir="/tmp/vepyr_cache",
    cache_type="ensembl",
    local_cache="tests/data/ensembl_cache",  # skip download
)
for path, rows in results:
    print(f"{path}: {rows:,} rows")

To rebuild only one raw entity while preserving the same release/source contract, use build_cache_entity(). Supported raw entities are variation, transcript, exon, translation, regulatory, and motif. The raw translation entity writes both translation_core and translation_sift. For example, a release-116 motif rebuild is:

results = vepyr.build_cache_entity(
    release=116,
    cache_dir="/tmp/vepyr_cache",
    entity="motif",
    cache_type="merged",
    local_cache="/data/ensembl-vep/homo_sapiens_merged/116_GRCh38",
    overwrite=True,
)

For an existing converted cache, e2e-testing/scripts/rebuild_cache_entity.py wraps this API in an all-shard verification, backup, transactional swap, and rollback workflow:

uv run python e2e-testing/scripts/rebuild_cache_entity.py \
    --release 116 --cache-type merged --entity translation --run

This vepyr release supports exactly cache 115 with VEP 115.2 semantics and cache 116 with VEP 116.0 semantics. build_cache() embeds bio.vep.cache_version in every generated Parquet shard. Annotation requires that metadata and validates it lazily per contig across every participating entity; metadata-less, mixed, malformed, or unsupported caches are rejected. Directory names and sidecar files are never used as annotation-cache identity. The optional expected_cache_version="115" (or "116") argument is a strict assertion, not an override.

2a. Annotate variants

A small 5-variant VCF for chr22 ships with the cache fixture:

import vepyr

cache_dir = "/tmp/vepyr_cache/115_GRCh38_ensembl"

lf = vepyr.annotate(
    vcf="tests/data/ensembl_cache/sample.vcf",
    cache_dir=cache_dir,
    check_existing=True,
    af=True,
    af_gnomadg=True,
    max_af=True,
)

df = lf.collect()
print(df.select("chrom", "start", "ref", "alt", "most_severe_consequence").head())

workers controls how many within-contig annotation pipelines run concurrently. workers=1 is the serial path; workers > 1 requires a tabix-indexed (bgzip + .tbi) input VCF.

df = vepyr.annotate(
    "input.vcf.gz",
    cache_dir,
    workers=4,
).collect()

build_cache() writes variation as chrN_warm.parquet and chrN_cold.parquet files, plus cold-position and variant-bloom indexes. Re-running build_cache() is idempotent by default; pass overwrite=True to rebuild existing cache outputs.

out = vepyr.annotate(
    "input.vcf.gz",
    cache_dir,
    workers=8,
    output_vcf="annotated.vcf",
)

2b. Write annotated VCF output

Instead of a LazyFrame, write results directly to a VCF file with CSQ in the INFO column — use .vcf.gz for bgzf compression or .vcf for plain text:

out_path = vepyr.annotate(
    vcf="tests/data/ensembl_cache/sample.vcf",
    cache_dir=cache_dir,
    check_existing=True,
    af=True,
    af_gnomadg=True,
    max_af=True,
    output_vcf="/tmp/annotated.vcf",  # or .vcf.gz for bgzf
)
print(f"Wrote annotated VCF to {out_path}")

3. Full --everything annotation (golden test data)

tests/data/golden has a pre-built chr1 cache, a 100-variant VCF, and a matching reference FASTA. Run a full --everything annotation:

import vepyr

lf = vepyr.annotate(
    vcf="tests/data/golden/input.vcf.gz",
    cache_dir="tests/data/golden/cache",
    everything=True,
    reference_fasta="tests/data/golden/reference.fa",
)

df = lf.collect()
print(f"{df.height} variants × {df.width} columns")
print(df.select("chrom", "start", "ref", "alt",
                "most_severe_consequence", "SYMBOL", "IMPACT").head(5))

Documentation

Build and serve the docs locally:

uv sync --extra docs
uv run mkdocs serve

Then open http://127.0.0.1:8000. Docs are auto-deployed to GitHub Pages on each tag push.

One-liner smoke test

Exercises cache build, indexed Parquet annotation, and VCF output:

uv run python -c "
import vepyr, tempfile, os
with tempfile.TemporaryDirectory() as d:
    r = vepyr.build_cache(115, d, cache_type='ensembl', local_cache='tests/data/ensembl_cache', show_progress=False)
    cache = os.path.join(d, '115_GRCh38_ensembl')
    print(f'build_cache : {len(r)} parquet files, {sum(n for _,n in r):,} rows')
    vcf = 'tests/data/ensembl_cache/sample.vcf'
    df1 = vepyr.annotate(vcf, cache, check_existing=True, af=True, max_af=True).collect()
    print(f'indexed     : {df1.height} variants × {df1.width} columns')
    out = os.path.join(d, 'annotated.vcf')
    vepyr.annotate(vcf, cache, check_existing=True, af=True, max_af=True, output_vcf=out, show_progress=False)
    print(f'vcf output  : {os.path.getsize(out):,} bytes')
    assert os.path.getsize(out) > 0, 'empty VCF'
lf = vepyr.annotate('tests/data/golden/input.vcf.gz', 'tests/data/golden/cache', everything=True, reference_fasta='tests/data/golden/reference.fa')
df = lf.collect()
print(f'everything  : {df.height} variants × {df.width} columns')
assert df.height > 0 and df.width > 80, 'smoke test failed'
print('smoke test passed')
"
Source Added fields Count
VCF CSQ fixed base fields Allele, Consequence, IMPACT, SYMBOL, Gene, etc. 18
--everything --hgvs flag-derived fields, de-duplicated against VCF base includes frequency, MANE, UniProt, HGVS offset, regulatory, etc. 59
VEP option-set implication: frequency/pubmed flags enable check_existing CLIN_SIG, SOMATIC, PHENO 3
--merged REFSEQ_MATCH, SOURCE, REFSEQ_OFFSET 3
--flag_pick_allele_gene PICK 1
BAM-edited cache auto-enables --use_transcript_ref + bam_edited GIVEN_REF, USED_REF, BAM_EDIT 3
Total 87
# Field Breakdown bucket
1 Allele VCF CSQ fixed base
2 Consequence VCF CSQ fixed base
3 IMPACT VCF CSQ fixed base
4 SYMBOL VCF CSQ fixed base
5 Gene VCF CSQ fixed base
6 Feature_type VCF CSQ fixed base
7 Feature VCF CSQ fixed base
8 BIOTYPE VCF CSQ fixed base
9 EXON VCF CSQ fixed base
10 INTRON VCF CSQ fixed base
11 HGVSc VCF CSQ fixed base
12 HGVSp VCF CSQ fixed base
13 cDNA_position VCF CSQ fixed base
14 CDS_position VCF CSQ fixed base
15 Protein_position VCF CSQ fixed base
16 Amino_acids VCF CSQ fixed base
17 Codons VCF CSQ fixed base
18 Existing_variation VCF CSQ fixed base
19 DISTANCE Default / --everything flag-derived
20 STRAND Default / --everything flag-derived
21 FLAGS Default / --everything flag-derived
22 PICK --flag_pick_allele_gene
23 VARIANT_CLASS --everything
24 SYMBOL_SOURCE --everything
25 HGNC_ID --everything
26 CANONICAL --everything
27 MANE --everything
28 MANE_SELECT --everything
29 MANE_PLUS_CLINICAL --everything
30 TSL --everything
31 APPRIS --everything
32 CCDS --everything
33 ENSP --everything
34 SWISSPROT --everything
35 TREMBL --everything
36 UNIPARC --everything
37 UNIPROT_ISOFORM --everything
38 REFSEQ_MATCH --merged
39 SOURCE --merged
40 REFSEQ_OFFSET --merged
41 GIVEN_REF BAM-edited cache / --use_transcript_ref
42 USED_REF BAM-edited cache / --use_transcript_ref
43 BAM_EDIT BAM-edited cache
44 GENE_PHENO --everything
45 SIFT --everything
46 PolyPhen --everything
47 DOMAINS --everything
48 miRNA --everything
49 HGVS_OFFSET --everything --hgvs
50 AF --everything
51 AFR_AF --everything
52 AMR_AF --everything
53 EAS_AF --everything
54 EUR_AF --everything
55 SAS_AF --everything
56 gnomADe_AF --everything
57 gnomADe_AFR_AF --everything
58 gnomADe_AMR_AF --everything
59 gnomADe_ASJ_AF --everything
60 gnomADe_EAS_AF --everything
61 gnomADe_FIN_AF --everything
62 gnomADe_MID_AF --everything
63 gnomADe_NFE_AF --everything
64 gnomADe_REMAINING_AF --everything
65 gnomADe_SAS_AF --everything
66 gnomADg_AF --everything
67 gnomADg_AFR_AF --everything
68 gnomADg_AMI_AF --everything
69 gnomADg_AMR_AF --everything
70 gnomADg_ASJ_AF --everything
71 gnomADg_EAS_AF --everything
72 gnomADg_FIN_AF --everything
73 gnomADg_MID_AF --everything
74 gnomADg_NFE_AF --everything
75 gnomADg_REMAINING_AF --everything
76 gnomADg_SAS_AF --everything
77 MAX_AF --everything
78 MAX_AF_POPS --everything
79 CLIN_SIG implied check_existing
80 SOMATIC implied check_existing
81 PHENO implied check_existing
82 PUBMED --everything
83 MOTIF_NAME --everything
84 MOTIF_POS --everything
85 HIGH_INF_POS --everything
86 MOTIF_SCORE_CHANGE --everything
87 TRANSCRIPTION_FACTORS --everything

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

vepyr-0.3.0.tar.gz (3.2 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

vepyr-0.3.0-cp310-abi3-win_amd64.whl (51.6 MB view details)

Uploaded CPython 3.10+Windows x86-64

vepyr-0.3.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (58.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

vepyr-0.3.0-cp310-abi3-macosx_11_0_arm64.whl (52.4 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

vepyr-0.3.0-cp310-abi3-macosx_10_12_x86_64.whl (55.2 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file vepyr-0.3.0.tar.gz.

File metadata

  • Download URL: vepyr-0.3.0.tar.gz
  • Upload date:
  • Size: 3.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for vepyr-0.3.0.tar.gz
Algorithm Hash digest
SHA256 f278c8763ad5b5eba1894377eacef1684d596f3c158d95fe2560e9ac58de51a0
MD5 074a171948adc38b9bf623bfa74d6e64
BLAKE2b-256 f9e811208a21f912fc3733ca78e138261c54b7b47fda8d441b75b6ea167f6fdf

See more details on using hashes here.

Provenance

The following attestation bundles were made for vepyr-0.3.0.tar.gz:

Publisher: publish_to_pypi.yml on biodatageeks/vepyr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vepyr-0.3.0-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: vepyr-0.3.0-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 51.6 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for vepyr-0.3.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 79016c82f90bba810627fc144431bde13d5bfaa8765cb8f3e9b6f57561bec16b
MD5 c32f8889ebe54fb24ee2a784ed2b22d3
BLAKE2b-256 2e7ae81ad1020b58be11601c7b8ce709520d83d2456a2dea0768af704573d4d2

See more details on using hashes here.

Provenance

The following attestation bundles were made for vepyr-0.3.0-cp310-abi3-win_amd64.whl:

Publisher: publish_to_pypi.yml on biodatageeks/vepyr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vepyr-0.3.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for vepyr-0.3.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7a246fd670e82df8e0f79fbfecdc946f002036d75e3c75d542c995656764fcc4
MD5 9769ec57387c050dbfa510715a764405
BLAKE2b-256 b7721575d05381bb94be359fed5cefb7cb34ebddcac363293da196971696a36e

See more details on using hashes here.

Provenance

The following attestation bundles were made for vepyr-0.3.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish_to_pypi.yml on biodatageeks/vepyr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vepyr-0.3.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

  • Download URL: vepyr-0.3.0-cp310-abi3-macosx_11_0_arm64.whl
  • Upload date:
  • Size: 52.4 MB
  • Tags: CPython 3.10+, macOS 11.0+ ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for vepyr-0.3.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e4643249442032ef057546bfb542625b1c08c5c4dab576f2b8c88d0e41cee591
MD5 d652fd60fc919a767b445279dd19682e
BLAKE2b-256 bef4261e364c62c80c179bf10081103deab9d934f5ebd97f5635a7c930473f28

See more details on using hashes here.

Provenance

The following attestation bundles were made for vepyr-0.3.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: publish_to_pypi.yml on biodatageeks/vepyr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vepyr-0.3.0-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for vepyr-0.3.0-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7b876a5c965c9df0ce0001858ee42af366fa8b1984610084d7bfca51347ce1e3
MD5 cc2dc1a90b4139c89564b149e5f26d6b
BLAKE2b-256 5e73ad0570f762c7054e80036d8df2a45d53682726c5d8c97d4d900d3c4093cc

See more details on using hashes here.

Provenance

The following attestation bundles were made for vepyr-0.3.0-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: publish_to_pypi.yml on biodatageeks/vepyr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.5.0

5 files

0.4.0

5 files

This release

0.3.0 This release

5 files

0.2.0

5 files

0.1.1

5 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page