UnifyVcf
Unifies the separate structural call files a sequencing pipeline produces into the single VCF Geneyx reads as an SV VCF.
The files it unifies are:
- SV – structural variants VCF
- CNV – copy number variants VCF
- Repeats – tandem repeat variants VCF
- ROH – Regions of Homozygosity (DRAGEN BED, or an already-converted VCF)
It does not only concatenate them. Each caller writes slightly different files, and Geneyx needs specific fields to read a record properly — so records are rewritten on the way out and the result is sorted and bgzipped. Run the entry point that matches the pipeline your VCFs came from.
This package is the single implementation behind both the customer-facing scripts in geneyx.analysis.api and the Geneyx primary pipeline, so both produce the same unified file.
Installation
pip install UnifyVcf
Python 3.9–3.13 on Linux, macOS or Windows, with no external tools required and
no dependencies outside the standard library. Sorting and BGZF compression are pure
Python; repeat filtering uses bedtools when it is installed and computes the same
intersection itself when it is not.
Wheels are published per platform and Python version because the implementation ships
compiled — see How this is distributed. If pip install
reports no matching distribution, your platform or Python version is outside the
published set; tell us which and we will add it.
Command line
dragen-unify-vcf -o OUTPUTPATH [-s SVPATH] [-c CNVPATH] [-r REPEATPATH] [-d ROH_BED_FILE]
ont-unify-vcf -o OUTPUTPATH [-s SVPATH] [-c CNVPATH] [-r REPEATPATH] [-modify]
pacbio-unify-vcf -o OUTPUTPATH [-s SVPATH] [-c CNVPATH] [-r FULLREPEATPATH] [-b REPEATLOCATIONSBEDFILEPATH]
| Argument | Meaning |
|---|---|
-o |
Path to the unified VCF, including its name. The output is compressed, so the name should end in .vcf, not .gz. |
-s |
Structural variants (SV) VCF. Gzipped or not. |
-c |
Copy number variants (CNV) VCF. Gzipped or not. |
-r |
Tandem repeats VCF. Gzipped or not. |
-d |
DRAGEN only. The ROH BED produced by DRAGEN when run with --vc-enable-roh. |
-modify |
ONT only. Marks repeat calls as repeats; add it when STRaglr is the repeat caller. Without it, ONT repeat records are left alone because they already carry SVTYPE=STR. |
-b |
PacBio only. A BED restricting which repeat loci are unified. Without -b the repeats are left out of the unified file entirely, because the full TRGT catalog is far wider than a report can interpret. |
A catalog of pathogenic hg38 STR loci ships with the package:
from UnifyVcf import default_repeat_catalog_path
pacbio-unify-vcf -o unified.vcf -s sv.vcf.gz \
-r sample.GRCh38.trgt.sorted.vcf.gz \
-b "$(python -c 'from UnifyVcf import default_repeat_catalog_path as p; print(p())')"
Python
from UnifyVcf import run
run(output_path="unified.vcf", sv_path="sv.vcf.gz", cnv_path="cnv.vcf.gz",
repeat_path="repeats.vcf.gz", roh_bed_path="roh.bed")
run keeps the signature the standalone UnifyVcf.py script published, so code that
did from UnifyVcf import run works unchanged once this package is installed.
A caller that already owns a sorting step — the Geneyx pipeline does — skips run and
composes the pieces instead, so it keeps its own sorting and indexing:
from UnifyVcf import RohFormat, UnifyOptions, read_struct_lines, unify_lines
options = UnifyOptions(
add_repeat_svtype=True, # append ;SVTYPE=REP to repeat records
roh_format=RohFormat.VCF, # ROH arrives already converted
drop_sv_filters=("InvBreakpoint",), # exclude SV records with this FILTER
)
unify_lines(read_struct_lines(paths, options), "unified.vcf", options) # unsorted
How the pieces fit
| Function | Does |
|---|---|
read_struct_lines(paths, options) |
Reads each input (gzipped or not), drops excluded SV records, and skips inputs that are empty or header-only. |
unify_lines(lines, output_path, options) |
Writes the unified, unsorted VCF. Returns None when there was nothing to write. |
sort_and_compress(vcf_path) |
Sorts by position and bgzips; returns the .gz path. |
filter_repeats_vcf(repeat_path, bed_path) |
Intersects a repeat VCF with a locus catalog, via bedtools or in Python. |
The first present input in SV → CNV → repeats order donates the header; the rest contribute records only.
Platform support
Everything runs on Linux, macOS and Windows. Two things used to be Linux-only and are no longer:
Sorting and compression. The output has to be sorted by position and compressed as
BGZF — a gzip variant written in <64KB blocks — because Geneyx indexes it with
tabix, and tabix cannot index plain gzip. Both are done in pure Python
(UnifyVcf.compression), so no sort or bgzip binary is needed. The ordering matches
what sort -k1,1 -k2,2n produced under the previous shell implementation, including
GNU sort's whole-line tie-break, so files keep the order Geneyx has always received.
Repeat filtering. bedtools intersect is used where bedtools is on PATH. Where it
is not, the same intersection is computed in Python: a record spans [POS-1, END),
taking END from INFO where present — repeat callers emit symbolic <STR> alleles
whose length is not the length of REF — and falling back to POS-1 + len(REF). The
engine that ran is logged. The CI runs the suite on both Linux and Windows, and the
Linux leg installs bedtools so the two engines are compared against each other rather
than one being skipped.
Force one or the other when you need to:
from UnifyVcf import BEDTOOLS, BUILTIN, filter_repeats_vcf
filter_repeats_vcf(repeats, catalog, engine=BUILTIN)
How this is distributed
The wheel ships compiled modules (.pyd on Windows, .so elsewhere), not .py
files. What it publishes instead is the interface:
UnifyVcf/*.pyitype stubs carrying every public signature and its description, generated from the source bytools/generate_stubs.py- the same docstrings inside the compiled modules, so
help(UnifyVcf.run)works py.typed, so mypy and IDEs read the stubs- this README
So autocomplete, type checking and help() behave normally; the function bodies are
just not there to read. __init__.py stays interpreted deliberately — it is only
re-exports, and it is the surface this README documents.
Consequences worth knowing:
- No sdist is published. An sdist would carry the sources, undoing the point.
pip download --no-binary :all:therefore has nothing to fetch. - One wheel per platform and Python version, built by cibuildwheel in CI, rather than the single pure-Python wheel a normal package gets.
- This is a deterrent, not a control. A compiled extension can still be analysed,
and string literals — log messages,
;SVTYPE=REPand the like — remain visible in the binary because the code has to use them at run time. It stops casual reading and copy-paste reuse; it is not a security boundary.
Working on the package
The sources live in this repository and run interpreted, so development and the test suite are unaffected:
pip install -e . # needs a C compiler; builds the extensions in place
python -m pytest tests
python tools/generate_stubs.py # after changing any public signature
python tools/generate_stubs.py --check # what CI enforces
tests/test_published_surface.py fails if a stub drifts from the source, so the
published description cannot silently stop matching the code.
Per-caller differences
These are options rather than separate implementations:
| DRAGEN | ONT | PacBio | Geneyx pipeline | |
|---|---|---|---|---|
;SVTYPE=REP on repeats |
yes | only with -modify |
yes | yes |
| ROH input | BED | – | – | VCF |
| Repeat filtering | – | – | bedtools against -b |
done as its own pipeline step |
SV FILTER exclusions |
none | none | none | InvBreakpoint (PacBio) |
The PacBio row and the pipeline row differ on InvBreakpoint deliberately: the Geneyx
pipeline drops those records from the unified file it analyses, and the customer-facing
script does not. Aligning them would change what customers upload, so it is a product
decision rather than a packaging one.
Known defect carried over
Pinned by a characterization test rather than fixed, so that packaging stayed a refactor. Worth its own change:
is_valid_line misses single-sample no-calls. The GT is read from the last
tab-separated cell, which on a single-sample record still carries the trailing newline,
so "./.\n" != "./." and the no-call is kept. Multi-sample records take the intended
path. The neighbouring gt_index == '0' compares an int to a str and is never true.
Two other defects in the original shell-based sorter are gone as a consequence of rewriting it for Windows, rather than by choice:
- It swallowed its own failures — a failed sort or
bgzipwas logged and execution continued, leaving an unsorted, uncompressed.vcfwhere the caller expected.vcf.gz. Failures now raise. - Its temp directory was
vcf-tempin the current working directory, so two runs in one directory deleted each other's temp files. Temporary files are now written beside the output and removed in afinally.
Tests
pytest tests
No bedtools, bgzip or network needed — the suite is file manipulation only. One test compares the two repeat-filtering engines and skips where bedtools is not installed.
Metadata
Release files for UnifyVcf 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
Total release size: 9.0 MB
Release files / unifyvcf-0.1.0-cp313-cp313-win_amd64.whl
| Download URL | unifyvcf-0.1.0-cp313-cp313-win_amd64.whl |
|---|---|
| Size | 189.4 kB |
| Tags | CPython 3.13 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
d333cb62d762f7d0faccfb83cd6ebe82f4d9176a8780dac704a5c432384183f1
|
|
BLAKE2b-256 checksum How to use checksums |
793d417ef3b0015a57a1218531ccc57c624670a31df9ba30679887940954951f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.13 Linux glibc 2.17+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
d5da3b53fd1b80883d3dcd10936cc68157b50005a3bc2efca18d66d6b2e45a0b
|
|
BLAKE2b-256 checksum How to use checksums |
97067aefd1b634ec0a0909fb4ad6faf33c099a9aea2efe2ccbc628190c152449
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp313-cp313-macosx_10_13_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp313-cp313-macosx_10_13_x86_64.whl |
|---|---|
| Size | 221.0 kB |
| Tags | CPython 3.13 macOS 10.13+ x86-64 |
|
SHA-256 checksum How to use checksums |
3ecc437576f9e182d8b6855ace8a9bb131c27b3e94f2d89f75c43663d58d864e
|
|
BLAKE2b-256 checksum How to use checksums |
c0259b271fe80c4d78f3b6b34620dcebfa72537d81930b8ab5dd1a73ffbe2d47
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp312-cp312-win_amd64.whl
| Download URL | unifyvcf-0.1.0-cp312-cp312-win_amd64.whl |
|---|---|
| Size | 191.1 kB |
| Tags | CPython 3.12 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
bc62a3342aac8313a18f2eeabdfcf0f002cce4d7788271856811ce878ccd86ff
|
|
BLAKE2b-256 checksum How to use checksums |
560bfe9d67f41bd62fb014c9da428a75294f845be7cf4f4613a31517907b963f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.5 MB |
| Tags | CPython 3.12 Linux glibc 2.17+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
88a75967ae5a93fd20c7853971548a6ab263041fb10066263a8e2e471ca62071
|
|
BLAKE2b-256 checksum How to use checksums |
1fbbcf05bb6726857443cc3c14c9055cae2f36802d643bcc930e078d6b4f223a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp312-cp312-macosx_10_13_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp312-cp312-macosx_10_13_x86_64.whl |
|---|---|
| Size | 222.1 kB |
| Tags | CPython 3.12 macOS 10.13+ x86-64 |
|
SHA-256 checksum How to use checksums |
49f14f412b42aad745448b5699acd363c15a858a8f5ab3353742099367260b36
|
|
BLAKE2b-256 checksum How to use checksums |
7098a4793e265c8602806918220aad466d0f60d5db1cba0c4ce112b15a80e72b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp311-cp311-win_amd64.whl
| Download URL | unifyvcf-0.1.0-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 196.3 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
3e641a7b6528a00af1de78fd7ff1da8df36acc5216b08343a9d0c4c791028b6f
|
|
BLAKE2b-256 checksum How to use checksums |
e20600f8ba7cea90a8c3cadd21b95e856421fb35437e4c7d10c0959fc0865422
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
616cf1dc7aac123ca2bcbcdfbf1f82c63b0c386de21653181558b054af794595
|
|
BLAKE2b-256 checksum How to use checksums |
a8b7b73544ca990717a8bac6274f8865941c72922d5a21b3a4dfb0eece9fd631
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp311-cp311-macosx_10_9_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp311-cp311-macosx_10_9_x86_64.whl |
|---|---|
| Size | 217.8 kB |
| Tags | CPython 3.11 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
55fb9ed42aa91409e14bd3c4c5013744a894653fecfbdb56841633d2a2289e3f
|
|
BLAKE2b-256 checksum How to use checksums |
f67e7f7914a9f134294c07f9cd4099349d7a7220c331a2875e41e0efb3c10162
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp310-cp310-win_amd64.whl
| Download URL | unifyvcf-0.1.0-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 197.8 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
db9dc626dda34ac4d1cf3bb52b730fbcbbde2a0e8a18ca91bbd140ffcffc2893
|
|
BLAKE2b-256 checksum How to use checksums |
d557c94f636a4f5dc48d5fe087c935609e527fe7d662eafb58f2b37f779c1982
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
3c6c68688f1e51627804efe61f3a4ffb45ebadb08b8936a023dcb59926344fe5
|
|
BLAKE2b-256 checksum How to use checksums |
c7cf0c5c838d8bca28bf2b089d9a05bf2798c6dd8e9d3ac9cb1d2a7d620b1c3b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp310-cp310-macosx_10_9_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp310-cp310-macosx_10_9_x86_64.whl |
|---|---|
| Size | 221.4 kB |
| Tags | CPython 3.10 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
d6f87297dd57a72c92bc35641003fa02ecf7305b041ecc26cd24928accacdc84
|
|
BLAKE2b-256 checksum How to use checksums |
4b6cba031b21936990fd822700b8ba45e5d4c73a05f3a5a123f9eb4a5cdf2d28
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp39-cp39-win_amd64.whl
| Download URL | unifyvcf-0.1.0-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 198.3 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
691c4bc6d840d609d0c86ce973fc1fa101b6f0ce04478af6e475c022b5f119eb
|
|
BLAKE2b-256 checksum How to use checksums |
37b7f5a9cb3cb83ff25504f10da115e4bf15c88e140647f40d4045be2899b3fc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.3 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
fad60a31649ef911577d81ffc8905f5da1176b808ac4eb3f406f8b1369c446e6
|
|
BLAKE2b-256 checksum How to use checksums |
65f31bf5b335e1ad0e212c1de835861efbcb038b30dadb0a8c6ce5fe1137933c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / unifyvcf-0.1.0-cp39-cp39-macosx_10_9_x86_64.whl
| Download URL | unifyvcf-0.1.0-cp39-cp39-macosx_10_9_x86_64.whl |
|---|---|
| Size | 222.9 kB |
| Tags | CPython 3.9 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
29913cd5fb9f42d04ff440e747fe0fd091509b73221351c77c7506be2f216f91
|
|
BLAKE2b-256 checksum How to use checksums |
df3da80a0d928f9aaea8cec49161151187b2f38fdf28c17007fa53e2f9c0824e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Security for UnifyVcf 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.