genoray
If you want to use NumPy with genetic variant data, genoray is for you! genoray enables ergonomic and efficient range queries of genotypes and dosages from VCF and PGEN (PLINK 2.0) files. genoray is also fully type-safe and has minimal dependencies.
Summary
The genoray API more-or-less boils down to just two classes and up to five methods:
VCFandPGENclasses for reading VCF and PGEN files, respectively.readread variants for a single range.chunkread variants for a single range in chunks.read_rangesread multiple ranges of variants at once.chunk_rangesread multiple ranges of variants in chunks.set_samplessubset and/or re-order the samples.
The other important arguments to know are mode (and phasing for VCF) to set the return type and max_mem for chunking. The modes that are available for each file format are always accessible from the class itself, e.g. VCF.Genos16, PGEN.GenosDosages, etc. You can also filter variants on the fly using the filter argument to class constructors.
Also included:
SparseVarandSparseVar2sparse variant stores for compact, range-queryable on-disk representations of genotype data.Referencefor reading reference genome sequence.- Mutation catalogues and signature refitting via
cosmic_signaturesandfit_signatures. - A
genoray index|write|viewCLI for building indices, converting VCF/PGEN to sparse formats, and inspecting variant files.
See the genoray-api skill and the docs for details.
Examples
VCF
We work with VCFs using the (you guessed it) VCF class:
from genoray import VCF
vcf = VCF("file.vcf.gz")
Querying data for a region is as simple as:
# shape: (samples ploidy variants)
genos = vcf.read("1") # read all variants on chromosome 1
You can also change the return type to be either genotypes and/or dosages by providing a mode argument:
vcf = VCF("file.vcf.gz", dosage_field="DS") # need a dosage_field to read dosages
genos, dosages = vcf.read("1", mode=VCF.Genos16Dosages)
Dosages have shape (samples, variants) and dtype np.float32.
A key feature of genoray is letting you work with data that is too large to fit into memory. For example:
vcf = VCF("file.vcf.gz", phasing=True) # include phasing status
# max_mem defaults to "4g", can also be capitalized or be "GB", for example
# Genos8 reduces precision to int8 from the default int16 that cyvcf2 uses
genos = vcf.chunk("1", max_mem="4g", mode=VCF.Genos8)
for chunk in genos:
# do something with chunk, each chunk is a NumPy array of shape (samples, ploidy+1, variants)
...
The chunk method will automatically chunk the data along the variants axis to respect the memory limit, returning a generator of data instead of everything at once.
PGEN
from genoray import PGEN
pgen = PGEN("file.pgen")
We can query data for a region in the same way as VCF:
# shape: (samples ploidy variants)
genos = pgen.read("1") # read all variants on chromosome 1
genos = pgen.chunk("1") # read all variants on chromosome 1
However, PGEN files also support reading multiple ranges at once since this improves throughput substantially:
# shape: (samples, ploidy, variants), shape: (n_ranges+1)
genos, offsets = pgen.read_ranges('1', starts=[1, 1000, 2000], ends=[1000, 2000, 3000])
first_range_genos = genos[..., offsets[0]:offsets[1]]
genos = pgen.chunk_ranges('1', starts=[1, 1000, 2000], ends=[1000, 2000, 3000])
for range_ in genos:
if range_ is None:
# no data for this range
continue
for chunk in range_:
# do something with chunk, each chunk is a NumPy array of shape (samples, ploidy, variants)
...
The read_ranges method takes starts and ends and returns data for each range and the offsets to slice out the variants for each range. Since the data is allocated as a single array, the offsets let you slice out the data for each range from the variants axis.
Like VCF, methods for PGENs accept a mode argument to change the return type to include genotypes, phasing, and/or dosages:
genos, phasing, dosages = pgen.read("1", mode=PGEN.GenosPhasingDosages)
The PGEN reader adheres to pgenlib's API, so the phasing information is in a separate boolean array instead of using an extra column like VCF/cyvcf2. The phasing information is a boolean array of shape (samples, variants) where True indicates that the genotype is phased and False indicates that it is unphased.
pgen = PGEN("hardcalls.pgen", dosage_path="dosage.pgen", ...)
Filtering
You can filter variants from VCF or PGEN files by a providing a function or polars expression to the constructor, respectively.
For VCFs, the function must accept a cyvcf2.Variant and return a boolean indicating whether to include the site.
# only include variants that are common in EUR
vcf = VCF("file.vcf.gz", filter=lambda v: v.INFO['AF_EUR'] > 0.05)
For PGENs, the expression operates on the .gvi index — a polars DataFrame with columns:
CHROM— contig namePOS— 1-based positionREF— reference alleleALT— list of alternate allelesILEN— list of indel lengths (one per ALT:len(ALT) - len(REF), or a signed size for symbolic SVs;nullfor un-sizable symbolic/breakend alleles)
Prefer the ready-made expressions in genoray.exprs — is_snp, is_indel, is_biallelic, is_symbolic, is_breakend, is_imprecise, and ILEN — and combine them with polars operators. For custom predicates, use pl.col("CHROM"/"POS"/"REF"/"ALT"/"ILEN") directly.
import genoray
from genoray import PGEN
# only include SNPs
pgen = PGEN("file.pgen", filter=genoray.exprs.is_snp)
# exclude symbolic alleles and breakends
pgen = PGEN("file.pgen", filter=~genoray.exprs.is_symbolic & ~genoray.exprs.is_breakend)
⚠️ Important ⚠️
- For the time being, ploidy is 2 for all classes in
genoray, but this could be more flexible for VCFs in the future. The PGEN format does not support ploidy other than 2. - Different file formats may use different data types for their respective representations of genotypes, phasing, and dosages.
- Ranges are 0-based, so starts begin at 0 and ends are exclusive.
- Missing genotypes and dosages are encoded as -1 and
np.nan, respectively. - Dosages from PGEN files may not exactly match VCF files (up to a fraction of a percent) because PLINK 2.0 must encode dosages with fixed precision which can not match what can be represented by text in a VCF (may also disagree with how BCF encodes dosage).
Contributing
To contribute to genoray, please fork the repository and create a pull request. We welcome contributions of all kinds, including bug fixes, new features, and documentation improvements. Please make sure to run the tests before submitting a pull request. We provide a Pixi environment that includes all development dependencies. To use the environment, install Pixi and run pixi run prek-install to activate pre-commit in your clone of the repo, and then run pixi s in the repository root directory. pixi s will activate the development environment and install all dependencies. You can then run the tests using pytest. ❗Note that all commits must adhere to conventional commits. If you have any questions or suggestions, please open an issue on the repository.
1 maintainer
Metadata
1 maintainer
Release files for genoray 5.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| genoray-5.0.0.tar.gz | 2.8 MB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| genoray-5.0.0-cp310-abi3-manylinux_2_28_x86_64.whl | CPython 3.10 | abi3 | Linux glibc 2.28+ x86-64 | Details |
| genoray-5.0.0-cp310-abi3-manylinux_2_28_aarch64.whl | CPython 3.10 | abi3 | Linux glibc 2.28+ ARM64 | Details |
| genoray-5.0.0-cp310-abi3-macosx_11_0_arm64.whl | CPython 3.10 | abi3 | macOS 11.0+ ARM64 | Details |
Total release size: 10.5 MB
Release files / genoray-5.0.0.tar.gz
| Download URL | genoray-5.0.0.tar.gz |
|---|---|
| Size | 2.8 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e2251c90cf6e9c09c0d55efd71e2a0aba4719c2edb3c259542cdef2b8b150f14
|
|
BLAKE2b-256 checksum How to use checksums |
440a850489f6850cbb8a0c9bc67df14c4ff890a8fb20410384fe344fbc67d3df
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.15 {"installer":{"name":"uv","version":"0.12.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / genoray-5.0.0-cp310-abi3-manylinux_2_28_x86_64.whl
| Download URL | genoray-5.0.0-cp310-abi3-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
1acf4a99c9afb196cfe8c9c53ad3e82668d092c6c72626c5ff6777f080afa07e
|
|
BLAKE2b-256 checksum How to use checksums |
c987a51030071ed0e4a9a9a411420d671353a1a5ea834b3e53091e2de44d4e48
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.15 {"installer":{"name":"uv","version":"0.12.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / genoray-5.0.0-cp310-abi3-manylinux_2_28_aarch64.whl
| Download URL | genoray-5.0.0-cp310-abi3-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
e5c7c7060fd8516b2cfd99333c00f40c31eb6823093a5474a4a717201ccd8c4d
|
|
BLAKE2b-256 checksum How to use checksums |
815fe8f697b1d468929acb0a40d31c080e351b4619258c33f8117bffbc52bbdc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.15 {"installer":{"name":"uv","version":"0.12.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / genoray-5.0.0-cp310-abi3-macosx_11_0_arm64.whl
| Download URL | genoray-5.0.0-cp310-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.3 MB |
| Tags | CPython 3.10 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
f5b5f3c4f296ed7e4013db8546c2528f32a520d5f09e36a0c4b3463ce1641258
|
|
BLAKE2b-256 checksum How to use checksums |
aefa9788f3e8457d419d51a6fa35cb334ebff40e2f8ef5a15a62a776f685e646
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.15 {"installer":{"name":"uv","version":"0.12.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
