genome_entropy

genome_entropy quantifies Shannon entropy across biological representations derived from genomic DNA. It finds open reading frames (ORFs), translates proteins, predicts structural-state encodings, and writes a non-redundant JSON record for downstream analysis.

Current multitask ModernProst models produce both Foldseek 3Di and 12-state (12st) encodings. For those models, genome_entropy also reports the empirical mutual information between aligned 3Di and 12-state assignments in bits. The command remains named encode3di for compatibility. Legacy ModernProst and ProstT5 models produce 3Di only, with 12-state and mutual-information fields written as null.

Capabilities

• DNA FASTA and GenBank input, including gzip-compressed GenBank files
• six-frame ORF discovery through the external get_orfs program
• translation with pygenetic-code
• 3Di and optional 12-state prediction with ModernProst or ProstT5
• raw Shannon entropy for DNA, protein, 3Di, and 12-state representations
• raw 3Di--12-state mutual information for dual-head ModernProst outputs
• CUDA, ROCm-through-PyTorch's CUDA API, Apple MPS, CPU, and multi-GPU encoding
• optional XGBoost or PyTorch classification of whether an ORF matches a GenBank CDS

ORF discovery is gene calling, not functional annotation. Likewise, a classifier score is a model estimate of agreement with the supplied GenBank annotations, not biological proof of a gene or function.

Installation

Python 3.10 or newer is required.

pip install genome_entropy
pip install "genome_entropy[ml]"  # optional ML dependencies

get_orfs is not installed by pip; install it separately and ensure the executable is on PATH, or set GET_ORFS_PATH. GPU users should install a PyTorch build appropriate for their CUDA or ROCm platform before installing this package. See the installation guide for development, CPU, GPU, HPC, caching, and offline-job instructions.

Quick start

# Pre-cache the default model when login nodes have internet access
genome_entropy download --model gbouras13/modernprost-50M

# DNA FASTA to unified JSON
genome_entropy run --input genome.fasta --output results.json

# GenBank matching accepts aligned ambiguous X residues in C-terminal suffixes
genome_entropy run --genbank genome.gbk.gz --output results.json

# Protein FASTA directly to structural-state records
genome_entropy encode3di --input proteins.faa --output structures.json

Run genome_entropy --help and genome_entropy COMMAND --help for the installed version's authoritative option list. Detailed examples are in the quick-start guide and CLI reference.

Extract one GenBank record

The dependency-free bin/extract_genbank_locus utility extracts one record from a large, uncompressed GenBank file. It matches the complete first token after LOCUS, writes from that LOCUS line through its terminating // line, and stops scanning immediately. Build it with a POSIX C compiler:

make -C bin
bin/extract_genbank_locus INPUT.gbk LOCUS_ID OUTPUT.gbk

For example, bin/extract_genbank_locus assemblies.gb NC_000913.3 ecoli.gb matches NC_000913.3 exactly; it will not match NC_000913.30. The output is created only after a complete matching record is found. Gzip input is not supported by this standalone utility; decompress it first.

Supported encoders

ModernProst loads model-provided Python code with trust_remote_code=True. Review and pin model revisions when your threat model requires reproducible, audited remote code. The legacy alias gbouras13/modernprost-profiles resolves to gbouras13/modernprost-profiles-deprecated with a warning.

See the model guide for provenance, precision, devices, multi-GPU behaviour, output semantics, and verified repository links.

Output and entropy

The pipeline writes schema 2.2.0, with features keyed by ORF identifier. Each feature contains location, DNA, protein, 3Di, optional 12-state, metadata, raw entropy, and (when available) raw 3Di--12-state mutual information. JSON and JSON-gzip input are supported where documented.

Normalised entropy is deliberately not serialised. Derive it downstream:

from genome_entropy.entropy import normalise_protein_entropy

value = normalise_protein_entropy(feature["entropy"]["protein_entropy"])

The generic formula is raw_entropy / math.log2(alphabet_size), using theoretical alphabet sizes 4 (DNA), 20 (protein), 20 (3Di), and 12 (12-state). See data formats and entropy for the complete schema, coordinate conventions, null semantics, and normalisation helpers.

Documentation and support

• GitHub Pages documentation
• Read the Docs
• Issue tracker
• Machine-learning guide
• NVIDIA and ROCm SLURM notes

This project is alpha software. Report reproducible bugs through the issue tracker and include the package version, command, platform, accelerator, and relevant log output without credentials or sensitive sequence data.

Citation and attribution

Please cite the software release used in your analysis and the methods relevant to your workflow:

• Heinzinger et al., Bilingual language model for protein sequence and structure (ProstT5), NAR Genomics and Bioinformatics (2024), doi:10.1093/nargab/lqae150.
• van Kempen et al., Fast and accurate protein structure search with Foldseek, Nature Biotechnology (2024), doi:10.1038/s41587-023-01773-0.
• Chen and Guestrin, XGBoost: A Scalable Tree Boosting System, KDD (2016), doi:10.1145/2939672.2939785, when using the ML workflow.

ModernProst model repositories and integration were provided by George Bouras. See the full attribution page for model and dependency links.

Licence

genome_entropy is distributed under the MIT License.