ferro-hgvs

A high-performance HGVS variant nomenclature parser and normalizer written in Rust.

WARNING: ALPHA SOFTWARE - USE AT YOUR OWN RISK

This software is currently in ALPHA. While we have extensively tested it across a wide variety of HGVS patterns, no guarantees are made regarding correctness or stability.

Features

• Full HGVS Parsing: All coordinate systems (g/c/n/r/p/m/o) and edit types
• Variant Normalization: 3'/5' shifting per HGVS specification
• High Performance: ~5M variants/sec single-threaded parsing (>12M/s parallel), zero-copy with nom
• Type-Safe: Leverages Rust's type system for correctness

Installation

Python

pip install ferro-hgvs

Pre-built wheels are available for Linux (x86_64, aarch64), macOS (x86_64, Apple Silicon), and Windows (x86_64) on Python 3.10+.

Rust

Add to your Cargo.toml:

[dependencies]
ferro-hgvs = "0.1"

Or install the CLI:

cargo install ferro-hgvs

Quick Start

CLI

# Parse a variant
ferro parse "NM_000088.3:c.459A>G"

# Parse from file
ferro parse -i variants.txt -f json

# Prepare reference data (downloads RefSeq, genome, cdot — RefSeq-only by default)
ferro prepare --output-dir ferro-reference

# Verify reference data is ready
ferro check --reference ferro-reference

# (Optional) pre-build the on-disk cdot cache as a setup step, so the one-time
# cache build doesn't slow the start of a real (or timed/benchmarked) run.
ferro check --reference ferro-reference --build-cache

# Normalize with reference
ferro normalize "NM_000088.3:c.459del" --reference ferro-reference/

Throughput tip: when normalizing many variants, feed them sorted by transcript accession (or by genomic position). ferro caches each resolved transcript, so consecutive variants on the same transcript skip the (dominant) cost of re-reading and re-building it from the reference. Sorted input keeps the relevant transcripts resident in the cache and is markedly faster on large batches — see Performance Comparison.

Optional reference data

A bare ferro prepare builds a RefSeq-only reference (accessions NM_/NR_/NP_/NG_). Two opt-in flags provision additional data — pass them at prepare time; they are what a fully-provisioned ("blessed") reference is built with:

# Add Ensembl support (accessions ENST/ENSG/ENSP). Downloads the Ensembl cdot
# metadata and cDNA FASTAs (~1 GB+); off by default. Without it, an ENST/ENSG/ENSP
# input reports "Reference not found" and the message points back at this flag.
ferro prepare --output-dir ferro-reference --ensembl

# Derive version-independent NG_ placements and the NG_→transcript-version map
# (ng_hosted_transcripts) for a curated list of RefSeqGene accessions. Required to
# resolve legacy gene-symbol selectors (NG_(GENE):c.…) and bare-NG_ hosted lookups.
ferro prepare --output-dir ferro-reference \
  --derive-ng-placements path/to/ng_accessions.txt

# A fully-provisioned reference combines both in one run:
ferro prepare --output-dir ferro-reference --ensembl \
  --derive-ng-placements path/to/ng_accessions.txt

Both flags are incremental: re-running ferro prepare over an existing reference adds the requested data and preserves already-provisioned artifacts.

Library

use ferro_hgvs::{parse_hgvs, HgvsVariant};

fn main() -> Result<(), ferro_hgvs::FerroError> {
    let variant = parse_hgvs("NM_000088.3:c.459A>G")?;

    match &variant {
        HgvsVariant::Cds(v) => println!("CDS variant: {}", v),
        HgvsVariant::Genome(v) => println!("Genomic variant: {}", v),
        _ => println!("Other: {}", variant),
    }

    Ok(())
}

Python

import ferro_hgvs

# Parse a variant
variant = ferro_hgvs.parse("NM_000088.3:c.459A>G")
print(variant.variant_type)  # "coding"
print(variant.reference)     # "NM_000088.3"
print(str(variant))          # "NM_000088.3:c.459A>G"

# Normalize with reference data
normalizer = ferro_hgvs.Normalizer(reference_json="ferro-reference/cdot.json")
normalized = normalizer.normalize("NM_000088.3:c.459del")

Supported HGVS Syntax

Edit Types

• Substitution: A>G, Val600Glu
• Deletion: del, 100_200del
• Insertion: 100_101insATG
• Deletion-Insertion: 100_102delinsATG
• Duplication: 100_102dup
• Inversion: 100_200inv
• Repeat: 100CAG[20]

CLI Commands

The ferro CLI provides commands beyond parsing and normalization:

Error Handling

ferro-hgvs provides configurable error handling with three modes:

# Use lenient mode to auto-correct common issues
ferro parse --error-mode lenient "p.val600glu"  # Corrects to p.Val600Glu

# Ignore specific warnings
ferro parse --ignore W1001,W2001 "p.val600glu"

# Get help on any error/warning code
ferro explain W1001
ferro explain --list

Configuration File

Create .ferro.toml in your project directory:

[error-handling]
mode = "lenient"
ignore = ["W1001", "W2001"]  # Silently correct these
reject = ["W3003"]           # Always reject these

Comparing normalization rules (FERRO_PARTITION)

Unstable. FERRO_PARTITION is an evaluation switch, not a supported feature. It is not covered by semantic versioning, its values may change, and it is expected to be removed once the normalization rule is settled. Do not depend on it in production pipelines.

Normalization cuts each changed region of sequence into allele members. FERRO_PARTITION selects which rule does that cutting, so a candidate rule can be measured against the shipped one over a real corpus before anything changes for users.

With the variable unset — or set to the empty string — output is byte-identical to a build with no switch at all.

Running an A/B comparison

Run the same input twice and diff the normalized descriptions. In tsv format the columns are line, input, normalized, changed, status, detail, so column 3 is the normalized string:

# 1. the shipped behaviour
ferro normalize --input variants.txt --reference /path/to/reference \
  --format tsv --error-mode lenient -j 10 > shipped.tsv

# 2. the candidate rule
FERRO_PARTITION=canonical ferro normalize --input variants.txt --reference /path/to/reference \
  --format tsv --error-mode lenient -j 10 > candidate.tsv

# 3. what moved
diff <(cut -f3 shipped.tsv) <(cut -f3 candidate.tsv) | head

# 4. how many moved, counting only rows that succeeded on both sides
#    (columns 3/5 are `normalized`/`status`; a row that failed carries no
#     normalized string, so comparing column 3 alone would score two different
#     failures as identical)
paste <(cut -f3,5 shipped.tsv) <(cut -f3,5 candidate.tsv) \
  | awk -F'\t' '$2 == "ok" && $4 == "ok" && $1 != $3' | wc -l

# and, separately, rows whose status itself changed
paste <(cut -f5 shipped.tsv) <(cut -f5 candidate.tsv) | awk -F'\t' '$1 != $2' | wc -l

This works on a stock release build — the switch is not behind a build feature, so no special binary or wheel is needed.

Two traps worth knowing

A misspelled value looks like success, and nothing tells you. An unrecognised value falls back to live, so FERRO_PARTITION=canonicl produces a clean, empty diff that reads as "the candidate changes nothing".

The fallback does emit a warning through the log facade — but the ferro CLI installs no logger, so that warning is not visible from the command line, and RUST_LOG will not surface it. There is no signal at all.

The defence is a positive control on an input known to differ — not on your own corpus, where a zero is ambiguous between "the switch is not taking effect" and "this corpus has no affected variants".

These three run against the built-in test data, so they need no --reference and no prepared reference directory:

printf 'NM_001234.1:c.[2del;9del]\nNM_001234.1:c.[3del;9del]\nNM_001234.1:c.[2del;9dup]\n' > control.txt

ferro normalize --input control.txt --format tsv --error-mode lenient | cut -f3
#   NM_001234.1:c.[2del;11del]   <- live
FERRO_PARTITION=canonical ferro normalize --input control.txt --format tsv --error-mode lenient | cut -f3
#   NM_001234.1:c.[2del;33del]   <- canonical

If those two produce the same output, the variable is not reaching ferro and any comparison you run is meaningless. Only once the control differs is a zero on your own corpus informative.

From Python, it must be set before the first normalization. The value is read once per process and cached, so:

import os
os.environ["FERRO_PARTITION"] = "canonical"   # must precede the first normalize call
import ferro_hgvs

Setting it after any variant has been normalized silently does nothing, and it cannot be changed within a running process. Comparing two rules therefore means two separate processes (or two CLI runs, as above), not two calls in one script.

Why ferro-hgvs?

ferro-hgvs provides the most comprehensive HGVS variant normalization across all pattern types, with performance orders of magnitude faster than alternatives.

Normalization Capabilities Comparison

* mutalyzer protein normalization requires network access for NP_→NM_ lookups (cannot be cached locally). ** mutalyzer intronic support is enabled by default via genomic-context rewriting; disable with --no-rewrite-intronic.

Performance Comparison

All tools are benchmarked in ferro's offline configuration — best case for every tool. Reference data is preloaded locally (a local UTA database and SeqRepo) and the network is disabled, so the figures below measure parse/normalize compute, not I/O. Out of the box, hgvs-rs, biocommons/hgvs, and mutalyzer resolve each variant against a remote UTA/SeqRepo or the Mutalyzer web API — a network round-trip per variant (~100–1000 ms), i.e. roughly 1–10 variants/sec, hundreds to thousands of times slower than shown here (an order-of-magnitude estimate from per-call network latency, not separately benchmarked). That local, offline setup is exactly what ferro's prepare command builds; ferro needs no external service.

Median patterns/sec over 5 reps on an Apple M2 Max, local/offline. All tools draw from one stratified ClinVar population; per-tool sample sizes are calibrated so each tool is measured over a meaningful interval — fast cells (e.g. ferro/hgvs-rs parse) draw from millions of patterns, while slower cells (e.g. the per-tool normalize columns) draw from as few as tens to thousands. All tools exclude process/interpreter startup from the timed region — the mutalyzer/biocommons Python subprocesses are timed by their own internal startup-excluded timer, matching ferro/hgvs-rs. Only ferro parallelizes natively (rayon); the other tools are single-threaded libraries, so their normalize @8 workers figures come from the benchmark harness running 8 independent instances in parallel, while parsing is not sharded for them — hence the single-threaded label in their parse @8 workers column (mutalyzer normalize likewise shows no gain at 8 workers: per-call cache and IPC overhead dominate, so sharding does not help). Every tool runs fully offline against local reference data — a local UTA database and SeqRepo, with mutalyzer's network lookups disabled — the configuration ferro's prepare command enables; the figures therefore reflect compute throughput, not per-variant network latency. Reference-data load is excluded for all tools. ferro full-population peak: parse 20.0M/s, normalize 77.0k/s. See docs/BENCHMARK_RUNBOOK.md for the full method.

Parse

Normalize

ferro thread scaling

Input ordering matters for batch throughput. Resolving a transcript (reading its full sequence from the reference and rebuilding its CDS/exon metadata) dominates per-variant cost. ferro memoizes resolved transcripts in a bounded in-memory cache, so repeated lookups of the same transcript are near-free. Providing variants sorted by transcript accession — or by genomic position, which clusters variants onto the same transcripts — maximizes the cache hit rate and can speed up large batches by an order of magnitude versus randomly-ordered input. Ordering matters most when the number of distinct transcripts in the run exceeds the cache capacity (very large or genome-wide inputs); below that, the working set stays resident regardless of order.

Reference Data: What ferro Prepares

The ferro prepare command downloads and organizes all reference data needed for comprehensive normalization. This data is then shared with other tools (mutalyzer, biocommons, hgvs-rs) to enable their local operation.

Key insight: Without ferro's reference preparation, other tools require network access for each variant lookup (adding 100-1000ms latency per variant). With ferro's cached reference data, all tools can operate fully offline with consistent, reproducible results.

Deriving version-independent NG_ placements (#728)

ferro prepare --derive-ng-placements <accessions.txt> derives genomic placements for the listed NG_ versions (one exact accession per line, e.g. NG_012337.3; blank lines and # comments ignored), writing derived_refseqgene_placements.json into the reference directory and wiring the manifest's derived_refseqgene_placements field. This fills version gaps the archived RefSeqGene→genome GFF3 snapshots do not cover. It needs cdot + the genome in the same prepare run and uses NCBI EFetch per accession; accessions that cannot be validated are skipped with a warning. The field is preserved across subsequent prepare runs.

Benchmark: Reference Data & Tool Comparison

The main ferro binary includes commands to prepare reference data (ferro prepare) and check its status (ferro check). The ferro-benchmark tool (build with --features benchmark) extends this for tool comparison benchmarks.

Quick Start

# Prepare ferro reference (main binary - no special features needed)
ferro prepare --output-dir data/ferro

# Check reference data
ferro check --reference data/ferro

# Normalize with ferro
ferro normalize -i patterns.txt --reference data/ferro

# For tool comparison, build with benchmark support
cargo build --release --features benchmark

# Prepare other tools (uses ferro reference for transcript data)
ferro-benchmark prepare mutalyzer --ferro-reference data/ferro --output-dir data/mutalyzer
ferro-benchmark prepare biocommons --seqrepo-dir data/seqrepo --uta-dump uta_20210129b.pgd.gz --ferro-reference data/ferro

# Compare results between tools
ferro-benchmark normalize mutalyzer -i patterns.txt -o mutalyzer.json --mutalyzer-settings data/mutalyzer/mutalyzer_settings.conf
ferro-benchmark compare results normalize ferro.json mutalyzer.json -o comparison.json

Supported tools: ferro-hgvs, mutalyzer, biocommons/hgvs, hgvs-rs

Note: The pixi.toml and pixi.lock files in this repository define a pixi environment for the Python-based external tools (mutalyzer, biocommons/hgvs, seqrepo) used in benchmarking. Run pixi shell to activate it.

See docs/BENCHMARK_GUIDE.md for detailed usage.

Development

cargo build
cargo test                        # default features
cargo clippy -- -D warnings

The commands above use the default feature set, and CI keeps them compiling (see the build job). They do not cover the whole suite — the feature-gated tests and the integration tree need dev, which is what CI runs and what you want before opening a PR:

cargo nextest run --features dev
cargo clippy --features dev --all-targets -- -D warnings

License

Licensed under the MIT License. See LICENSE for details.

Disclaimer

This software is under active development. While we make a best effort to test this software and to fix issues as they are reported, this software is provided as-is without any warranty (see the license for details). Please submit an issue, and better yet a pull request as well, if you discover a bug or identify a missing feature. Please contact Fulcrum Genomics if you are considering using this software or are interested in sponsoring its development.

Contributing

See CONTRIBUTING.md for guidelines.