Variation Normalization

The Variation Normalizer parses and translates free-text descriptions of genomic variations into computable objects conforming to the Variation Representation Specification (VRS), enabling consistent and accurate variant harmonization across a diversity of genomic knowledge resources.

Live OpenAPI endpoint

Installation

Install from PyPI:

python3 -m pip install variation-normalizer

About

Variation Normalization works by using four main steps: tokenization, classification, validation, and translation. During tokenization, we split strings on whitespace and parse to determine the type of token. During classification, we specify the order of tokens a classification can have. We then do validation checks such as ensuring references for a nucleotide or amino acid matches the expected value and validating a position exists on the given transcript. During translation, we return a VRS Allele object.

Variation Normalization is limited to the following types of variants:

• HGVS expressions and text representations (ex: BRAF V600E):
  • protein (p.): substitution, deletion, insertion, deletion-insertion
  • coding DNA (c.): substitution, deletion, insertion, deletion-insertion
  • genomic (g.): substitution, deletion, ambiguous deletion, insertion, deletion-insertion, duplication
• gnomAD-style VCF (chr-pos-ref-alt, ex: 7-140753336-A-T)
  • genomic (g.): substitution, deletion, insertion

Variation Normalizer accepts input from GRCh37 or GRCh8 assemblies.

We are working towards adding more types of variations, coordinates, and representations.

VRS Versioning

The variation-normalization repo depends on VRS models, and therefore each variation-normalizer package on PyPI uses a particular version of VRS. The correspondences between packages may be summarized as:

Previous VRS Versioning

The correspondences between the packages that are no longer maintained may be summarized as:

Available Endpoints

/to_vrs

Returns a list of validated VRS Variations.

/normalize

Returns a VRS Variation aligned to the prioritized transcript. The Variation Normalizer relies on Common Operations On Lots-of Sequences Tool (cool-seq-tool) for retrieving the prioritized transcript data. More information on the transcript selection algorithm can be found here.

If a genomic variation query is given a gene (E.g. BRAF g.140753336A>T), the associated cDNA representation will be returned. This is because the gene provides additional strand context. If a genomic variation query is not given a gene, the GRCh38 representation will be returned.

Development

Clone the repo:

git clone https://github.com/cancervariants/variation-normalization.git
cd variation-normalization

For a development install, we recommend using Pipenv. See the pipenv docs for direction on installing pipenv in your compute environment.

Once installed, from the project root dir, just run:

pipenv shell
pipenv update && pipenv install --dev

Required resources

Variation Normalization relies on some local data caches which you will need to set up. We provide instructions on how to setup your development environment using Docker.

• SeqRepo: You must setup SeqRepo locally following these steps.
• Gene Normalizer: The Variation Normalizer uses Gene Normalizer to get normalized gene concept information.
• Universal Transcript Archive (UTA): The Variation Normalizer uses Common Operations On Lots-of Sequences Tool (cool-seq-tool) which uses UTA as the underlying PostgreSQL database.

SeqRepo

Variation Normalization relies on seqrepo, which you must download yourself.

Variation Normalizer uses seqrepo to retrieve sequences at given positions on a transcript.

From the root directory:

pip install seqrepo
sudo mkdir /usr/local/share/seqrepo
sudo chown $USER /usr/local/share/seqrepo
seqrepo pull -i 2024-12-20/  # Replace with latest version using `seqrepo list-remote-instances` if outdated

If you get an error similar to the one below:

PermissionError: [Error 13] Permission denied: '/usr/local/share/seqrepo/2024-12-20/._fkuefgd' -> '/usr/local/share/seqrepo/2024-12-20/'

You will want to do the following:
(Might not be ._fkuefgd, so replace with your error message path)

sudo mv /usr/local/share/seqrepo/2024-12-20._fkuefgd /usr/local/share/seqrepo/2024-12-20
exit

Use the SEQREPO_ROOT_DIR environment variable to set the path of an already existing SeqRepo directory. The default is /usr/local/share/seqrepo/latest.

UTA

You must download uta_20241220.pgd.gz from https://dl.biocommons.org/uta/ using a web browser and move it to the root of the repository.

Docker Installation (Preferred)

We recommend installing the Variation Normalizer using Docker.

Requirements

• Docker

Build, (re)create, and start containers

docker volume create uta_vol
docker compose up

[!IMPORTANT] This assumes you have a local SeqRepo installed at /usr/local/share/seqrepo/2024-12-20. If you have it installed elsewhere, please update the SEQREPO_ROOT_DIR environment variable in compose.yaml.
If you're using Docker Desktop, you'll want to go to Settings -> Resources -> File sharing and add /usr/local/share/seqrepo under the Virtual file shares section. Otherwise, you will get the following error: OSError: Unable to open SeqRepo directory /usr/local/share/seqrepo/2024-12-20.

[!TIP] If you want a clean slate, run docker compose down -v to remove containers and volumes, then docker compose up --build to rebuild and start fresh containers.

Point your browser to http://localhost:8001/variation/.

Code QC

Code style is managed by Ruff and checked prior to commit.

To perform formatting and check style:

python3 -m ruff format . && python3 -m ruff check --fix .

We use pre-commit to run conformance tests.

This ensures:

• Style correctness
• No large files
• AWS credentials are present
• Private key is present

Pre-commit must be installed before your first commit. Use the following command:

pre-commit install

Testing

From the root directory of the repository:

pytest tests/

Dependency management

Production runtime dependencies need to be updated in three places:

• pyproject.toml declares dependencies for the wheel that's published to PyPI
• requirements.txt declares dependencies for our Elastic Beanstalk-based deployment
  • Note that it can be trivially regenerated with the command uv pip compile pyproject.toml -o requirements.txt --no-annotate
• Pipfile is used as a backup for Elastic Beanstalk dependency management

Note that dev/testing dependencies only need to be updated in pyproject.toml.

Creating a new release

1. Version number must be updated manually. It's declared under project.version in pyproject.toml. Ensure that the version value for the Docker image in compose.yaml is similarly updated.
2. Once a commit with an updated version is merged to the staging branch, create a new tag + GitHub release (from the staging branch). This triggers the PyPI and GHCR publishing workflows. Presently, new commits to staging should not be merged to main.