clinvar-proto

Generates clinvar_proto — a typed Python package for parsing NCBI ClinVar VCV XML into protobuf messages — from the ClinVar VCV XSD.

This repository is the generator, not the package. It holds the inputs (ClinVar_VCV_2.6.xsd, clinvar_transforms.yaml) and drives xsd-former (the xsdformer CLI) to emit the clinvar_proto source tree, which is then built into a wheel and published to PyPI. The generated tree (generated/) and build outputs (dist/) are gitignored — only the inputs are version-controlled.

Sibling to pubmed-proto; same generate-on-demand model.

Consuming clinvar_proto

Depend on the published wheel, not this repo:

pip install clinvar_proto      # or: uv add clinvar_proto

from lxml import etree
from clinvar_proto import xml_converter

tree = etree.parse('clinvar_release.xml')
archive_el = tree.getroot().find('VariationArchive')

proto = xml_converter.VariationArchiveType(archive_el)  # XML -> protobuf
data = proto.SerializeToString()  # protobuf wire bytes, for storage

The converters take any ElementTree-like element, so lxml is the consumer's choice, not a dependency of the wheel (clinvar_proto needs only protobuf). Release files are large, so consumers normally stream them with lxml.etree.iterparse on VariationArchive and convert one element at a time. Use lxml >= 6.1 if you do: earlier releases resolve external entities by default in iterparse (CVE-2026-41066).

The package exposes two modules (both typed; ships py.typed):

Developing the generator

Requires uv.

make generate   # XSD + transforms -> generated/clinvar_proto/
make build      # generate, then build the wheel into dist/
make clean      # remove generated/ and dist/
uv run --group test pytest   # record gate over real ClinVar records

Shaping the output is done in clinvar_transforms.yaml — inlining single- child wrappers, flattening list wrappers, and attaching documentation comments to the generated messages and fields. See the xsd-former docs for the transform reference.

Provenance & attribution

ClinVar_VCV_2.6.xsd is the NCBI ClinVar VCV XSD, version 2.6 (dated 2026-02-26):

https://ftp.ncbi.nlm.nih.gov/pub/clinvar/xsd_public/ClinVar_VCV_2.6.xsd

The vendored file is byte-identical to upstream (MD5 a7b65e5a166dc5f36a7eea9127d56f4e, matching NCBI's published ClinVar_VCV_2.6.xsd.md5). NCBI ClinVar data and schemas are U.S. Government works and public domain in the United States; the MIT LICENSE in this repo covers CPG's own files (transforms, generator wiring, tests), not the NCBI XSD.

Courtesy of the U.S. National Library of Medicine. NLM/NCBI does not endorse this package. The vendored XSD and the ClinVar records under tests/records/ are pinned snapshots and do not necessarily reflect the most current data available from NCBI — fetch from NCBI directly for current data.

The XSD is vendored deliberately, not fetched at build time: pinning the exact bytes keeps the generated schema reproducible, and NCBI's XSD version is bumped in place under a new filename rather than being immutable per release. When NCBI publishes a new version, re-vendor the file from https://ftp.ncbi.nlm.nih.gov/pub/clinvar/xsd_public/, point the Makefile and tests at it, regenerate, and run the record gate.

Releasing

The published version is build.version in clinvar_transforms.yaml (what xsdformer stamps into the wheel). To release:

1. Bump build.version in clinvar_transforms.yaml.
2. Publish a GitHub Release tagged vX.Y.Z matching that version.

The release workflow generates, builds, and publishes to PyPI via Trusted Publishing (OIDC). It fails if the tag and build.version disagree.