Gen3SchemaDev 2.6.0

Tool for data modelling in Gen3

Developing a Gen3 Data Dictionary

This repository aims to provide the documentation, learning materials, and software tools to facilitate the creation of a data model in Gen3.

Pre-Reading: Please become familiar with some core data modelling concepts before starting.

Using Gen3SchemaDev as a data modelling tool

• Quickstart
• Guide to creating your first dictionary
• Troubleshooting

Fixing $ref properties in an existing dictionary

JSON Schema draft-04 (which Gen3 uses) ignores any keyword sitting as a direct sibling of $ref, so properties written as {description: ..., $ref: _definitions.yaml#/X} lose their description in the data-dictionary viewer. The fix-refs command rewrites such properties in place, moving the $ref into an allOf list so annotations survive resolution:

gen3schemadev fix-refs -y path/to/dictionary --dry-run   # preview changes
gen3schemadev fix-refs -y path/to/dictionary             # rewrite in place

The command is idempotent and never touches bare refs, refs already inside allOf/anyOf/oneOf, the properties: {$ref: ...} merge construct, or _definitions.yaml/_terms.yaml/_settings.yaml.

Both fix-refs and validate also warn about description: null placeholders anywhere in the dictionary (commonly in _definitions.yaml): the Gen3 metaschema requires descriptions to be strings, and such placeholders fail metaschema validation once exposed through bare or allOf-wrapped refs. Remove the null keys to resolve the warning.

Deep dive into Gen3 Data Modelling

Special Thanks to Marion Shadbolt for providing the source material

1. Gen3 Dictionary Overview
2. Gen3 Schema Explained
   1. Descriptors
   2. Links
   3. Properties
3. Example of a Gen3 Schema yaml
4. Handy tips
5. FAIR Concepts for Data Modelling

For Developers

Installation and testing

# To install
pip install poetry
poetry install
source $(poetry env info --path)/bin/activate
gen3schemadev --version

# To run tests
poetry run pytest

Contributing

Gen3SchemaDev is an open source project, and we highly encourage any contributions and PRs. Specifically we need the community to help with the following:

1. Keeping the gen3 metaschema up to date
2. Adding business rule logic to the rule validator module.
   1. For example, a node with the category data_file should have a collection of required data file properties such as md5sum, filesize, etc.

For PRs, please follow the contributing guidelines.

License

Apache 2.0