nci-cidc-schemas 0.27.20

The CIDC data model and tools for working with it.

cidc-schemas

This repository contains formal definitions of the CIDC metadata model using json-schema syntax and vocabulary.

View documentation at https://nci-cidc.github.io/cidc-schemas/

Installation

To install the latest released version, run:

pip install nci-cidc-schemas

Development

Project Structure

• cidc_schemas/ - a python module for generating, validating, and reading manifest and assay templates.
  • schemas/ - json specifications defining the CIDC metadata model.
    • templates/ - schemas for generating and validating manifest and assay templates.
    • assays/ - schemas for defining assay data models.
    • artifacts/ - schemas for defining artifacts.
• docs/ - the most recent build of the data model documentation, along with templates and scripts for re-generating the documentation.
• template_examples/ - example populated Excel files for template specifications in schemas/templates, and .csvs auto-generated from those .xlsxs that allow to transparently keep track of changes in them.
• tests/ - tests for the cidc_schemas module.

Developer Setup

Install necessary dependencies.

pip install -r requirements.dev.txt

Install and configure pre-commit hooks.

pre-commit install

Running tests

This repository has unit tests in the tests folder. After installing dependencies the tests can be run via the command

pytest tests

Building documentation

Pre-commit hooks ensure documentation is automatically up-to date. To build the documentation manually, run the following commands:

python setup.py install # install helpers from the cidc_schemas library
python docs/generate_docs.py

This will output the generated html documents in docs/docs. If the updated docs are pushed up and merged into master, they will be viewable at https://nci-cidc.github.io/cidc-schemas/.

Using the Command-Line Interface

This project comes with a command-line interface for validating schemas and generating/validating assay and manifest templates.

Install the CLI

Clone the repository and cd into it

git clone [e-mail]:NCI-CIDC/cidc-schemas.git
cd cidc-schemas

Install the cidc_schemas package (this adds the cidc_schemas CLI to your console)

python setup.py install

Run cidc_schemas --help to see available options.

If you're making changes to the module and want those changes to be reflected in the CLI without reinstalling the cidc_schemas module every time, run

python3 -m cidc_schemas.cli [args]

Generate templates

Create a template for a given template configuration.

cidc_schemas generate_template -m templates/manifests/pbmc_template.json -o pbmc.xlsx

Validate filled-out templates

Check that a populated template file is valid with respect to a template specification.

cidc_schemas validate_template -m templates/manifests/pbmc_template.json -x template_examples/pbmc_template.xlsx

Validate JSON schemas

Check that a JSON schema conforms to the JSON Schema specifications.

cidc_schemas validate_schema -f shipping_core.json