ckanext-dcat-usmetadata 0.1.6

DCAT USMetadata Form App for CKAN

ckanext-dcat_usmetadata

This extension provides a new dataset form for inventory.data.gov. The form is tailored to managing metadata meeting the DCAT-US Schema.

Usage

Dependencies

This module currently depends on the USMetadata app for server-side validation and rendering. Make sure it is enabled in CKAN's plugins.

Installation

To install this package, activate CKAN virtualenv (e.g. "source /path/to/virtenv/bin/activate"), then run

(virtualenv) pip install ckanext-dcat-usmetadata

In your CKAN .ini file add dcat_usmetadata to your enabled plugins:

ckan.plugins = [YOUR PLUGINS HERE...] dcat_usmetadata

Development

Prerequisites

These tools are required for development.

• Node.js 12.x
• GNU Make

Install global dependencies.

$ make setup

Setup

Install Node.js dependencies.

$ yarn install

Build the JS application.

$ yarn run build

Build the docker containers.

$ make build

Run the tests.

$ make test

Testing

There are several levels of testing:

TODO complete this, make sure it's accurate.

Suite	Description	Command
unit tests for the JS app
browser tests for the JS app powered by cypress
python integration tests for ckan		make test
e2e		yarn e2e

Linting

TODO

Lint the python code.

$ make lint-all

Lint the JavaScript code.

$ make app-lint

Metadata app

The Metadata app is a Create React App-bootstrapped project.

To run the app use make app-up

TODO briefly describe how the metdata application relates to the CKAN extension.

Development

This project uses cosmos for development.

Run CKAN locally (make up) and get the Admin user's API Key. Add a test org for development purposes and get the id. Add these values to indicated place in /metadata-app/src/index.js.

Run make app-cosmos to start the cosmos server, which will watch the metadata-app/src directory for changes.

Run the unit tests.

$ make app-test

Update Jest snapshots

Some tests render a fixture component with Jest and then match against a known good snapshot (HTML rendering) of the component. When you edit a component, you'll usually have to update the snapshot and inspect the diff to make sure all changes are as intended.

$ cd metadata-app && yarn test --updateSnapshot

Local development and end-to-end testing

We use the inventory app locally for development and end-to-end (e2e) testing.

To build the latest JS code and update assets in the CKAN extension, you can run the following command from the root directory of this project:

$ yarn build

For convenience, we have prepared a single script that you can run to perform end-to-end tests locally. Don't forget to yarn build prior to running e2e tests:

$ yarn e2e

Note, it may be necessary to remove cached images when rebuilding the inventory app docker container, in order to ensure that the new usmetadata-app template is included in the build. If you want to make sure that you aren't using cached builds, you can try:

$ docker-compose build --no-cache --pull ckanext-dcat_usmetadata_app

With the dcat_usmetadata extension running in the inventory app, use the following command to run e2e tests:

$ npx cypress run

To run e2e tests interactively use:

$ npx cypress open

Publishing a new version of the extension

We publish this extension to PyPI - https://pypi.org/project/ckanext-dcat-usmetadata/. This is done by CI job that is triggered on tagged commit on master branch. When you need to release a new version of the extension, you need to:

0. Create a new branch for releasing a new version of the extension. You can name your branch with the following convention: release/x.y.z;
1. Update version in setup.py;
2. Get your PR merged to master branch;
3. Using GitHub UI, create a new release which will trigger a build and deploy jobs.

In the CI job, the following is done:

• It builds the JS bundles and puts them into the relevant directory so the extension can use them;
• It runs integration tests to make sure everything is working as expected;
• It packages the extension and publishes it to PyPI.

Below is a sequence diagram demonstrating the flow (you need to have github + mermaid chrome extension to view it):

sequenceDiagram
    Developer->>Git: Push tagged commit to master branch
    Git-->>CI/CD: Trigger deployment
    CI/CD-->>CI/CD: Build assets (JS bundles)
    CI/CD-->>CI/CD: Build python package
    CI/CD-->>CI/CD: Run tests
    CI/CD-->>PyPI: Publish the package
    Inventory-->>PyPI: Install

Ways to Contribute

The Data.gov team manages all Data.gov updates, bugs, and feature additions via GitHub's public issue tracker in this repository.

If you do not already have a GitHub account, you can sign up for GitHub here. In the spirit of open source software, everyone is encouraged to help improve this project. Here are some ways you can contribute:

• by reporting bugs
• by suggesting new features
• by translating content to a new language
• by writing or editing documentation
• by writing specifications
• by writing code and documentation (no pull request is too small: fix typos, add code comments, clean up inconsistent whitespace)
• by reviewing pull requests.
• by closing issues

Submit Great Issues

• Before submitting a new issue, check to make sure a similar issue isn't already open. If one is, contribute to that issue thread with your feedback.
• When submitting a bug report, please try to provide as much detail as possible, i.e. a screenshot or gist that demonstrates the problem, the technology you are using, and any relevant links.

Ready for your Help

Issues labeled help wanted make it easy for you to find ways you can contribute today.

Public Domain

This project constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105. Additionally, we waive copyright and related rights in the work worldwide through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.