ssb-datadoc 0.2.5

Document dataset metadata. For use in Statistics Norway's metadata system.

Datadoc

Document datasets in Statistics Norway

Usage

From Jupyter

1. Open https://jupyter.dapla-staging.ssb.no or another Jupyter Lab environment
2. Datadoc comes preinstalled in Statistics Norway environments. Elsewhere, run Run pip install ssb-datadoc to install
3. Upload a dataset to your Jupyter server (e.g. https://github.com/statisticsnorway/datadoc/blob/master/klargjorte_data/befolkning/person_testdata_p2021-12-31_p2021-12-31_v1.parquet)
4. Run the demo.ipynb Notebook
5. Datadoc will open in the notebook

Contributing

Prerequisites

• Python >=3.10
• Poetry, install via curl -sSL https://install.python-poetry.org | python3 -

Dependency Management

Poetry is used for dependency management. Poe the Poet is used for running poe tasks within poetry's virtualenv. Upon cloning this project first install necessary dependencies, then run the tests to verify everything is working.

Install all dependencies

poetry install --all-extras

Add dependencies

Main

poetry add <python package name>

Dev

poetry add --group dev <python package name>

Run tests

poetry run poe test

Run project locally

To run the project locally:

poetry run poe datadoc

Run project locally in Jupyter

To run the project locally in Jupyter run:

poetry run poe jupyter

A Jupyter instance should open in your browser. Open and run the cells in the .ipynb file to demo datadoc.

Running the Dockerized Application Locally

docker run -p 8050:8050 \
-v $HOME/.config/gcloud/application_default_credentials.json/:/application_default_credentials.json \
-e GOOGLE_APPLICATION_CREDENTIALS="/application_default_credentials.json" \
datadoc

Bump version

poetry run poe bump-patch-version

:warning: Run this on the default branch

This command will:

1. Increment version strings in files
2. Commit the changes
3. Tag the commit with the new version

Then just run git push origin --tags to push the changes and trigger the release process.