Software tools used in the management of PDS4 Sub-models (formerly known as PDS4 Data Dictionaries).
PDS4 Sub-Model Management
This repo contains tools and software needed for managing PDS4 Sub-Models (formerly known as Data Dictionaries)
Include any system-wide requirements (
yum install, …) Python 3 should be used regardless as Python 2 reached end-of-life on January 1st, 2020.
pip install my_pds_module
If possible, make it so that your program works out of the box without any additional configuration—but see the Configuration section for details.
To execute, run:
(put your run commands here)
Code of Conduct
All users and developers of the NASA-PDS software are expected to abide by our Code of Conduct. Please read this to ensure you understand the expectations of our community.
To develop this project, use your favorite text editor, or an integrated development environment with Python support, such as PyCharm.
For information on how to contribute to NASA-PDS codebases please take a look at our Contributing guidelines.
Install in editable mode and with extra developer dependencies into your virtual environment of choice:
pip install --editable '.[dev]'
pre-commit install && pre-commit install -t pre-push
To isolate and be able to re-produce the environment for this package, you should use a Python Virtual Environment. To do so, run:
python -m venv venv
Then exclusively use
venv/bin/pip, etc. (It is no longer recommended to use
If you have
tox installed and would like it to create your environment and install dependencies for you run:
tox --devenv <name you'd like for env> -e dev
Dependencies for development are specified as the
setup.cfg; they are installed into the virtual environment as follows:
pip install --editable '.[dev]'
All the source code is in a sub-directory under
You should update the
setup.cfg file with:
- name of your module
- license, default apache, update if needed
- download url, when you release your package on github add the url here
- install_requires, add the dependencies of you package
- extras_require, add the development Dependencies of your package
- entry_points, when your package can be called in command line, this helps to deploy command lines entry points pointing to scripts in your package
For the packaging details, see https://packaging.python.org/tutorials/packaging-projects/ as a reference.
It is convenient to use ConfigParser package to manage configuration. It allows a default configuration which can be overwritten by the user in a specific file in their environment. See https://pymotw.com/2/ConfigParser/
candidates = ['my_pds_module.ini', 'my_pds_module.ini.default'] found = parser.read(candidates)
You should not use
print()vin the purpose of logging information on the execution of your code. Depending on where the code runs these information could be redirected to specific log files.
To make that work, start each Python file with:
import logging logger = logging.getLogger(__name__)
To log a message:
main routine, include:
to get a basic logging system configured.
extras_require included in the template repo installs
flake8 (plus some plugins), and
mypy along with default configuration for all of them. You can run all of these (and more!) with:
tox -e lint
So that your code is readable, you should comply with the PEP8 style guide. Our code style is automatically enforced in via black and flake8. See the Tooling section for information on invoking the linting pipeline.
❗Important note for template users❗
The included pre-commit configuration file executes
flake8 (along with
mypy) across the entire
src folder and not only on changed files. If you're converting a pre-existing code base over to this template that may result in a lot of errors that you aren't ready to deal with.
You can instead execute
flake8 only over a diff of the current changes being made by modifying the
entry: git diff -u | flake8 --diff
Or you can change the
pre-commit config so
flake8 is only called on changed files which match a certain filtering criteria:
- repo: local hooks: - id: flake8 name: flake8 entry: flake8 files: ^src/|tests/ language: system
Python offers a large variety of libraries. In PDS scope, for the most current usage we should use:
|configparser||manage and parse configuration files|
|argparse||command line argument documentation and parsing|
|requests||interact with web APIs|
|lxml||read/write XML files|
|json||read/write JSON files|
|pyyaml||read/write YAML files|
|pystache||generate files from templates|
Some of these are built into Python 3; others are open source add-ons you can include in your
This section describes testing for your package.
A complete "build" including test execution, linting (
flake8, etc.), and documentation build is executed via:
Your project should have built-in unit tests, functional, validation, acceptance, etc., tests.
For unit testing, check out the unittest module, built into Python 3.
Tests objects should be in packages
test modules or preferably in project 'tests' directory which mirrors the project package structure.
Our unit tests are launched with command:
If you want your tests to run automatically as you make changes start up
pytest in watch mode with:
One should use the
behave package and push the test results to "testrail".
Your project should use Sphinx to build its documentation. PDS' documentation template is already configured as part of the default build. You can build your projects docs with:
python setup.py build_sphinx
You can access the build files in the following directory relative to the project root:
pip install wheel python setup.py sdist bdist_wheel
NASA PDS packages can publish automatically using the Roundup Action, which leverages GitHub Actions to perform automated continuous integration and continuous delivery. A default workflow that includes the Roundup is provided in the
.github/workflows/unstable-cicd.yaml file. (Unstable here means an interim release.)
Create the package:
python setup.py bdist_wheel
Publish it as a Github release.
Publish on PyPI (you need a PyPI account and configure
pip install twine twine upload dist/*
Or publish on the Test PyPI (you need a Test PyPI account and configure
pip install twine twine upload --repository testpypi dist/*
The template repository comes with our two "standard" CI/CD workflows,
unstable-cicd. The unstable build runs on any push to
main (± ignoring changes to specific files) and the stable build runs on push of a release branch of the form
release/<release version>. Both of these make use of our GitHub actions build step, Roundup. The
unstable-cicd will generate (and constantly update) a SNAPSHOT release. If you haven't done a formal software release you will end up with a
v0.0.0-SNAPSHOT release (see NASA-PDS/roundup-action#56 for specifics).
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for pds.sumo_tools-0.2.6-py3-none-any.whl