Skip to main content

A Python package for keeping track of your data processing steps

Project description

PyPI version Documentation Status Downloads License: MIT

Command line provenance

cmdline_provenance is a Python package for keeping track of your data processing steps.

It was inspired by the popular NCO and CDO command line tools, which automatically generate a record of what was executed at the command line, append that record to the history attribute from the input (netCDF) data file, and then set the new extended record as the history attribute of the output (netCDF) data file.

For example, after selecting the 2001-2005 time period from a rainfall data file and then deleting the long_name file attribute, the command log would look as follows:

Fri Dec 08 10:05:47 2017: ncatted -O -a long_name,pr,d,,
Fri Dec 01 07:59:16 2017: cdo seldate,2001-01-01,2005-12-31

Following this simple approach to data provenance, it is possible maintain a record of all data processing steps from intial download/creation of your data files to the end result (e.g. a .png image).

cmdline_provenance contains a series of functions for generating history records in the NCO/CDO format, and for combining the current record with previous records to maintain a complete command log.



pip install cmdline-provenance


conda install cmdline_provenance

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for cmdline-provenance, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size cmdline_provenance-1.0.0-py3-none-any.whl (5.6 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size cmdline_provenance-1.0.0.tar.gz (4.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page