Skip to main content

Utility for moving data between CKAN instances using the API

Project description

Overview

Utility to migrate data between two CKAN instances using the CKAN api.

Getting Started

config/transformationConfig.json

This file currently contains the configuration that is being used at DataBC for the transformation between prod and test. A detailed description of this file is available here

Running

Env vars:

Configure the following env vars:

  • export CKAN_API_KEY_DEST=<api key for the destination instance, should be for a ckan sysadmin user>
  • export CKAN_URL_TST=
  • export CKAN_API_KEY_SRC=<api key for source destination instance, should be sysadmin>
  • export CKAN_URL_SRC=
  • export CKAN_DO_NOT_WRITE_URL=<usually the source instance, adds checks to make no methods that make changes are not being called on this instance>
  • export CKAN_NEW_USER_PSWD=
  • export CKAN_TRANSFORMATION_CONFIG=

Optional env vars, These are optional vars that should NOT be used in production/ deployed versions of this code. They are parameters that help with the debugging of this code.

  • export LOG_FILE_PATH= if LOG_FILE_PATH is not set then logging will only go to console.
  • export DUMP_DEBUG_DATA= When the debug param is set the objects returned by the rest api get cached. Subsequent runs of the script will re-use cached objects. Also dumps comparison object data to help debug issues with change control.

Finally, environment variables are defined in the constants, making them easy to change

Running

After the module has been installed:

pip install bcdc2bcdc

and the the env vars described above have been set, you can run the script using

python3 runBCDC2BCDC.py

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bcdc2bcdc-0.0.10.tar.gz (69.4 kB view hashes)

Uploaded source

Built Distribution

bcdc2bcdc-0.0.10-py3-none-any.whl (89.5 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page