A package to load user-specified metadata models into Google Cloud Data Catalog
Project description
datacatalog-custom-model-manager
A Python package intended to load user-specified metadata models into Google Cloud Data Catalog, comprising Custom Entries, Tag Templates, and Tags.
It is powered by datacatalog-custom-entries- manager, datacatalog- tag-template-processor, and datacatalog-tag-manager, leveraging their features in the format of a single CLI.
Load Custom Model
From CSV files
- SCHEMAS
Loading custom models from CSV requires a folder (--files-folder
argument in the below commands)
containing three files: entries.csv
, tag_templates.csv
, and tags.csv
. Please refer to the
following external resources for further details on their layouts:
FILE | LAYOUT DOCUMENTATION |
---|---|
entries.csv |
datacatalog-custom-entries-manager README |
tag_templates.csv |
datacatalog-tag-template-processor README |
tags.csv |
datacatalog-tag-manager README |
- SAMPLE INPUT
- sample-input/egeria-business-glossary for reference;
- Sample metadata for datacatalog-custom-model-manager (Google Sheets) might help to create/export the mandatory CSV files.
- COMMANDS
Python + virtualenv
pip install datacatalog-custom-model-manager
datacatalog-custom-model load \
--files-folder <CSV-FILES-PATH> \
--project-id <YOUR-PROJECT-ID> --location-id <YOUR-LOCATION-ID> \
[--delete-existing-tag-templates]
Docker
docker build --rm --tag datacatalog-custom-model-manager .
docker run --rm --tty \
--volume <CREDENTIALS-FILE-FOLDER>:/credentials --volume <CSV-FILE-FOLDER>:/data \
datacatalog-custom-model-manager load \
--files-folder <CSV-FILES-PATH> \
--project-id <YOUR-PROJECT-ID> --location-id <YOUR-LOCATION-ID> \
[--delete-existing-tag-templates]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for datacatalog-custom-model-manager-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a80a50d7c5114c36ad9644d79680818007c05104df1c3899aa5832cee87a8052 |
|
MD5 | 83c53ccdd2c627b20508ecf964fca126 |
|
BLAKE2b-256 | 71ce9cff321d80b676e72221e6fe228a081a62eabd6650b9df3b0f001ff8527d |
Hashes for datacatalog_custom_model_manager-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b413912e9b00eb0dcc83f96bfea8d451e7eb7054eb21aa095d462b2edad64d9a |
|
MD5 | 38e9b6e1b6670c808681ff00ef8d3962 |
|
BLAKE2b-256 | fd3d27c3cfc70590d20387a382708568d77309b782248bc67e95dee25f1c97c0 |