Package for ingesting Tableau metadata into Google Cloud Data Catalog
Project description
google-datacatalog-tableau-connector
Package for ingesting Tableau metadata into Google Cloud Data Catalog, currently supporting below asset types:
- Workbook
- Sheet
- Dashboard
Disclaimer: This is not an officially supported Google product.
Table of Contents
- 1. Installation
- 2. Environment setup
- 3. Running the connector
- 4. Set up a Tableau demo server
- 5. Developer environment
- 7. Troubleshooting
1. Installation
Install this library in a virtualenv using pip. virtualenv is a tool to create isolated Python environments. The basic problem it addresses is one of dependencies and versions, and indirectly permissions.
With virtualenv, it's possible to install this library without needing
system install permissions, and without clashing with the installed system
dependencies. Make sure you use Python 3.6+
.
1.1. Mac/Linux
pip3 install virtualenv
virtualenv --python python3.6 <your-env>
source <your-env>/bin/activate
<your-env>/bin/pip install google-datacatalog-tableau-connector
1.2. Windows
pip3 install virtualenv
virtualenv --python python3.6 <your-env>
<your-env>\Scripts\activate
<your-env>\Scripts\pip.exe install google-datacatalog-tableau-connector
1.3. Install from source
1.3.1. Get the code
git clone https://github.com/GoogleCloudPlatform/datacatalog-connectors-bi/
cd datacatalog-connectors-bi/google-datacatalog-tableau-connector
1.3.2. Create and activate a virtualenv
pip3 install virtualenv
virtualenv --python python3.6 <your-env>
source <your-env>/bin/activate
1.3.3. Install the library
pip install .
2. Environment setup
2.1. Auth credentials
2.1.1. Create a service account and grant it below roles
- Data Catalog Admin
2.1.2. Download a JSON key and save it as
<YOUR-CREDENTIALS_FILES_FOLDER>/tableau2dc-datacatalog-credentials.json
Please notice this folder and file will be required in next steps.
2.2. Set environment variables
Replace below values according to your environment:
export GOOGLE_APPLICATION_CREDENTIALS=data_catalog_credentials_file
export TABLEAU2DC_TABLEAU_SERVER=tableau_server
export TABLEAU2DC_TABLEAU_API_VERSION=tableau_api_version
export TABLEAU2DC_TABLEAU_USERNAME=tableau_username
export TABLEAU2DC_TABLEAU_PASSWORD=tableau_password
export TABLEAU2DC_TABLEAU_SITE=tableau_site (optional)
export TABLEAU2DC_DATACATALOG_PROJECT_ID=google_cloud_project_id
Replace above values according to your environment. The Data Catalog credentials file was saved in step 2.1.2.
3. Running the connector
3.1. Python entry point
- Virtualenv
google-datacatalog-tableau-connector \
--tableau-server $TABLEAU2DC_TABLEAU_SERVER \
--tableau-api-version $TABLEAU2DC_TABLEAU_API_VERSION \
--tableau-username $TABLEAU2DC_TABLEAU_USERNAME \
--tableau-password $TABLEAU2DC_TABLEAU_PASSWORD \
[--tableau-site $TABLEAU2DC_TABLEAU_SITE \]
--datacatalog-project-id $TABLEAU2DC_DATACATALOG_PROJECT_ID
3.2. Docker entry point
docker build --rm --tag tableau2datacatalog .
docker run --rm --tty -v YOUR-CREDENTIALS_FILES_FOLDER:/data \
tableau2datacatalog \
--tableau-server $TABLEAU2DC_TABLEAU_SERVER \
--tableau-api-version $TABLEAU2DC_TABLEAU_API_VERSION \
--tableau-username $TABLEAU2DC_TABLEAU_USERNAME \
--tableau-password $TABLEAU2DC_TABLEAU_PASSWORD \
[--tableau-site] $TABLEAU2DC_TABLEAU_SITE \]
--datacatalog-project-id $TABLEAU2DC_DATACATALOG_PROJECT_ID
4. Set up a Tableau demo server
To quickly set up a Tableau demo server, please visit https://www.tableau.com/developer.
Click on SIGN UP FOR THE TABLEAU DEVELOPER PROGRAM. Once you have signed up you will receive an e-mail with subject Tableau Online Developer - Activate your Site.
In the e-mail contents click on: Activate My Developer Site. Once you've done that you will receive another e-mail with subject You've Successfully Created Your Site.
Then you will be able to use your Tableau Online dev server.
5. Developer environment
5.1. Install and run Yapf formatter
pip install --upgrade yapf
# Auto update files
yapf --in-place --recursive src tests
# Show diff
yapf --diff --recursive src tests
# Set up pre-commit hook
# From the root of your git project.
curl -o pre-commit.sh https://raw.githubusercontent.com/google/yapf/master/plugins/pre-commit.sh
chmod a+x pre-commit.sh
mv pre-commit.sh .git/hooks/pre-commit
6.2. Install and run Flake8 linter
pip install --upgrade flake8
flake8 src tests
6.3. Run Tests
python setup.py test
6.4. Additional resources
Please refer to the Developer Resources documentation.
7. Troubleshooting
In the case a connector execution hits Data Catalog quota limit, an error will be raised and logged with the following detailment, depending on the performed operation READ/WRITE/SEARCH:
status = StatusCode.RESOURCE_EXHAUSTED
details = "Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute' of service 'datacatalog.googleapis.com' for consumer 'project_number:1111111111111'."
debug_error_string =
"{"created":"@1587396969.506556000", "description":"Error received from peer ipv4:172.217.29.42:443","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute' of service 'datacatalog.googleapis.com' for consumer 'project_number:1111111111111'.","grpc_status":8}"
For more information on Data Catalog quota, please refer to: Data Catalog quota docs
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for google-datacatalog-tableau-connector-0.6.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 375de8768be9bf81d99c741706a55045f6f6100a7538d7a89ad750e28ef3355d |
|
MD5 | 5e5e21e71e3cbb121e5e55afc215506a |
|
BLAKE2b-256 | e5779dde5c82f9c25bac2439dc6b6c1915d8815d16eed6f8d15b392e9c2c7c26 |
Hashes for google_datacatalog_tableau_connector-0.6.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df1b72b1be16eedf628073f5ea1b442b9bf3b8e0612ca3932cb5e9f537a478a7 |
|
MD5 | 430366f884a562ad85991844a4095dde |
|
BLAKE2b-256 | 31e654abff02680ac1f698108395981e3f802c662651ba660cb4c3c13c4702d0 |