DBT workflows factory for Google Cloud Platform
Project description
dbt-workflows-factory
Creates dbt based GCP workflows.
Installation
Use the package manager pip to install dbt-workflows-factory for [dp (data-pipelines-cli)]:
pip install dbt-workflows-factory
How to run
To call from cli, you can
python -m dbt_workflows_factory.cli convert \
--image-uri my-image-uri \
--gcs-key-volume-remote-path google-cloud-storage/path/ \
--gcs-key-volume-mount-path /etc/gcs-key/ \
--gcs-key-volume-container-mount-path /etc/gcs-key/:/etc/gcs-key/:ro \
--container-gcp-key-path /etc/gcs-key/path.json \
--container-gcp-project-id some-project-id \
--pretty \
tests/unit/dbt_workflows_factory/test_data/manifest.json > workflow_source.json
Project Organization
- .devcontainer - This directory contains required files for creating a Codespace.
- .github
- workflows - Contains GitHub Actions used for building, testing and publishing.
- publish.yml - Publish wheels to https://pypi.org/
- pull-request.yml - Build and Test pull requests before commiting to main.
- template-sync.yml - Update GitHub Repo with enhancments to base template
- workflows - Contains GitHub Actions used for building, testing and publishing.
- docs - collect documents (default format .md)
- src - place new source code here
- python_package - sample package (this can be deleted when creating a new repository)
- tests - contains Python based test cases to validation src code
- .pre-commit-config.yaml - Contains various pre-check fixes for Python
- pyproject.toml - Python Project Declaration
Publish to PyPi from GitHub
In order to publish to PyPi, a repostirory secret must be created, "PYPI_PASSWORD". In order to publish to the Test PyPi, a second secret must be added, "TEST_PYPI_PASSWORD".
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for dbt-workflows-factory-0.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a871c46a87af75917583251bbedfef277ce926ede0b08f3be595853a156bbed |
|
MD5 | 218c842b48ad10b60c8a3cc3fd7d6f0b |
|
BLAKE2b-256 | 2a2a64a02d16fb9a314eb461c000ab3e471487c6175ab4720a1069ec9886387a |
Close
Hashes for dbt_workflows_factory-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7f5806a442d841e6cb2a220a300eeb38f8f123f15dd33f9ba14de252121d804 |
|
MD5 | 7621145c60f89af60cdd1831b8d66765 |
|
BLAKE2b-256 | 3fdb46662453b1fd941db13e0a7b33ac392092e20818bf235bb770147b89f380 |