DBT workflows factory for Google Cloud Platform
Project description
dbt-workflows-factory
Creates dbt based GCP workflows.
Installation
Use the package manager pip to install dbt-workflows-factory for [dp (data-pipelines-cli)]:
pip install dbt-workflows-factory
How to run
To call from cli, you can
python -m dbt_workflows_factory.cli convert \
--image-uri my-image-uri \
--gcs-key-volume-remote-path google-cloud-storage/path/ \
--gcs-key-volume-mount-path /etc/gcs-key/ \
--gcs-key-volume-container-mount-path /etc/gcs-key/:/etc/gcs-key/:ro \
--container-gcp-key-path /etc/gcs-key/path.json \
--container-gcp-project-id some-project-id \
--pretty \
tests/unit/dbt_workflows_factory/test_data/manifest.json > workflow_source.json
Project Organization
- .devcontainer - This directory contains required files for creating a Codespace.
- .github
- workflows - Contains GitHub Actions used for building, testing and publishing.
- publish.yml - Publish wheels to https://pypi.org/
- pull-request.yml - Build and Test pull requests before commiting to main.
- template-sync.yml - Update GitHub Repo with enhancments to base template
- workflows - Contains GitHub Actions used for building, testing and publishing.
- docs - collect documents (default format .md)
- src - place new source code here
- python_package - sample package (this can be deleted when creating a new repository)
- tests - contains Python based test cases to validation src code
- .pre-commit-config.yaml - Contains various pre-check fixes for Python
- pyproject.toml - Python Project Declaration
Publish to PyPi from GitHub
In order to publish to PyPi, a repostirory secret must be created, "PYPI_PASSWORD". In order to publish to the Test PyPi, a second secret must be added, "TEST_PYPI_PASSWORD".
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file dbt-workflows-factory-0.2.0.tar.gz
.
File metadata
- Download URL: dbt-workflows-factory-0.2.0.tar.gz
- Upload date:
- Size: 17.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a871c46a87af75917583251bbedfef277ce926ede0b08f3be595853a156bbed |
|
MD5 | 218c842b48ad10b60c8a3cc3fd7d6f0b |
|
BLAKE2b-256 | 2a2a64a02d16fb9a314eb461c000ab3e471487c6175ab4720a1069ec9886387a |
File details
Details for the file dbt_workflows_factory-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: dbt_workflows_factory-0.2.0-py3-none-any.whl
- Upload date:
- Size: 16.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7f5806a442d841e6cb2a220a300eeb38f8f123f15dd33f9ba14de252121d804 |
|
MD5 | 7621145c60f89af60cdd1831b8d66765 |
|
BLAKE2b-256 | 3fdb46662453b1fd941db13e0a7b33ac392092e20818bf235bb770147b89f380 |