Skip to main content

DBT workflows factory for Google Cloud Platform

Project description

dbt-workflows-factory

Python Version PyPI Version Downloads

Creates dbt based GCP workflows.

Installation

Use the package manager pip to install dbt-workflows-factory for [dp (data-pipelines-cli)]:

pip install dbt-workflows-factory

How to run

To call from cli, you can

python -m dbt_workflows_factory.cli convert \
    --image-uri my-image-uri \
    --gcs-key-volume-remote-path google-cloud-storage/path/ \
    --gcs-key-volume-mount-path /etc/gcs-key/ \
    --gcs-key-volume-container-mount-path /etc/gcs-key/:/etc/gcs-key/:ro \
    --container-gcp-key-path /etc/gcs-key/path.json \
    --container-gcp-project-id some-project-id \
    --pretty \
    tests/unit/dbt_workflows_factory/test_data/manifest.json > workflow_source.json

Project Organization

  • .devcontainer - This directory contains required files for creating a Codespace.
  • .github
    • workflows - Contains GitHub Actions used for building, testing and publishing.
      • publish.yml - Publish wheels to https://pypi.org/
      • pull-request.yml - Build and Test pull requests before commiting to main.
      • template-sync.yml - Update GitHub Repo with enhancments to base template
  • docs - collect documents (default format .md)
  • src - place new source code here
    • python_package - sample package (this can be deleted when creating a new repository)
  • tests - contains Python based test cases to validation src code
  • .pre-commit-config.yaml - Contains various pre-check fixes for Python
  • pyproject.toml - Python Project Declaration

Publish to PyPi from GitHub

In order to publish to PyPi, a repostirory secret must be created, "PYPI_PASSWORD". In order to publish to the Test PyPi, a second secret must be added, "TEST_PYPI_PASSWORD".

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt-workflows-factory-0.2.0.tar.gz (17.1 kB view details)

Uploaded Source

Built Distribution

dbt_workflows_factory-0.2.0-py3-none-any.whl (16.2 kB view details)

Uploaded Python 3

File details

Details for the file dbt-workflows-factory-0.2.0.tar.gz.

File metadata

  • Download URL: dbt-workflows-factory-0.2.0.tar.gz
  • Upload date:
  • Size: 17.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.4

File hashes

Hashes for dbt-workflows-factory-0.2.0.tar.gz
Algorithm Hash digest
SHA256 1a871c46a87af75917583251bbedfef277ce926ede0b08f3be595853a156bbed
MD5 218c842b48ad10b60c8a3cc3fd7d6f0b
BLAKE2b-256 2a2a64a02d16fb9a314eb461c000ab3e471487c6175ab4720a1069ec9886387a

See more details on using hashes here.

File details

Details for the file dbt_workflows_factory-0.2.0-py3-none-any.whl.

File metadata

File hashes

Hashes for dbt_workflows_factory-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f7f5806a442d841e6cb2a220a300eeb38f8f123f15dd33f9ba14de252121d804
MD5 7621145c60f89af60cdd1831b8d66765
BLAKE2b-256 3fdb46662453b1fd941db13e0a7b33ac392092e20818bf235bb770147b89f380

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page