prophecy-build-tool-test 1.0.4.5

Prophecy-build-tool (PBT) provides utilities to build and distribute projects created from the Prophecy IDE.

prophecy-build-tool

Prophecy is designed to enable all users to be productive with data engineering. It also replaces legacy ETL products. To learn more about Prophecy visit https://docs.prophecy.io/.

Prophecy-built-tool (PBT) allows you to quickly build projects generated by Prophecy (your standard Spark Scala and PySpark pipelines) to integrate them with your own CI / CD (e.g. Github Actions), build system (e.g. Jenkins), and orchestration (e.g. Databricks Workflows).

Installation

To install PBT, simply run:

pip3 install prophecy-build-tool

Running

To build and deploy your Prophecy project containing Python projects and Databricks Jobs run

pbt deploy --path /path/to/your/prophecy_project/

Sample output:

Prophecy-build-tool v1.0.0

Found 1 pipelines: customers_orders (python)
Found 1 jobs: daily_report

Building 1 pipeline 🚰

  Building pipeline pipelines/customers_orders [1/1]
    running build
    running build_py

✅ Build complete!

Deploying 1 job ⏱

  Deploying job jobs/daily_report [1/1]
    Uploading cs-1.0-py3-none-any.whl to dbfs:/FileStore/prophecy/artifacts/...
    Updating an existing job: daily_report

✅ Deployment complete!