Prophecy-build-tool (PBT) provides utilities to build and distribute projects created from the Prophecy IDE.
Project description
prophecy-build-tool
Prophecy is designed to enable all users to be productive with data engineering. It also replaces legacy ETL products. To learn more about Prophecy visit https://docs.prophecy.io/.
Prophecy-built-tool (PBT) allows you to quickly build projects generated by Prophecy (your standard Spark Scala and PySpark pipelines) to integrate them with your own CI / CD (e.g. Github Actions), build system (e.g. Jenkins), and orchestration (e.g. Databricks Workflows).
Installation
To install PBT, simply run:
pip3 install prophecy-build-tool
Running
To build and deploy your Prophecy project containing Python projects and Databricks Jobs run
pbt deploy --path /path/to/your/prophecy_project/
Sample output:
Prophecy-build-tool v1.0.0
Found 1 pipelines: customers_orders (python)
Found 1 jobs: daily_report
Building 1 pipeline 🚰
Building pipeline pipelines/customers_orders [1/1]
running build
running build_py
✅ Build complete!
Deploying 1 job ⏱
Deploying job jobs/daily_report [1/1]
Uploading cs-1.0-py3-none-any.whl to dbfs:/FileStore/prophecy/artifacts/...
Updating an existing job: daily_report
✅ Deployment complete!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for prophecy-build-tool-test-1.0.4.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | da6efcaca52ecee4cd72036c155e90e40af35d20964ef9328184eeb5371404d7 |
|
MD5 | 8ec84342b2a27e4ace6e806e1e36922e |
|
BLAKE2b-256 | 5538a259d6ded01cea9cc41243b54f2edb6350f6fb6ac315d041eb50b84c3e78 |
Hashes for prophecy_build_tool_test-1.0.4.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b19c0f0675aaf36edb3437e61d144bc28b34ba8dccb6e730fcc6421159d8e5e2 |
|
MD5 | 0a4a63f637a9084d7b23b4616f1522d9 |
|
BLAKE2b-256 | 920edbbde2b4e277480dbb868b3a760785279375313c7a25cd70b03b93b84a3d |