Prophecy-build-tool (PBT) provides utilities to build and distribute projects created from the Prophecy IDE.
Project description
prophecy-build-tool
Prophecy is designed to enable all users to be productive with data engineering. It also replaces legacy ETL products. To learn more about Prophecy visit https://docs.prophecy.io/.
Prophecy-built-tool (PBT) allows you to quickly build projects generated by Prophecy (your standard Spark Scala and PySpark pipelines) to integrate them with your own CI / CD (e.g. Github Actions), build system (e.g. Jenkins), and orchestration (e.g. Databricks Workflows).
Installation
To install PBT, simply run:
pip install prophecy-build-tool
Running
To build and deploy your Prophecy project containing Python projects and Databricks Jobs run
pbt deploy --path /path/to/your/prophecy_project/
Sample output:
Prophecy-build-tool v1.0.0
Found 1 pipelines: customers_orders (python)
Found 1 jobs: daily_report
Building 1 pipeline 🚰
Building pipeline pipelines/customers_orders [1/1]
running build
running build_py
✅ Build complete!
Deploying 1 job ⏱
Deploying job jobs/daily_report [1/1]
Uploading cs-1.0-py3-none-any.whl to dbfs:/FileStore/prophecy/artifacts/...
Updating an existing job: daily_report
✅ Deployment complete!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for prophecy-build-tool-test-1.0.4.8.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 20ffdacce7e4e496ae32f245fabd9637275b315f407e94b951dc4937367a6e6c |
|
MD5 | 88b4a265447bfd37841df397127a9817 |
|
BLAKE2b-256 | 955018e2188b513cbfdfd7093a9ffea90c26fc96f2862108bcc785802d677a6c |
Hashes for prophecy_build_tool_test-1.0.4.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8d4ff7d22e61b6ecd36f06f93410bc7674cf7842c5de041224304b682a25a01c |
|
MD5 | 8ba97df4e298eb25a34024d28c2f16af |
|
BLAKE2b-256 | d864ec2e8677230e11289dfcf9ea9213cba03699ae66b04a87c00cfb0b08e601 |