Keep your local python scripts installed and in sync with a databricks notebook. Shortens the feedback loop to develop projects using a hybrid enviroment
Project description
DB-Rocket
Keep your local python scripts installed and in sync with a databricks notebook. Every change on your local machine is automatically applied to the notebook. Shortens the feedback loop to develop git based projects. Removes the need to setup a local development environment.
Installation
pip3 install databricks-rocket
Please sure your python interpreter is 3.7 or higher.
Setup
Make sure you have a token installed for databricks cli (offical documentation). First, create a token for yourself on Databricks. Then set it up locally by running:
databricks configure --token
Then, the databricks token must be exported in your environment.
export DATABRICKS_TOKEN="mydatabrickstoken"
If your project is not a pip package already you have to turn it into one. You can use dbrocket to do that.
dbrocket setup
Will create a setup.py for you.
Using db-rocket
dbrocket launch
The command returns the exact command you have to perform in your notebook next.
Example:
We are now building your Python repo as a library...
Done! in your notebook install the library by running:
%pip install --upgrade pip
%pip install /dbfs/temp/username/databricks_rocket-1.1.3-py3-none-any.whl --force-reinstall
Create a cell in a notebook and paste the content (example below).
Support
- Databricks: >=7
- Python: >=3.7
- Tested on Platform: Linux, MacOs. Windows will probably not work but contributions are welcomed!
Acknowledgments
- Thanks Leon Poli for the Logo :)
- Thanks Stephane Leonard for source-code and documentation improvements :)
- Thanks Malachi Soord for the CICD setup and README improvements
Contributions are welcomed!
Security
For security issues please contact security@getyourguide.com.
Legal
db-rocket is licensed under the Apache License, Version 2.0. See LICENSE for the full text.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for databricks_rocket-1.3.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 76429e151175feeacaabefba43591df8d15d859e69c61ba9ba76b5d0551eea33 |
|
MD5 | 7951529944faad45ef46e87936d6d4ef |
|
BLAKE2b-256 | 40983201e3817011aa253f032aa1d131b045b41a2d3967642d95e0f6e28f8e16 |