Pyspark tools for everyday use
Project description
pyspark-me
Pyspark and Databricks tools for everyday life
Synopsis
Create Databricks connection
# Get Databricks workspace connection
dbc = pysparkme.databricks.connect(
bearer_token='dapixyzabcd09rasdf',
url='https://westeurope.azuredatabricks.net')
Databricks workspace
# List root workspace directory
dbc.workspace.ls('/')
# Check if workspace item exists
dbc.workspace.exists('/explore')
# Check if workspace item is a directory
dbc.workspace.is_directory('/')
# Export notebook in default (SOURCE) format
dbc.workspace.export('/my_notebook')
# Export notebook in HTML format
dbc.workspace.export('/my_notebook', 'HTML')
Databricks CLI
Get CLI help
python -m pysparkme.databricks.cli --help
Export the whole Databricks workspace into a directory explore/export.
Databricks token is taken from DATABRICKS_BEARER_TOKEN environment variable.
python -m pysparkme.databricks.cli workspace export -o explore/export ''
Build and publish
python setup.py sdist bdist_wheel
python -m twine upload dist/*
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyspark-me-0.0.4.tar.gz
(7.6 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pyspark-me-0.0.4.tar.gz.
File metadata
- Download URL: pyspark-me-0.0.4.tar.gz
- Upload date:
- Size: 7.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.7.6rc1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c1d51837c07b25d3846db604c5b5d6d2846f42906ebf7fa49d7a0b69beb6e733
|
|
| MD5 |
009b4280452666ba0d9c2a7418f16793
|
|
| BLAKE2b-256 |
613f69a593e45a9ea062f6f8c3efab21c9de3ce86ec50f80b88fbd0ab4abda87
|
File details
Details for the file pyspark_me-0.0.4-py3-none-any.whl.
File metadata
- Download URL: pyspark_me-0.0.4-py3-none-any.whl
- Upload date:
- Size: 18.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.7.6rc1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0688b7ceebf2c1505d7126595a04a527d01056be9ad0b1ee5f1690d7bbd9f1b1
|
|
| MD5 |
36ddeff0629f770782b2295d075fdfd5
|
|
| BLAKE2b-256 |
66a505a2d1def183cb985a62d24f1dacbc03cb3e0ba6585c9e5e4f6358687583
|
