databricks-client 0.0.1

REST client for Databricks

databricks_client

About

A REST client for Databricks.

This open-source project is not developed by nor affiliated with Databricks.

Installing

pip install databricks_client

Usage

import databricks_client

client = databricks_client.create("https://northeurope.azuredatabricks.net/api/2.0")
client.auth_pat_token(pat_token)
clusters_list = client.get('clusters/list')
for cluster in clusters_list["clusters"]:
    print(cluster)

Usage with Azure Active Directory

Note: Azure AD authentication for Databricks is currently in preview.

The client generates short-lived Azure AD tokens. If you need to use your client for longer than the lifetime (typically 30 minutes), rerun client.auth_azuread periodically.

Azure AD authentication with Azure CLI

Install the Azure CLI.

pip install databricks_client[azurecli]
az login

import databricks_client

client = databricks_client.create("https://northeurope.azuredatabricks.net/api/2.0")
client.auth_azuread("/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-workspace")
# or client.auth_azuread(resource_group="my-rg", workspace_name="my-workspace")
clusters_list = client.get('clusters/list')
for cluster in clusters_list["clusters"]:
    print(cluster)

This is recommended with Azure DevOps Pipelines using the Azure CLI task.

Azure AD authentication with ADAL

pip install databricks_client
pip install adal

import databricks_client
import adal

authority_host_uri = 'https://login.microsoftonline.com'
authority_uri = authority_host_uri + '/' + tenant_id
context = adal.AuthenticationContext(authority_uri)

def token_callback(resource):
    return context.acquire_token_with_client_credentials(resource, client_id, client_secret)["accessToken"]

client = databricks_client.create("https://northeurope.azuredatabricks.net/api/2.0")
client.auth_azuread("/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-workspace", token_callback)
# or client.auth_azuread(resource_group="my-rg", workspace_name="my-workspace", token_callback=token_callback)
clusters_list = client.get('clusters/list')
for cluster in clusters_list["clusters"]:
    print(cluster)

Example usages

Generating a PAT token

response = client.post(
    'token/create',
    json={"lifetime_seconds": 60, "comment": "Unit Test Token"}
)
pat_token = response['token_value']

Uploading a notebook

import base64

with open(notebook_file, "rb") as f:
    file_content = f.read()

client.post(
    'workspace/import',
    json={
        "content": base64.b64encode(file_content).decode('ascii'),
        "path": notebook_path,
        "overwrite": False,
        "language": "PYTHON",
        "format": "SOURCE"
    }
)