dbxio 0.1.0

High-level Databricks client

dbxio: High-level Databricks client

Overview

dbxio is a high-level client for Databricks that simplifies working with tables and volumes. It provides a simple interface for reading and writing data, creating and deleting objects, and running SQL queries and fetching results.

Installation

dbxio requires Python 3.9 or later. You can install dbxio using pip:

pip install dbxio

dbxio by Example

import dbxio

client = dbxio.DbxIOClient.from_cluster_settings(
   cluster_type=dbxio.ClusterType.SQL_WAREHOUSE,
   http_path='sql/protocolv1/o/2350007385231210/abcdefg-12345',
   server_hostname='adb-1234567890.00.azuredatabricks.net',
)

# read table
table = list(dbxio.read_table('catalog.schema.table', client=client))

# write table
data = [
   {'col1': 1, 'col2': 'a', 'col3': [1, 2, 3]},
   {'col1': 2, 'col2': 'b', 'col3': [4, 5, 6]},
]
schema = dbxio.TableSchema(
   [
      {'name': 'col1', 'type': dbxio.types.IntType()},
      {'name': 'col2', 'type': dbxio.types.StringType()},
      {'name': 'col3', 'type': dbxio.types.ArrayType(dbxio.types.IntType())},
   ]
)
dbxio.bulk_write_table(
   dbxio.Table('domain.schema.table', schema=schema),
   data,
   client=client,
   abs_name='blob_storage_name',
   abs_container_name='container_name',
   append=True,
)

Cloud Support

dbxio supports the following cloud providers:

• Azure
• Nebius over Azure
• AWS (will be added soon)
• GCP (will be added soon)

Project Information

• Docs
• PyPI
• Contributing