Skip to main content

Databricks bundle for the Pyfony framework

Project description

Databricks bundle

This bundle allows you to create simple functional Databricks pipelines, which can be easily auto-documented and unit-tested. It is part of the Bricksflow framework.

alt text

Installation

Install the bundle via Poetry:

$ poetry add databricks-bundle && poetry add databricks-connect --dev && poetry add py4j --dev

Add the DatabricksBundle.autodetect() to your application's Kernel.py to activate it:

from pyfony.kernel.BaseKernel import BaseKernel
from databricksbundle.DatabricksBundle import DatabricksBundle

class Kernel(BaseKernel):
    
    def _registerBundles(self):
        return [
            # ...
            DatabricksBundle.autodetect(),
            # ...
        ]

Usage

  1. Writing functional pipelines
  2. Recommended pipelines structure
  3. Configuring pipelines
  4. Using dependencies
  5. Databricks Connect setup

Project details


Release history Release notifications | RSS feed

This version

0.5.7

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

databricks_bundle-0.5.7-py3-none-any.whl (25.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page