vdk-impala 0.3.604201902

Versatile Data Kit SDK plugin provides support for Impala database.

This plugin allows vdk-core to interface with and execute queries against an Impala database.

Usage

Run

pip install vdk-impala

After this, data jobs will have access to a Impala database connection, managed by Versatile Data Kit SDK.

If it is the only database plugin installed , vdk would automatically use it. Otherwise, users need to set VDK_DB_DEFAULT_TYPE=IMPALA as an environment variable or set 'db_default_type' option in the data job config file (config.ini).

For example

    def run(job_input: IJobInput):
        job_input.execute_query("select 'Hi Impala!'")

Configuration

Run vdk config-help - search for those prefixed with "IMPALA_" to see what configuration options are available.

Disclaimer

This plugin is tested against a specific impala version. The version comes from the docker-compose.yaml container's impala version. For more information on the imapala version tested against please google the docker image.

Testing

Testing this plugin locally requires installing the dependencies listed in vdk-plugins/vdk-impala/requirements.txt

Run

pip install -r requirements.txt