Skip to main content

Provider for Apache Airflow. Implements apache-airflow-providers-apache-livy package

Project description

Package apache-airflow-providers-apache-livy

Release: 3.3.0rc1

Apache Livy

Provider package

This is a provider package for apache.livy provider. All classes for this provider package are in airflow.providers.apache.livy python package.

You can find package information and changelog for the provider in the documentation.

Installation

You can install this package on top of an existing Airflow 2 installation (see Requirements below for the minimum Airflow version supported) via pip install apache-airflow-providers-apache-livy

The package supports the following python versions: 3.7,3.8,3.9,3.10

Requirements

PIP package

Version required

apache-airflow

>=2.3.0

apache-airflow-providers-http

aiohttp

asgiref

Cross provider package dependencies

Those are dependencies that might be needed in order to use all the features of the package. You need to install the specified provider packages in order to use them.

You can install such cross-provider dependencies when installing from PyPI. For example:

pip install apache-airflow-providers-apache-livy[http]

Dependent package

Extra

apache-airflow-providers-http

http

Changelog

3.3.0

Features

  • Add Livy Operator with deferrable mode (#29047)

3.2.0

This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy.

Misc

  • Move min airflow version to 2.3.0 for all providers (#27196)

Features

  • Add template to livy operator documentation (#27404)

  • Add Spark's 'appId' to xcom output (#27376)

  • add template field renderer to livy operator (#27321)

3.1.0

Features

  • Add auth_type to LivyHook (#25183)

3.0.0

Breaking changes

2.2.3

Bug Fixes

  • Fix: Add extra headers to all livy API requests instead of only to post_batch (#22510)

2.2.2

Bug Fixes

  • Fix mistakenly added install_requires for all providers (#22382)

2.2.1

Misc

  • Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)

2.2.0

Features

  • Added retries to LivyHook #19384 (#21550)

Misc

  • Support for Python 3.10

  • add how-to guide for livy operator (#21529)

2.1.0

Features

  • Fetching and logging livy session logs for LivyOperrator (#17393)

Misc

  • Optimise connection importing for Airflow 2.2.0

2.0.0

Breaking changes

  • Auto-apply apply_default decorator (#15667)

1.1.0

Features

  • Extend HTTP extra_options to LivyHook and operator (#14816)

1.0.1

  • Updated documentation and readme files.

1.0.0

  • Initial version of the provider.

Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page