apache-airflow-providers-apache-spark 6.1.0rc1

Provider package apache-airflow-providers-apache-spark for Apache Airflow

Package apache-airflow-providers-apache-spark

Release: 6.1.0

Apache Spark

Provider package

This is a provider package for apache.spark provider. All classes for this provider package are in airflow.providers.apache.spark python package.

You can find package information and changelog for the provider in the documentation.

Installation

You can install this package on top of an existing Airflow installation (see Requirements below for the minimum Airflow version supported) via pip install apache-airflow-providers-apache-spark

The package supports the following python versions: 3.10,3.11,3.12,3.13,3.14

Requirements

PIP package	Version required
apache-airflow	>=2.11.0
apache-airflow-providers-common-compat	>=1.12.0
pyspark-client	>=4.0.0
grpcio-status	>=1.67.0
requests	>=2.32.0
tenacity	>=8.3.0

Cross provider package dependencies

Those are dependencies that might be needed in order to use all the features of the package. You need to install the specified providers in order to use them.

You can install such cross-provider dependencies when installing from PyPI. For example:

pip install apache-airflow-providers-apache-spark[cncf.kubernetes]

Dependent package	Extra
apache-airflow-providers-cncf-kubernetes	cncf.kubernetes
apache-airflow-providers-common-compat	common.compat

Optional dependencies

Extra	Dependencies
cncf.kubernetes	apache-airflow-providers-cncf-kubernetes>=7.4.0
openlineage	apache-airflow-providers-openlineage
pyspark	pyspark>=4.0.0

The changelog for the provider package can be found in the changelog.