airflow-provider-sap-hana 1.1.0

An Airflow provider to connect to SAP HANA

Airflow SAP HANA Provider

This packages enables Airflow to connect to SAP HANA using the official SAP Database cursor hdbcli. This allows you to use the built-in Airflow operators and database hook methods, including methods that interact with SQLAlchemy, via sqlalchemy-hana.

Table of Contents generated with DocToc

• Installing airflow-provider-sap-hana
• Setting Up the Airflow/HANA Express Environment
  • Initializing Airflow
  • Starting HANA Express
  • Start the Remaining Services

Installing airflow-provider-sap-hana

pip install airflow-provider-sap-hana

Setting Up the Airflow/HANA Express Environment

This repository includes a docker-compose file designed to quickly set up an environment with everything needed to run the example DAG. The docker-compose file is based on the file provided in the Airflow quick-start guide but with several key modifications.

• An additional service hana-express - This creates a SAP HANA Express instance, a tenant database named HXE, and a schema named 'AIRFLOW'

• Additional environment variable AIRFLOW_CONN_HANA_DEFAULT which contains the URI needed to connect to the HANA express instance.

• A custom Dockerfile to extend Airflow to include airflow-provider-sap-hana and Faker, to generate the mock data for the DAG.

The Airflow quick-start guide and the original docker-compose file can be found here Running Airflow in Docker

Initializing Airflow

The following directories and environment files need to be created before you initialize Airflow.

mkdir -p ./dags ./logs ./plugins ./config
echo -e "AIRFLOW_UID=$(id -u)" > .env

To initialize Airflow, run.

docker compose up airflow-init

After initialization is complete, you should see a message like this:

airflow-init_1       | Upgrades done
airflow-init_1       | Admin user airflow created
airflow-init_1       | 2.10.4
start_airflow-init_1 exited with code 0

Starting HANA Express

Before you start HANA Express, you need to create a directory to persist data outside the container and grant it the proper permissions.

mkdir -p ./hana
sudo chown 12000:79 ./hana

To start hana-express, run.

docker compose up hana-express -d

If you want to have visibility to the start-up process, run.

docker compose up hana-express -d && docker logs -f hana-express

This should take several minutes.

Start the Remaining Services

To start the remaining Airflow services, run.

docker compose up -d