Avionix Airflow
Why are there so many guides on how to set up Apache Airflow when the code could just be given to you?!
Welcome to Avionix Airflow, an out of the box solution to installing airflow on Kubernetes.
Currently supported platforms are Minikube and AWS EKS (managed nodegroups only), pull requests adding support for GKE and AKS are welcome.
Overview of Avionix Airflow Capabilities
Avionix airflow provides the following out of the box solutions for airflow
Airflow Webserver and Scheduler with configured RBAC for the KubernetesExecutor
Airflow metric collection (statsd -> telegraf -> elasticsearch)
Airflow log collection (stdout -> filebeat -> elasticsearch)
Metric and log visualization (elasticsearch & postgres -> grafana)
Installation
pip install avionix_airflowRequirements
Configuration
Avionix airflow is configured by using Options objects, there are currently 4 different objects:
AirflowOptions
PostgresOptions
MonitoringOptions
CloudOptions (Specified using LocalOptions or AwsOptions)
RedisOptions (Only for use with CeleryExecutor, *not recommended*)
These options are then passed into the function get_chart_builder, which can be used to retrieve an avionix chart builder object. For instruction on how to use the builder object, see avionix
FAQ
- How do I change the grafana dashboard?
You can change the dashboard by setting the MonitoringOptions role to “Admin”
Documentation
Additional documentation can be found here
1 maintainer
Metadata
1 maintainer
Release files for avionix-airflow 0.2.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| avionix_airflow-0.2.4.tar.gz | 64.1 kB | Details |
Release files / avionix_airflow-0.2.4.tar.gz
| Download URL | avionix_airflow-0.2.4.tar.gz |
|---|---|
| Size | 64.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0c567a95dce86294df5c583cf56b0cbc63af3577020c25b7249e12227ca0315f
|
|
BLAKE2b-256 checksum How to use checksums |
de4424eca3655008935b49f2e923bf661404879e76bd0595a50dbaf325ce1c30
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.2.0 pkginfo/1.6.0 requests/2.24.0 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.50.2 CPython/3.9.0
|
