Skip to main content

Marquez integration with Airflow

Project description


CircleCI codecov status Gitter version license

A library that integrates Airflow DAGs with Marquez for automatic metadata collection.


This library is under active development at WeWork.



$ pip install marquez-airflow

To install from source run:

$ python install


To use this library, the line from airflow import DAG needs to be replaced by from marquez_airflow import DAG, see example below:

from marquez_airflow import DAG
from airflow.operators.dummy_operator import DummyOperator

DAG_NAME = 'my_DAG_name'

default_args = {
    'marquez_location': 'github://data-dags/dag_location/',
    'marquez_input_urns': ["s3://some_data", "s3://more_data"],
    'marquez_output_urns': ["s3://output_data"],
    'owner': ...,
    'depends_on_past': False,
    'start_date': ...,

dag = DAG(DAG_NAME, schedule_interval='*/10 * * * *',
          default_args=default_args, description="yet another DAG")

run_this = DummyOperator(task_id='run_this', dag=dag)
run_this_too = DummyOperator(task_id='run_this_too', dag=dag)


See for more details about how to contribute.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for marquez-airflow, version 0.2.0
Filename, size File type Python version Upload date Hashes
Filename, size marquez-airflow-0.2.0.tar.gz (10.5 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page