Skip to main content

Orchestrate your dbt projects in Airflow

Project description

https://raw.githubusercontent.com/astronomer/astronomer-cosmos/main/docs/_static/cosmos-logo.svg

fury ossrank downloads pre-commit.ci status

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code. Benefits include:

  • Run dbt projects against Airflow connections instead of dbt profiles

  • Native support for installing and running dbt in a virtual environment to avoid dependency conflicts with Airflow

  • Run tests immediately after a model is done to catch issues early

  • Utilize Airflow’s data-aware scheduling to run models immediately after upstream ingestion

  • Turn each dbt model into a task/task group complete with retries, alerting, etc.

Quickstart

Check out the Getting Started guide on our docs. See more examples at /dev/dags and at the cosmos-demo repo.

Example Usage

You can render a Cosmos Airflow DAG using the DbtDag class. Here’s an example with the jaffle_shop project:

https://github.com/astronomer/astronomer-cosmos/blob/24aa38e528e299ef51ca6baf32f5a6185887d432/dev/dags/basic_cosmos_dag.py#L1-L42

This will generate an Airflow DAG that looks like this:

/docs/_static/jaffle_shop_dag.png

Community

  • Join us on the Airflow Slack at #airflow-dbt

Changelog

We follow Semantic Versioning for releases. Check CHANGELOG.rst for the latest changes.

Contributing Guide

All contributions, bug reports, bug fixes, documentation improvements, enhancements are welcome.

A detailed overview an how to contribute can be found in the Contributing Guide.

As contributors and maintainers to this project, you are expected to abide by the Contributor Code of Conduct.

License

Apache License 2.0

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

astronomer_cosmos-1.5.1a1.tar.gz (83.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

astronomer_cosmos-1.5.1a1-py3-none-any.whl (116.9 kB view details)

Uploaded Python 3

File details

Details for the file astronomer_cosmos-1.5.1a1.tar.gz.

File metadata

  • Download URL: astronomer_cosmos-1.5.1a1.tar.gz
  • Upload date:
  • Size: 83.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.4

File hashes

Hashes for astronomer_cosmos-1.5.1a1.tar.gz
Algorithm Hash digest
SHA256 13bb0740be18e12d04a14997f187fab3d7f8d47b7e3c80d27a780eefb71ac4bb
MD5 d87b19f941254671ddbe2e0d5ef3a586
BLAKE2b-256 ed61d345d31088f51e89c9fd313426165414fb37fca3e06a7eb1a88cfacc028b

See more details on using hashes here.

File details

Details for the file astronomer_cosmos-1.5.1a1-py3-none-any.whl.

File metadata

File hashes

Hashes for astronomer_cosmos-1.5.1a1-py3-none-any.whl
Algorithm Hash digest
SHA256 13dd86dc97b14feda98006a3332c5515deace03faa239f08cbb136b9bbd61352
MD5 eed76436a48609e59c56bbb769a6f213
BLAKE2b-256 8a12730e090c18d637bbf9f0b5daa158913fe112951561ded1f3ca87e75096f6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page