Dagster is an opinionated programming model for data pipelines.
Project description
.. image:: https://user-images.githubusercontent.com/28738937/44878798-b6e17e00-ac5c-11e8-8d25-2e47e5a53418.png :align: center
.. docs-include
.. image:: https://badge.fury.io/py/dagster.svg :target: https://badge.fury.io/py/dagster .. image:: https://coveralls.io/repos/github/dagster-io/dagster/badge.svg?branch=master :target: https://coveralls.io/github/dagster-io/dagster?branch=master .. image:: https://circleci.com/gh/dagster-io/dagster.svg?style=svg :target: https://circleci.com/gh/dagster-io/dagster .. image:: https://readthedocs.org/projects/dagster/badge/?version=master :target: https://dagster.readthedocs.io/en/master/ .. image:: https://img.shields.io/pypi/l/dagster.svg :target: http://www.apache.org/licenses/LICENSE-2.0.txt .. image:: https://img.shields.io/pypi/pyversions/dagster.svg :target: https://pypi.org/project/dagster/
============ Introduction
Dagster is an opinionated system and programming model for data pipelines. This process goes by many names -- ETL (extract-transform-load), ELT (extract-load-transform), model production, data integration, and so on -- but in essence they all describe the same activity: performing a set of computations structured as a DAG (directed, acyclic graph) that end up producing data assets, whether those assets be tables, files, machine-learning models, etc.
This project contains the core programming model and abstraction stack behind Dagster; a stateless single-node, single-process execution engine; and a CLI tool for driving that engine.
Go to https://dagster.readthedocs.io/ for complete documentation, including a step-by-step tutorial and notes on the demo project.
For details on contributing or running the project for development, see https://dagster.readthedocs.io/en/latest/contributing.html.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dagster-0.4.3rc1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38aac6a8d87e4c4a82233f7461cccd790dc9303f06552b3b713d8dcec0b69680 |
|
MD5 | 7b6fd2dc655d5dffa70c130207514708 |
|
BLAKE2b-256 | 3557ef2beb695dd17d0581f65c96e779025510a1dfc86248ba54ba47c51f9d1a |