Dagster is an opinionated programming model for data pipelines.
Project description
.. image:: https://user-images.githubusercontent.com/28738937/44878798-b6e17e00-ac5c-11e8-8d25-2e47e5a53418.png :align: center
.. docs-include
.. image:: https://badge.fury.io/py/dagster.svg :target: https://badge.fury.io/py/dagster .. image:: https://coveralls.io/repos/github/dagster-io/dagster/badge.svg?branch=master :target: https://coveralls.io/github/dagster-io/dagster?branch=master .. image:: https://circleci.com/gh/dagster-io/dagster.svg?style=svg :target: https://circleci.com/gh/dagster-io/dagster .. image:: https://readthedocs.org/projects/dagster/badge/?version=master :target: https://dagster.readthedocs.io/en/master/ .. image:: https://img.shields.io/pypi/l/dagster.svg :target: http://www.apache.org/licenses/LICENSE-2.0.txt .. image:: https://img.shields.io/pypi/pyversions/dagster.svg :target: https://pypi.org/project/dagster/
============ Introduction
Dagster is an opinionated system and programming model for data pipelines. This process goes by many names -- ETL (extract-transform-load), ELT (extract-load-transform), model production, data integration, and so on -- but in essence they all describe the same activity: performing a set of computations structured as a DAG (directed, acyclic graph) that end up producing data assets, whether those assets be tables, files, machine-learning models, etc.
This project contains the core programming model and abstraction stack behind Dagster; a stateless single-node, single-process execution engine; and a CLI tool for driving that engine.
Go to https://dagster.readthedocs.io/ for complete documentation, including a step-by-step tutorial and notes on the demo project.
For details on contributing or running the project for development, see https://dagster.readthedocs.io/en/latest/contributing.html.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dagster-0.4.3.post2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83699484974ebc8e5ed6d3effc1a0cd7a07c3cc6913b23467ccd75ca504c65cc |
|
MD5 | ca358696c444328852811365a4892595 |
|
BLAKE2b-256 | 9d58b94bb813a5afb745712f71ecbe160ad3b12e5151347974188e54fa02c4e0 |