Dagster is an opinionated programming model for data pipelines.
Project description
.. image:: https://user-images.githubusercontent.com/28738937/44878798-b6e17e00-ac5c-11e8-8d25-2e47e5a53418.png :align: center
.. docs-include
.. image:: https://badge.fury.io/py/dagster.svg :target: https://badge.fury.io/py/dagster .. image:: https://coveralls.io/repos/github/dagster-io/dagster/badge.svg?branch=master :target: https://coveralls.io/github/dagster-io/dagster?branch=master .. image:: https://circleci.com/gh/dagster-io/dagster.svg?style=svg :target: https://circleci.com/gh/dagster-io/dagster .. image:: https://readthedocs.org/projects/dagster/badge/?version=master :target: https://dagster.readthedocs.io/en/master/ .. image:: https://img.shields.io/pypi/l/dagster.svg :target: http://www.apache.org/licenses/LICENSE-2.0.txt .. image:: https://img.shields.io/pypi/pyversions/dagster.svg :target: https://pypi.org/project/dagster/
============ Introduction
Dagster is an opinionated system and programming model for data pipelines. This process goes by many names -- ETL (extract-transform-load), ELT (extract-load-transform), model production, data integration, and so on -- but in essence they all describe the same activity: performing a set of computations structured as a DAG (directed, acyclic graph) that end up producing data assets, whether those assets be tables, files, machine-learning models, etc.
This project contains the core programming model and abstraction stack behind Dagster; a stateless single-node, single-process execution engine; and a CLI tool for driving that engine.
Go to https://dagster.readthedocs.io/ for complete documentation, including a step-by-step tutorial and notes on the demo project.
For details on contributing or running the project for development, see https://dagster.readthedocs.io/en/latest/contributing.html.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file dagster-0.3.0.post3.tar.gz
.
File metadata
- Download URL: dagster-0.3.0.post3.tar.gz
- Upload date:
- Size: 110.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ea4971c36f2077da3cf201359922e9c20ff30e8115cdaa9e47aeafc119da64aa |
|
MD5 | a9c2b754c9ff95c9d949c2e53c19ae6c |
|
BLAKE2b-256 | 53bb1119ea4e38e374ca851ae81a1c33564bc1ecfd9934ffa6b5341ed87bef69 |
File details
Details for the file dagster-0.3.0.post3-py3-none-any.whl
.
File metadata
- Download URL: dagster-0.3.0.post3-py3-none-any.whl
- Upload date:
- Size: 268.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b21b5a772e1e90d0639478bfb40339d3ed023941b40efc3cd79e404c4461beb5 |
|
MD5 | 1e641b5eb0d8477ae36d5b43795ca570 |
|
BLAKE2b-256 | f4e7c529fabbfcb54098acf2268340ede1497b5647972b7023c50e9b5ca3d439 |