Skip to main content

Dagster is an opinionated programming model for data pipelines.

Project description

.. image:: https://user-images.githubusercontent.com/28738937/44878798-b6e17e00-ac5c-11e8-8d25-2e47e5a53418.png :align: center

.. docs-include

.. image:: https://badge.fury.io/py/dagster.svg :target: https://badge.fury.io/py/dagster .. image:: https://coveralls.io/repos/github/dagster-io/dagster/badge.svg?branch=master :target: https://coveralls.io/github/dagster-io/dagster?branch=master .. image:: https://circleci.com/gh/dagster-io/dagster.svg?style=svg :target: https://circleci.com/gh/dagster-io/dagster .. image:: https://readthedocs.org/projects/dagster/badge/?version=master :target: https://dagster.readthedocs.io/en/master/ .. image:: https://img.shields.io/pypi/l/dagster.svg :target: http://www.apache.org/licenses/LICENSE-2.0.txt .. image:: https://img.shields.io/pypi/pyversions/dagster.svg :target: https://pypi.org/project/dagster/

============ Introduction

Dagster is an opinionated system and programming model for data pipelines. This process goes by many names -- ETL (extract-transform-load), ELT (extract-load-transform), model production, data integration, and so on -- but in essence they all describe the same activity: performing a set of computations structured as a DAG (directed, acyclic graph) that end up producing data assets, whether those assets be tables, files, machine-learning models, etc.

This project contains the core programming model and abstraction stack behind Dagster; a stateless single-node, single-process execution engine; and a CLI tool for driving that engine.

Go to https://dagster.readthedocs.io/ for complete documentation, including a step-by-step tutorial and notes on the demo project.

For details on contributing or running the project for development, see https://dagster.readthedocs.io/en/latest/contributing.html.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dagster-0.3.0.post3.tar.gz (110.5 kB view details)

Uploaded Source

Built Distribution

dagster-0.3.0.post3-py3-none-any.whl (268.3 kB view details)

Uploaded Python 3

File details

Details for the file dagster-0.3.0.post3.tar.gz.

File metadata

  • Download URL: dagster-0.3.0.post3.tar.gz
  • Upload date:
  • Size: 110.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.5

File hashes

Hashes for dagster-0.3.0.post3.tar.gz
Algorithm Hash digest
SHA256 ea4971c36f2077da3cf201359922e9c20ff30e8115cdaa9e47aeafc119da64aa
MD5 a9c2b754c9ff95c9d949c2e53c19ae6c
BLAKE2b-256 53bb1119ea4e38e374ca851ae81a1c33564bc1ecfd9934ffa6b5341ed87bef69

See more details on using hashes here.

File details

Details for the file dagster-0.3.0.post3-py3-none-any.whl.

File metadata

  • Download URL: dagster-0.3.0.post3-py3-none-any.whl
  • Upload date:
  • Size: 268.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.5

File hashes

Hashes for dagster-0.3.0.post3-py3-none-any.whl
Algorithm Hash digest
SHA256 b21b5a772e1e90d0639478bfb40339d3ed023941b40efc3cd79e404c4461beb5
MD5 1e641b5eb0d8477ae36d5b43795ca570
BLAKE2b-256 f4e7c529fabbfcb54098acf2268340ede1497b5647972b7023c50e9b5ca3d439

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page