A dagster plugin that allows you to run your Meltano project inside Dagster.
Project description
Dagster-meltano
A dagster plugin that allows you to run Meltano using Dagster.
Installation
You can install using pip install dagster-meltano
.
Examples
An example of automatically loading all jobs and schedules from your Meltano project.
from dagster import repository
from dagster_meltano import load_jobs_from_meltano_project
meltano_jobs = load_jobs_from_meltano_project("<path-to-meltano-root>")
@repository
def repository():
return [meltano_jobs]
An example of running a abitrary meltano run
command.
from dagster import repository, job
from dagster_meltano import meltano_resource, meltano_run_op
@job(resource_defs={"meltano": meltano_resource})
def meltano_run_job():
tap_done = meltano_run_op("tap-1 target-1")()
meltano_run_op("tap-2 target-2")(tap_done)
@repository()
def repository():
return [meltano_run_job]
Development using VSCode
- Open this repository in Visual Studio Code.
- Install the Remote - Containers plugin for Visual Studio Code.
- Go to the example Meltano project root
cd meltano_project
- Install all plugins
meltano install
- Start dagit
meltano invoke dagster:start
- Visit
localhost:3000
to access Dagit.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dagster_meltano-1.0.1.tar.gz
(9.9 kB
view hashes)
Built Distribution
Close
Hashes for dagster_meltano-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b006f78ebad78a44e8b19ca87cd7bc8c527d9bd508a415f2dc588695fa11bf8c |
|
MD5 | 546cbc6f648becccbcb42f4b6f638c50 |
|
BLAKE2b-256 | d23bfa9376ac0fc798396fafba7a0dc9cbc144aa32079f62a672e465dbf76d64 |