A dagster plugin that allows you to run Meltano in Dagster
Project description
Dagster-meltano (Under development)
A dagster plugin that allows you to run Meltano pipelines using Dagster.
Installation
- Install using pip
pip install dagster-meltano
. - Make sure you have an installed Meltano project.
- Point the plugin to the root of the Meltano project by defining
MELTANO_PROJECT_ROOT
.
Example
An example of a Dagster pipeline that runs a Meltano elt process.
from dagster import pipeline
from dagster_meltano.solids import meltano_elt_solid
@pipeline
def meltano_pipeline():
meltano_elt_solid(
tap='tap-csv',
target='target-jsonl',
job_id='csv-to-jsonl' #Optional
)
Development
- Open this repository in Visual Studio Code.
- Install the Remote - Containers plugin for Visual Studio Code.
- Wait for the container setup, it should automatically install all Meltano plugins and start Dagit.
- Visit
localhost:3000
to access Dagit.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dagster-meltano-0.0.1.tar.gz
(2.6 kB
view hashes)
Built Distribution
Close
Hashes for dagster_meltano-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fed0fca94187852b0aa1cdfb9a1415514fca5551df37d25c8865769fb3277aa6 |
|
MD5 | 8beca2a5784dd6e3ebef51b8a7f39d24 |
|
BLAKE2b-256 | 4390e47ac5f84b4cf1de6b64f6584c13df0e6b2ee9c4464d648aa2a7f340c6aa |