tap-pipedream 0.4.0

Singer tap for Pipedream, built with the Meltano SDK for Singer Taps.

tap-pipedream

Singer tap for Pipedream. Built with the Meltano Singer SDK.

Capabilities

• catalog
• state
• discover
• about
• stream-maps
• schema-flattening

Settings

Setting	Required	Default	Description
token	True	None	API Token for Pipedream
start_date	False	None	Earliest timestamp to get data from
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.
flattening_enabled	False	None	'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth	False	None	The max depth to flatten schemas.

A full list of supported settings and capabilities is available by running: tap-pipedream --about

Source Authentication and Authorization

To generate an API key, follow the instructions in the docs.

Usage

You can easily run tap-pipedream by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-pipedream --version
tap-pipedream --help
tap-pipedream --config CONFIG --discover > ./catalog.json

Developer Resources

Initialize your Development Environment

pipx install hatch

Create and Run Tests

Run integration tests:

hatch run test:integration

You can also test the tap-pipedream CLI interface directly:

hatch run sync:console -- --about --format=json

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-pipedream
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-pipedream --version
# OR run a test `elt` pipeline:
meltano elt tap-pipedream target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.