Skip to main content

Runs Singer.io taps and targets, adding additional tooling.

Project description

singer-runner

Singer Runner manages tap and target processes, as well as metrics, state, and configuration.

Features

  • Run a tap or target
  • Pass run options via CLI paramters or JSON/YAML config file.
  • Use local file system or S3 for piping the singer stream, storing state, and storing metrics.
  • Metric storage, piping, and state storage can be extended / customized by inheriting from the base classes.

Usage

Requires python 3, tested with python 3.7

Install

pip install singer-runner

Run

$ singer-runner 
Usage: singer-runner [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  run-tap
  run-target

Concepts

  • Pipes
    • Pipes move a stream of Singer messages from tap to target. A pipe could be as simple as a local file, a file in S3, or Kafka.
  • State Storage
  • Metrics Storage

Programmatic Usage

Singer runner can be used within any python application. The primary functions are in singer_runner.runner including:

  • run_tap runs a tap
  • run_target runs a target

Classes in the singer_runner.metrics, singer_runner.pipes, and singer_runner.state can be used as arguemnts, along with catalog/config.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

singer-runner-0.0.2.tar.gz (8.3 kB view hashes)

Uploaded Source

Built Distribution

singer_runner-0.0.2-py3-none-any.whl (15.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page