Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Runs Singer.io taps and targets, adding additional tooling.

Project description

singer-runner

Singer Runner manages tap and target processes, as well as metrics, state, and configuration.

Features

  • Run a tap or target
  • Pass run options via CLI paramters or JSON/YAML config file.
  • Use local file system or S3 for piping the singer stream, storing state, and storing metrics.
  • Metric storage, piping, and state storage can be extended / customized by inheriting from the base classes.

Usage

Requires python 3, tested with python 3.7

Install

pip install singer-runner

Run

$ singer-runner 
Usage: singer-runner [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  run-tap
  run-target

Concepts

  • Pipes
    • Pipes move a stream of Singer messages from tap to target. A pipe could be as simple as a local file, a file in S3, or Kafka.
  • State Storage
  • Metrics Storage

Programmatic Usage

Singer runner can be used within any python application. The primary functions are in singer_runner.runner including:

  • run_tap runs a tap
  • run_target runs a target

Classes in the singer_runner.metrics, singer_runner.pipes, and singer_runner.state can be used as arguemnts, along with catalog/config.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for singer-runner, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size singer_runner-0.0.2-py3-none-any.whl (15.8 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size singer-runner-0.0.2.tar.gz (8.3 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page