Skip to main content

Runs Singer.io taps and targets, adding additional tooling.

Project description

singer-runner

Singer Runner manages tap and target processes, as well as metrics, state, and configuration.

Features

  • Run a tap or target
  • Pass run options via CLI paramters or JSON/YAML config file.
  • Use local file system or S3 for piping the singer stream, storing state, and storing metrics.
  • Metric storage, piping, and state storage can be extended / customized by inheriting from the base classes.

Usage

Requires python 3, tested with python 3.7

Install

pip install singer-runner

Run

$ singer-runner 
Usage: singer-runner [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  run-tap
  run-target

Concepts

  • Pipes
    • Pipes move a stream of Singer messages from tap to target. A pipe could be as simple as a local file, a file in S3, or Kafka.
  • State Storage
  • Metrics Storage

Programmatic Usage

Singer runner can be used within any python application. The primary functions are in singer_runner.runner including:

  • run_tap runs a tap
  • run_target runs a target

Classes in the singer_runner.metrics, singer_runner.pipes, and singer_runner.state can be used as arguemnts, along with catalog/config.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

singer-runner-0.0.2.tar.gz (8.3 kB view details)

Uploaded Source

Built Distribution

singer_runner-0.0.2-py3-none-any.whl (15.8 kB view details)

Uploaded Python 3

File details

Details for the file singer-runner-0.0.2.tar.gz.

File metadata

  • Download URL: singer-runner-0.0.2.tar.gz
  • Upload date:
  • Size: 8.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3

File hashes

Hashes for singer-runner-0.0.2.tar.gz
Algorithm Hash digest
SHA256 b22af5b4772362e021909c4f04a255eb16e22f2373a652f5a2d9f8b870c342cc
MD5 b0d7127a8465914f5c79c5108fe863e0
BLAKE2b-256 72d84d450b59311ddebf5619b92f0dc03dbd993b1c6b424932d9186b1d266c84

See more details on using hashes here.

File details

Details for the file singer_runner-0.0.2-py3-none-any.whl.

File metadata

  • Download URL: singer_runner-0.0.2-py3-none-any.whl
  • Upload date:
  • Size: 15.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3

File hashes

Hashes for singer_runner-0.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 11117303fd100b8afc73df8de0bc87ca664cd50b8245f840ecb98c5aec6fbc3d
MD5 10aa5258273c34303c837cd3e632be5b
BLAKE2b-256 76ef44f844b5125f77f25d47c2bc8ca215ddb43cab364211fe35d9f13e6f37f7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page