scalefree-target-s3-json 0.2

Singer.io target for writing JSON files and upload to S3

target-s3-json

Singer target that uploads loads data to S3 in JSON format following the Singer spec.

How to use it

Install

First, make sure Python 3 is installed on your system or follow these installation instructions for Mac or Ubuntu.

It's recommended to use a virtualenv:

  python3 -m venv env-target-s3-json

Install the package target-s3-json in the virtualenv:

Either install it from github:

  source env-target-s3-json/bin/activate
  pip install git+https://github.com/ScalefreeCOM/scalefree-target-s3-json
  deactivate

or from PyPI:

  source env-target-s3-json/bin/activate
  pip install scalefree-target-s3-json
  deactivate

To run

Like any other target that's following the singer specification:

  some-singer-tap --catalog [catalog.json] | ~/environment/env-target-s3-json/bin/python3 env-target-s3-json/lib/python3.7/site-packages/target_s3_json/__init__.py --config [config.json]

Note: To avoid version conflicts run tap and targets in separate virtual environments.

Configuration settings

Running the target connector requires a config.json file. An example with the minimal settings:

{
 "aws_access_key_id": "ACCESS-KEY",
 "aws_secret_access_key": "SECRET",
 "s3_bucket": "BUCKET",
 "s3_key_prefix": "SOME-PREFIX/",
 "delimiter": ","
}

License

Apache License Version 2.0

See LICENSE to see the full text.