Skip to main content

Singer.io target for writing JSON Line files

Project description

target-jsonl

A Singer target that writes data to JSONL (JSON Lines) files.

How to use it

target-jsonl works together with any other Singer Tap to move data from sources like Braintree, Freshdesk and Hubspot to JSONL formatted files.

Install

We will use tap-exchangeratesapi to pull currency exchange rate data from a public data set as an example.

First, make sure Python 3 is installed on your system or follow these installation instructions for Mac or Ubuntu.

It is recommended to install each Tap and Target in a separate Python virtual environment to avoid conflicting dependencies between any Taps and Targets.

 # Install tap-exchangeratesapi in its own virtualenv
python3 -m venv ~/.virtualenvs/tap-exchangeratesapi
source ~/.virtualenvs/tap-exchangeratesapi/bin/activate
pip install tap-exchangeratesapi
deactivate

# Install target-jsonl in its own virtualenv
python3 -m venv ~/.virtualenvs/target-jsonl
source ~/.virtualenvs/target-jsonl/bin/activate
pip install target-jsonl
deactivate

Run

We can now run tap-exchangeratesapi and pipe the output to target-jsonl.

~/.virtualenvs/tap-exchangeratesapi/bin/tap-exchangeratesapi | ~/.virtualenvs/target-jsonl/bin/target-jsonl

The data by default will be written to a file called exchange_rate-{timestamp}.jsonl in your working directory.

 cat exchange_rate-{timestamp}.jsonl
{"CAD": 1.3954067515, "HKD": 7.7503228187, "ISK": 147.1130787678, "PHP": 50.5100534957, "DKK": 6.8779745434, "HUF": 327.9376498801, "CZK": 25.018446781, "GBP": 0.8059214167, "RON": 4.4673491976, "SEK": 9.9002029146, "IDR": 15321.0016602103, "INR": 75.6516325401, "BRL": 5.4711307877, "RUB": 73.6220254566, "HRK": 6.9765725881, "JPY": 106.548607268, "THB": 32.420217672, "CHF": 0.9750046117, "EUR": 0.9223390518, "MYR": 4.3475373547, "BGN": 1.8039107176, "TRY": 6.988286294, "CNY": 7.0764619074, "NOK": 10.3973436635, "NZD": 1.6446227633, "ZAR": 18.4316546763, "USD": 1.0, "MXN": 24.1217487548, "SGD": 1.4152370411, "AUD": 1.5361556908, "ILS": 3.5102379635, "KRW": 1218.9540675152, "PLN": 4.1912931194, "date": "2020-04-29T00:00:00Z"}

Optional Configuration

target-jsonl takes an optional configuration file that can be used to set formatting parameters like the delimiter - see config.sample.json for examples. To run target-jsonl with the configuration file, use this command:

~/.virtualenvs/tap-exchangeratesapi/bin/tap-exchangeratesapi | ~/.virtualenvs/target-jsonl/bin/target-jsonl -c my-config.json

Here is a brief description of the optional config keys

destination_path - Specifies where to write the resulting .jsonl file to. By default, the file gets written in your working directory.

custom_name - Specifies a custom name for the filename, instead of the stream name (i.e. {custom_name}-{timestamp}.jsonl, asumming do_timestamp_file is true). By default, the stream name will be used.

do_timestamp_file - specifies if the file should get timestamped. By default, the resulting file will have a timestamp in the file name (i.e. exchange_rate-{timestamp}.jsonl as described above in the Run section). If this option gets set to false, the resulting file will not have a timestamp associated with it (i.e. exchange_rate.jsonl in our example).


Copyright © 2020 Andy Huynh

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

target-jsonl-0.1.4.tar.gz (4.5 kB view details)

Uploaded Source

Built Distribution

target_jsonl-0.1.4-py3-none-any.whl (15.7 kB view details)

Uploaded Python 3

File details

Details for the file target-jsonl-0.1.4.tar.gz.

File metadata

  • Download URL: target-jsonl-0.1.4.tar.gz
  • Upload date:
  • Size: 4.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.0 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.0

File hashes

Hashes for target-jsonl-0.1.4.tar.gz
Algorithm Hash digest
SHA256 c0c5a0cb69421439ca9014ea3bff8e13d36fc2bcd56a1883d7278c01b12ec53c
MD5 5b1c7b854e0bf13e1cbf554604879712
BLAKE2b-256 b0892331ae3a9601baea273c44681c93ede3f34dbc6bf79991d54477bff3e9d2

See more details on using hashes here.

File details

Details for the file target_jsonl-0.1.4-py3-none-any.whl.

File metadata

  • Download URL: target_jsonl-0.1.4-py3-none-any.whl
  • Upload date:
  • Size: 15.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.0 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.0

File hashes

Hashes for target_jsonl-0.1.4-py3-none-any.whl
Algorithm Hash digest
SHA256 b3d7e747c87e3ecdc0f7b0edc31b43ed09e9adfef65d6df392d02d3ab7be6ab1
MD5 8a0279a1af7958176336398850b5090b
BLAKE2b-256 2410a2f2f5008a09d156a11442901366f3055ffd52bf4acb88e17c1199463a7d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page