Singer.io target for writing JSON Line files
Project description
target-jsonl
A Singer target that writes data to JSONL (JSON Lines) files.
How to use it
target-jsonl
works together with any other Singer Tap to move data from sources like Braintree, Freshdesk and Hubspot to JSONL formatted files.
Install
We will use tap-exchangeratesapi
to pull currency exchange rate data from a public data set as an example.
First, make sure Python 3 is installed on your system or follow these installation instructions for Mac or Ubuntu.
It is recommended to install each Tap and Target in a separate Python virtual environment to avoid conflicting dependencies between any Taps and Targets.
# Install tap-exchangeratesapi in its own virtualenv
python3 -m venv ~/.virtualenvs/tap-exchangeratesapi
source ~/.virtualenvs/tap-exchangeratesapi/bin/activate
pip install tap-exchangeratesapi
deactivate
# Install target-jsonl in its own virtualenv
python3 -m venv ~/.virtualenvs/target-jsonl
source ~/.virtualenvs/target-jsonl/bin/activate
pip install target-jsonl
deactivate
Run
We can now run tap-exchangeratesapi
and pipe the output to target-jsonl
.
~/.virtualenvs/tap-exchangeratesapi/bin/tap-exchangeratesapi | ~/.virtualenvs/target-jsonl/bin/target-jsonl
The data by default will be written to a file called exchange_rate-{timestamp}.jsonl
in your working directory.
› cat exchange_rate-{timestamp}.jsonl
{"CAD": 1.3954067515, "HKD": 7.7503228187, "ISK": 147.1130787678, "PHP": 50.5100534957, "DKK": 6.8779745434, "HUF": 327.9376498801, "CZK": 25.018446781, "GBP": 0.8059214167, "RON": 4.4673491976, "SEK": 9.9002029146, "IDR": 15321.0016602103, "INR": 75.6516325401, "BRL": 5.4711307877, "RUB": 73.6220254566, "HRK": 6.9765725881, "JPY": 106.548607268, "THB": 32.420217672, "CHF": 0.9750046117, "EUR": 0.9223390518, "MYR": 4.3475373547, "BGN": 1.8039107176, "TRY": 6.988286294, "CNY": 7.0764619074, "NOK": 10.3973436635, "NZD": 1.6446227633, "ZAR": 18.4316546763, "USD": 1.0, "MXN": 24.1217487548, "SGD": 1.4152370411, "AUD": 1.5361556908, "ILS": 3.5102379635, "KRW": 1218.9540675152, "PLN": 4.1912931194, "date": "2020-04-29T00:00:00Z"}
Optional Configuration
target-jsonl
takes an optional configuration file that can be used to set formatting parameters like the delimiter - see config.sample.json for examples. To run target-jsonl
with the configuration file, use this command:
~/.virtualenvs/tap-exchangeratesapi/bin/tap-exchangeratesapi | ~/.virtualenvs/target-jsonl/bin/target-jsonl -c my-config.json
Here is a brief description of the optional config keys
destination_path
- Specifies where to write the resulting .jsonl
file to. By default, the file gets written in your working directory.
custom_name
- Specifies a custom name for the filename, instead of the stream name (i.e. {custom_name}-{timestamp}.jsonl
, asumming do_timestamp_file
is true
). By default, the stream name will be used.
do_timestamp_file
- specifies if the file should get timestamped. By default, the resulting file will have a timestamp in the file name (i.e. exchange_rate-{timestamp}.jsonl
as described above in the Run
section). If this option gets set to false
, the resulting file will not have a timestamp associated with it (i.e. exchange_rate.jsonl
in our example).
Copyright © 2020 Andy Huynh
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file target-jsonl-0.1.4.tar.gz
.
File metadata
- Download URL: target-jsonl-0.1.4.tar.gz
- Upload date:
- Size: 4.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.0 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0c5a0cb69421439ca9014ea3bff8e13d36fc2bcd56a1883d7278c01b12ec53c |
|
MD5 | 5b1c7b854e0bf13e1cbf554604879712 |
|
BLAKE2b-256 | b0892331ae3a9601baea273c44681c93ede3f34dbc6bf79991d54477bff3e9d2 |
File details
Details for the file target_jsonl-0.1.4-py3-none-any.whl
.
File metadata
- Download URL: target_jsonl-0.1.4-py3-none-any.whl
- Upload date:
- Size: 15.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.0 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b3d7e747c87e3ecdc0f7b0edc31b43ed09e9adfef65d6df392d02d3ab7be6ab1 |
|
MD5 | 8a0279a1af7958176336398850b5090b |
|
BLAKE2b-256 | 2410a2f2f5008a09d156a11442901366f3055ffd52bf4acb88e17c1199463a7d |