Versatile Data Kit SDK ingestion plugin to ingest data into a file.
Project description
VDK-INGEST-FILE Plugin
This plugin provides functionality to ingest data into a file. It is intended for local testing.
Usage
To use the plugin, just install it, and set the method
attribute of send_object_for_ingestion()
,
and send_tabular_data_for_ingestion()
functions to "file".
Example:
def run(job_input: IJobInput):
# Do something to get data for ingestion
payload = get_some_data()
# Ingest the data
job_input.send_object_for_ingestion(payload=payload,
destination_table="aa_test_table",
method="file",
target="name_of_file"
)
The above example shows how to ingest json data. In this case, there is only one argument that is required: payload
.
It needs to be a json object, and can contain the destination_table
inside, using the @table
key. For example:
{
"@table": "destination_table_name",
"column1": "value1",
"column2": "value2",
"column3": "value3",
}
The target
attribute is being used to specify the name of the file, where the data will be ingested. If not specified, it is constructed,
using the model, table.<creation-timestamp>.json
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for vdk-ingest-file-0.1.715017056.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3d4809df9ca0e9ebaa569eb7362f0c527422bf5c667612cada70d6ab47330cad |
|
MD5 | d56e3a866b6bafdced788a107a9e1f08 |
|
BLAKE2b-256 | a23bb30eb5fcb7f358473967e7e2d2a99526b92028d8e904b0f1edacbbe2ece2 |