vdk-ingest-file

VDK-INGEST-FILE Plugin

This plugin provides functionality to ingest data into a file. It is intended for local testing.

Usage

To use the plugin, just install it, and set the method attribute of send_object_for_ingestion(), and send_tabular_data_for_ingestion() functions to "file".

Example:

def run(job_input: IJobInput):
    # Do something to get data for ingestion
    payload = get_some_data()

    # Ingest the data
    job_input.send_object_for_ingestion(payload=payload,
                                        destination_table="aa_test_table",
                                        method="file",
                                        target="name_of_file"
                                        )

The above example shows how to ingest json data. In this case, there is only one argument that is required: payload.
It needs to be a json object, and can contain the destination_table inside, using the @table key. For example:

{
    "@table": "destination_table_name",
    "column1": "value1",
    "column2": "value2",
    "column3": "value3",
}

The target attribute is being used to specify the name of the file, where the data will be ingested. If not specified, it is constructed, using the model, table.<creation-timestamp>.json.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

vdk-ingest-file-0.1.348963555.tar.gz (2.6 kB view details)

Uploaded Aug 9, 2021 Source

File details

Details for the file vdk-ingest-file-0.1.348963555.tar.gz.

File metadata

Download URL: vdk-ingest-file-0.1.348963555.tar.gz
Upload date: Aug 9, 2021
Size: 2.6 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.4.2 importlib_metadata/4.6.3 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.7.11

File hashes

Hashes for vdk-ingest-file-0.1.348963555.tar.gz
Algorithm	Hash digest
SHA256	`b993ef62dafae21780ac237070061f74063319e19dd590cb7827a05feadc2129`
MD5	`466bcc78e7074ddce40c1a269cee5f73`
BLAKE2b-256	`28b39c3ce7b2243f20d7a173a81347263b20ce8129c9ee1ba3eba1dad038c9da`