Skip to main content

Versatile Data Kit SDK ingestion plugin to ingest data via http requests.

Project description

VDK-INGEST-HTTP Plugin

This plugin provides functionality to ingest data over http.

Usage

To use the plugin, just install it, and set the method attribute of send_object_for_ingestion(), and send_tabular_data_for_ingestion() functions to "http".

Example:

def run(job_input: IJobInput):
    # Do something to get data for ingestion
    payload = get_some_data()

    # Ingest the data
    job_input.send_object_for_ingestion(payload=payload,
                                        destination_table="aa_test_table",
                                        method="http",
                                        target="http://example.com/data-source"
                                        )

The above example shows how to ingest json data. In this case, there are three arguments that are required: payload, method and target. This would be fixed in the future, so that only payload would be required.
The payload needs to be a json object, and should contain the destination_table inside, using the @table key. For example:

{
    "@table": "destination_table_name",
    "column1": "value1",
    "column2": "value2",
    "column3": "value3",
}

The method attribute needs to be provided for the time being.
The target attribute should specify the url endpoint, where the data will be ingested.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

vdk-ingest-http-0.1.352155979.tar.gz (3.4 kB view details)

Uploaded Source

File details

Details for the file vdk-ingest-http-0.1.352155979.tar.gz.

File metadata

  • Download URL: vdk-ingest-http-0.1.352155979.tar.gz
  • Upload date:
  • Size: 3.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.3 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.7.11

File hashes

Hashes for vdk-ingest-http-0.1.352155979.tar.gz
Algorithm Hash digest
SHA256 36ee777bbb0c755d2ef67ad2f135f5da54beb82bbc7a0412f3d15a5e34039cf6
MD5 c51dbeacd1230f91a844a64aac7273a2
BLAKE2b-256 0a03cccea4f27f0c9ab5f15937a73a7c16ee4bd452cfc662678214b14ff63f85

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page