VDK-INGEST-HTTP Plugin
This plugin provides functionality to ingest data over http.
Usage
To use the plugin, just install it, and set the method attribute of send_object_for_ingestion(),
and send_tabular_data_for_ingestion() functions to "http".
Example:
def run(job_input: IJobInput):
# Do something to get data for ingestion
payload = get_some_data()
# Ingest the data
job_input.send_object_for_ingestion(payload=payload,
destination_table="aa_test_table",
method="http",
target="http://example.com/data-source"
)
The above example shows how to ingest json data. In this case, there are three arguments that are required: payload,
method and target. This would be fixed in the future, so that only payload would be required.
The payload needs to be a json object, and should contain the destination_table inside, using the @table key. For example:
{
"@table": "destination_table_name",
"column1": "value1",
"column2": "value2",
"column3": "value3",
}
The method attribute needs to be provided for the time being.
The target attribute should specify the url endpoint, where the data will be ingested.
Testing
To develop or test locally (from the current directory)
# install dev and test dependencies
pip install -r requirements
# install the plugin in editable mode
pip install -e .
# run the tests
pytest
1 maintainer
Metadata
1 maintainer
Release files for vdk-ingest-http 0.1.376776639
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| vdk-ingest-http-0.1.376776639.tar.gz | 3.5 kB | Details |
Release files / vdk-ingest-http-0.1.376776639.tar.gz
| Download URL | vdk-ingest-http-0.1.376776639.tar.gz |
|---|---|
| Size | 3.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
751ee0cb6f018e5900bc8a2b02c5b2ec75ee11a0dd3ea87b92b4b0a0108f68a1
|
|
BLAKE2b-256 checksum How to use checksums |
9d68875cdbd362329c825c1e57eed87a20cb63a26f8d8b52ad12cae5e77b6a86
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.7.11
|
