An Apache Airflow plugin to upload to Socrata.
Project description
Airflow-Socrata
Simple hooks and operators for uploading data to Socrata.
Features
- Upsert or reupload PostgreSQL tables to Socrata
Install
Using pip:
pip3 install airflow-socrata-plugin
Usage
Create a connection named http_socrata
of type http
to store Socrata credentials. You can also pass in conn_name
parameter in DAG definition to override.
Create a connection named etl_postgres
of type postgres
to store PostgreSQL credentials. You can also pass in postgres_conn_name
parameter in DAG definition to override.
By default, the plugin looks for the specified table under public
schema. The schema can be specified with postgres_schema
parameter.
The plugin is published as a pip package. Refer to the example DAG for available parameters.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file airflow-socrata-plugin-0.0.1.tar.gz
.
File metadata
- Download URL: airflow-socrata-plugin-0.0.1.tar.gz
- Upload date:
- Size: 3.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ea6de2951a5b35b14170e08bb11dc7a769a0634e0fcd757b4f3ae38a4687ab1a |
|
MD5 | 17d7f7a1269358ee196212ea76398556 |
|
BLAKE2b-256 | ae53fcc697cd5af3b3b972e886aa5c17edc56fb5e70c6be01ed5e2f645c9f03d |
File details
Details for the file airflow_socrata_plugin-0.0.1-py3-none-any.whl
.
File metadata
- Download URL: airflow_socrata_plugin-0.0.1-py3-none-any.whl
- Upload date:
- Size: 5.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8f6db3dbe93708a8c6398169374271e6f70b46e31eb4df55d8f3fc07a853107 |
|
MD5 | 0da03e687507986f6029cf31aad75373 |
|
BLAKE2b-256 | a69391f0ed3770feaf7a8791809eb79178ee22a8f1287e5435ea6e59eb85b838 |