An Apache Airflow plugin to upload to Socrata.
Project description
Airflow-Socrata
Simple hooks and operators for uploading data to Socrata.
Features
- Upsert or reupload PostgreSQL tables to Socrata
Install
Using pip:
pip3 install airflow-socrata-plugin
Usage
Create a connection named http_socrata
of type http
to store Socrata credentials. You can also pass in conn_name
parameter in DAG definition to override.
Create a connection named etl_postgres
of type postgres
to store PostgreSQL credentials. You can also pass in postgres_conn_name
parameter in DAG definition to override.
By default, the plugin looks for the specified table under public
schema. The schema can be specified with postgres_schema
parameter.
The plugin is published as a pip package. Refer to the example DAG for available parameters.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for airflow-socrata-plugin-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ea6de2951a5b35b14170e08bb11dc7a769a0634e0fcd757b4f3ae38a4687ab1a |
|
MD5 | 17d7f7a1269358ee196212ea76398556 |
|
BLAKE2b-256 | ae53fcc697cd5af3b3b972e886aa5c17edc56fb5e70c6be01ed5e2f645c9f03d |
Hashes for airflow_socrata_plugin-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8f6db3dbe93708a8c6398169374271e6f70b46e31eb4df55d8f3fc07a853107 |
|
MD5 | 0da03e687507986f6029cf31aad75373 |
|
BLAKE2b-256 | a69391f0ed3770feaf7a8791809eb79178ee22a8f1287e5435ea6e59eb85b838 |