An Apache Airflow plugin to upload to Socrata.
Project description
Airflow-Socrata
Simple hooks and operators for uploading data to Socrata.
Features
- Upsert or reupload PostgreSQL tables to Socrata
Install
Using pip:
pip3 install airflow-socrata-plugin
Usage
Create a connection named http_socrata of type http to store Socrata credentials. You can also pass in conn_name parameter in DAG definition to override.
Create a connection named etl_postgres of type postgres to store PostgreSQL credentials. You can also pass in postgres_conn_name parameter in DAG definition to override.
By default, the plugin looks for the specified table under public schema. The schema can be specified with postgres_schema parameter.
The plugin is published as a pip package. Refer to the example DAG for available parameters.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file airflow-socrata-plugin-0.0.1.tar.gz.
File metadata
- Download URL: airflow-socrata-plugin-0.0.1.tar.gz
- Upload date:
- Size: 3.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ea6de2951a5b35b14170e08bb11dc7a769a0634e0fcd757b4f3ae38a4687ab1a
|
|
| MD5 |
17d7f7a1269358ee196212ea76398556
|
|
| BLAKE2b-256 |
ae53fcc697cd5af3b3b972e886aa5c17edc56fb5e70c6be01ed5e2f645c9f03d
|
File details
Details for the file airflow_socrata_plugin-0.0.1-py3-none-any.whl.
File metadata
- Download URL: airflow_socrata_plugin-0.0.1-py3-none-any.whl
- Upload date:
- Size: 5.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a8f6db3dbe93708a8c6398169374271e6f70b46e31eb4df55d8f3fc07a853107
|
|
| MD5 |
0da03e687507986f6029cf31aad75373
|
|
| BLAKE2b-256 |
a69391f0ed3770feaf7a8791809eb79178ee22a8f1287e5435ea6e59eb85b838
|
