Skip to main content

An Apache Airflow plugin to upload to Socrata.

Project description

Airflow-Socrata

Simple hooks and operators for uploading data to Socrata.

Features

  • Upsert or reupload PostgreSQL tables to Socrata

Install

Using pip:

pip3 install airflow-socrata-plugin

Usage

Create a connection named http_socrata of type http to store Socrata credentials. You can also pass in conn_name parameter in DAG definition to override.

Create a connection named etl_postgres of type postgres to store PostgreSQL credentials. You can also pass in postgres_conn_name parameter in DAG definition to override.

By default, the plugin looks for the specified table under public schema. The schema can be specified with postgres_schema parameter.

The plugin is published as a pip package. Refer to the example DAG for available parameters.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflow-socrata-plugin-0.0.1.tar.gz (3.4 kB view details)

Uploaded Source

Built Distribution

airflow_socrata_plugin-0.0.1-py3-none-any.whl (5.9 kB view details)

Uploaded Python 3

File details

Details for the file airflow-socrata-plugin-0.0.1.tar.gz.

File metadata

  • Download URL: airflow-socrata-plugin-0.0.1.tar.gz
  • Upload date:
  • Size: 3.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3

File hashes

Hashes for airflow-socrata-plugin-0.0.1.tar.gz
Algorithm Hash digest
SHA256 ea6de2951a5b35b14170e08bb11dc7a769a0634e0fcd757b4f3ae38a4687ab1a
MD5 17d7f7a1269358ee196212ea76398556
BLAKE2b-256 ae53fcc697cd5af3b3b972e886aa5c17edc56fb5e70c6be01ed5e2f645c9f03d

See more details on using hashes here.

File details

Details for the file airflow_socrata_plugin-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: airflow_socrata_plugin-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 5.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3

File hashes

Hashes for airflow_socrata_plugin-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 a8f6db3dbe93708a8c6398169374271e6f70b46e31eb4df55d8f3fc07a853107
MD5 0da03e687507986f6029cf31aad75373
BLAKE2b-256 a69391f0ed3770feaf7a8791809eb79178ee22a8f1287e5435ea6e59eb85b838

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page