Skip to main content

An Apache Airflow plugin to upload to Socrata.

Project description

Airflow-Socrata

Simple hooks and operators for uploading data to Socrata.

Features

  • Upsert or reupload PostgreSQL tables to Socrata

Install

Using pip:

pip3 install airflow-socrata-plugin

Usage

Create a connection named http_socrata of type http to store Socrata credentials. You can also pass in conn_name parameter in DAG definition to override.

Create a connection named etl_postgres of type postgres to store PostgreSQL credentials. You can also pass in postgres_conn_name parameter in DAG definition to override.

By default, the plugin looks for the specified table under public schema. The schema can be specified with postgres_schema parameter.

The plugin is published as a pip package. Refer to the example DAG for available parameters.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflow-socrata-plugin-0.0.1.tar.gz (3.4 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page