Skip to main content
Python Software Foundation 20th Year Anniversary Fundraiser  Donate today!

An Apache Airflow plugin to upload to Socrata.

Project description

Airflow-Socrata

Simple hooks and operators for uploading data to Socrata.

Features

  • Upsert or reupload PostgreSQL tables to Socrata

Install

Using pip:

pip3 install airflow-socrata-plugin

Usage

Create a connection named http_socrata of type http to store Socrata credentials. You can also pass in conn_name parameter in DAG definition to override.

Create a connection named etl_postgres of type postgres to store PostgreSQL credentials. You can also pass in postgres_conn_name parameter in DAG definition to override.

By default, the plugin looks for the specified table under public schema. The schema can be specified with postgres_schema parameter.

The plugin is published as a pip package. Refer to the example DAG for available parameters.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for airflow-socrata-plugin, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size airflow_socrata_plugin-0.0.1-py3-none-any.whl (5.9 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size airflow-socrata-plugin-0.0.1.tar.gz (3.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page