Skip to main content

Extension for `kedro` datasets

Project description

kedroio

A module extending the datasets that come shipped with kedro

Code style: black pre-commit

Example usage

-- example.sql
select *
from "database"."table_name"
limit 5;
# conf/base/catalog.py
my_athena_dataset:
  type: kedroio.datasets.aws.athena.AthenaQueryDataSet
  filepath: data/01_raw/example.csv
  sql_filepath: example.sql
  bucket: example-bucket
  workgroup: primary
  subfolder: data
  region_name: eu-west-2
  read_result: true # read into pandas DataFrame
  overwrite: false # skip download if filepath exists

Testing

Start moto server for mocked AWS resources

moto_server

Run tests

pytest tests/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kedroio-0.1.3.tar.gz (10.8 kB view hashes)

Uploaded Source

Built Distribution

kedroio-0.1.3-py3-none-any.whl (13.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page