Skip to main content

Extension for `kedro` datasets

Project description

kedroio

A module extending the datasets that come shipped with kedro

Code style: black pre-commit

Example usage

-- example.sql
select *
from "database"."table_name"
limit 5;
# conf/base/catalog.py
my_athena_dataset:
  type: kedroio.datasets.aws.athena.AthenaQueryDataSet
  filepath: data/01_raw/example.csv
  sql_filepath: example.sql
  bucket: example-bucket
  workgroup: primary
  subfolder: data
  region_name: eu-west-2
  read_result: true # read into pandas DataFrame
  overwrite: false # skip download if filepath exists

Testing

Start moto server for mocked AWS resources

moto_server

Run tests

pytest tests/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kedroio-0.1.3.tar.gz (10.8 kB view details)

Uploaded Source

Built Distribution

kedroio-0.1.3-py3-none-any.whl (13.4 kB view details)

Uploaded Python 3

File details

Details for the file kedroio-0.1.3.tar.gz.

File metadata

  • Download URL: kedroio-0.1.3.tar.gz
  • Upload date:
  • Size: 10.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.13 CPython/3.8.11 Darwin/21.1.0

File hashes

Hashes for kedroio-0.1.3.tar.gz
Algorithm Hash digest
SHA256 355cde317a9f5d2dbd2b952fe9c9a6ab119ef184026d82a1245687ebca44e671
MD5 93876b6195ad3bdea7e34be1ac508787
BLAKE2b-256 758699c9e97f3a279a6a2186019d2b4f35d76491f9dc783578d51f9426ff4292

See more details on using hashes here.

File details

Details for the file kedroio-0.1.3-py3-none-any.whl.

File metadata

  • Download URL: kedroio-0.1.3-py3-none-any.whl
  • Upload date:
  • Size: 13.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.13 CPython/3.8.11 Darwin/21.1.0

File hashes

Hashes for kedroio-0.1.3-py3-none-any.whl
Algorithm Hash digest
SHA256 5ff9af1c7711138c8821ae0828a852b7840549e51ec31e3d8de1eb674b009ec1
MD5 189551fe902386cc4946a20be5205c7d
BLAKE2b-256 71146b93ef55f4fb3d7541a543dba1136e41171c1fd51cd5d3f7029f8eff96ab

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page