Extension for `kedro` datasets
Project description
kedroio
A module extending the datasets that come shipped with kedro
Example usage
-- example.sql
select *
from "database"."table_name"
limit 5;
# conf/base/catalog.py
my_athena_dataset:
type: kedroio.datasets.aws.athena.AthenaQueryDataSet
filepath: data/01_raw/example.csv
sql_filepath: example.sql
bucket: example-bucket
workgroup: primary
subfolder: data
region_name: eu-west-2
read_result: true # read into pandas DataFrame
overwrite: false # skip download if filepath exists
Testing
Start moto
server for mocked AWS resources
moto_server
Run tests
pytest tests/
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kedroio-0.1.3.tar.gz
(10.8 kB
view details)
Built Distribution
kedroio-0.1.3-py3-none-any.whl
(13.4 kB
view details)
File details
Details for the file kedroio-0.1.3.tar.gz
.
File metadata
- Download URL: kedroio-0.1.3.tar.gz
- Upload date:
- Size: 10.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.1.13 CPython/3.8.11 Darwin/21.1.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 355cde317a9f5d2dbd2b952fe9c9a6ab119ef184026d82a1245687ebca44e671 |
|
MD5 | 93876b6195ad3bdea7e34be1ac508787 |
|
BLAKE2b-256 | 758699c9e97f3a279a6a2186019d2b4f35d76491f9dc783578d51f9426ff4292 |
File details
Details for the file kedroio-0.1.3-py3-none-any.whl
.
File metadata
- Download URL: kedroio-0.1.3-py3-none-any.whl
- Upload date:
- Size: 13.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.1.13 CPython/3.8.11 Darwin/21.1.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ff9af1c7711138c8821ae0828a852b7840549e51ec31e3d8de1eb674b009ec1 |
|
MD5 | 189551fe902386cc4946a20be5205c7d |
|
BLAKE2b-256 | 71146b93ef55f4fb3d7541a543dba1136e41171c1fd51cd5d3f7029f8eff96ab |