Codenize your data sources
Project description
akagi
Free software: MIT license
Features
akagi supports iter and save interface for various data sources such as Amazon Redshift, Amazon S3 (more in future).
Installation
Install via pip:
pip install akagi
or from source:
$ git clone https://github.com/ayemos/akagi akagi $ cd akagi $ python setup.py install
Example
RedshiftDataSource
with RedshiftDataSource.for_query(
'log-redshift-unload.ap-northeast-1', # S3 Bucket for intermediate storage
'select * from (select user_id, path from logs.imp limit 10000)', # Your Query here
'logs', # schema
'imp' # table (Those two are used to generate unique prefix for S3 object (e.g. logs/imp/20170312_081527)
) as ds:
ds.save('./akagi_test') # save results to local
for d in ds:
print(d) # iterate on result
S3DataSource
with S3DataSource.for_prefix(
'image-data.ap-northeast-1',
'data/image_net/zebra',
FileFormat.BINARY) as ds:
...
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
akagi-0.1.11-py3.5.egg
(31.3 kB
view hashes)