Pandas on AWS.
Project description
AWS Data Wrangler
Pandas on AWS
NOTE
Due the new major version 1.0.0
with breaking changes, please make sure that all your old projects has dependencies frozen on the desired version (e.g. pip install awswrangler==0.3.2
).
Source | Downloads | Page | Installation Command |
---|---|---|---|
PyPi | Link | pip install awswrangler |
|
Conda | Link | conda install -c conda-forge awswrangler |
Quick Start
Install the Wrangler with: pip install awswrangler
import awswrangler as wr
import pandas as pd
df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
# Storing data on Data Lake
wr.s3.to_parquet(
df=df,
path="s3://bucket/dataset/",
dataset=True,
database="my_db",
table="my_table"
)
# Retrieving the data directly from Amazon S3
df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)
# Retrieving the data from Amazon Athena
df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")
# Getting Redshift connection (SQLAlchemy) from Glue Catalog Connections
engine = wr.catalog.get_engine("my-redshift-connection")
# Retrieving the data from Amazon Redshift Spectrum
df = wr.db.read_sql_query("SELECT * FROM external_schema.my_table", con=engine)
Read The Docs
- What is AWS Data Wrangler?
- Install
- Tutorials
- 01 - Introduction
- 02 - Sessions
- 03 - Amazon S3
- 04 - Parquet Datasets
- 05 - Glue Catalog
- 06 - Amazon Athena
- 07 - Databases (Redshift, MySQL and PostgreSQL)
- 08 - Redshift - Copy & Unload.ipynb
- 09 - Redshift - Append, Overwrite and Upsert
- 10 - Parquet Crawler
- 11 - CSV Datasets
- 12 - CSV Crawler
- 13 - Merging Datasets on S3
- 15 - EMR
- 16 - EMR & Docker
- API Reference
- License
- Contributing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
awswrangler-1.1.2.tar.gz
(55.7 kB
view hashes)
Built Distributions
awswrangler-1.1.2-py3.6.egg
(132.2 kB
view hashes)
Close
Hashes for awswrangler-1.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9450b132e688736e4c5dff5ab3e3574bfd8fcd788f47763311115d778673bc2f |
|
MD5 | 2cbfdff08bd359c1d74e83546ca8b60c |
|
BLAKE2b-256 | 223337e44636c942ac2dfb34d8c3ff8b10055a3b51a98ad61cf1a95a38660016 |