Pandas on AWS.
Project description
AWS Data Wrangler
Pandas on AWS
NOTE
Due the new major version 1.*.*
with breaking changes, please make sure that all your old projects has dependencies frozen on the desired version (e.g. pip install awswrangler==0.3.2
).
Source | Downloads | Page | Installation Command |
---|---|---|---|
PyPi | Link | pip install awswrangler |
|
Conda | Link | conda install -c conda-forge awswrangler |
Quick Start
Install the Wrangler with: pip install awswrangler
import awswrangler as wr
import pandas as pd
df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
# Storing data on Data Lake
wr.s3.to_parquet(
df=df,
path="s3://bucket/dataset/",
dataset=True,
database="my_db",
table="my_table"
)
# Retrieving the data directly from Amazon S3
df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)
# Retrieving the data from Amazon Athena
df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")
# Getting Redshift connection (SQLAlchemy) from Glue Catalog Connections
engine = wr.catalog.get_engine("my-redshift-connection")
# Retrieving the data from Amazon Redshift Spectrum
df = wr.db.read_sql_query("SELECT * FROM external_schema.my_table", con=engine)
Read The Docs
- What is AWS Data Wrangler?
- Install
- Tutorials
- 01 - Introduction
- 02 - Sessions
- 03 - Amazon S3
- 04 - Parquet Datasets
- 05 - Glue Catalog
- 06 - Amazon Athena
- 07 - Databases (Redshift, MySQL and PostgreSQL)
- 08 - Redshift - Copy & Unload.ipynb
- 09 - Redshift - Append, Overwrite and Upsert
- 10 - Parquet Crawler
- 11 - CSV Datasets
- 12 - CSV Crawler
- 13 - Merging Datasets on S3
- 15 - EMR
- 16 - EMR & Docker
- API Reference
- License
- Contributing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
awswrangler-1.1.0.tar.gz
(54.6 kB
view hashes)
Built Distributions
awswrangler-1.1.0-py3.6.egg
(130.2 kB
view hashes)
Close
Hashes for awswrangler-1.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 571ebf91fdf76a05a9ab5dbd45e81947134b99324f857c57f4b9de80a3f9adfb |
|
MD5 | fcdaee82b7d35368d24b492ac6ccc530 |
|
BLAKE2b-256 | a0b8f6c654123fa42480920d1e32930315a5185dcad3d8e8380df08f99fd7ae5 |