PyStarburst DataFrame API allows you to query and transform data in Starburst products in a data pipeline without having to download the data locally.
Project description
PyStarburst DataFrame API
PyStarburst DataFrame API allows you to query and transform data in Starburst products in a data pipeline without having to download the data locally.
Documentation
See PyStarburst API documentation here.
Getting started
Install pystarburst
pip install pystarburst
Connect to a Starburst server
The parameters are the same connect parameters as in Trino Python Client.
from pystarburst import Session
connection_parameters = {
"host": "localhost",
"port": 8080,
"user": "admin",
"catalog": "tpch",
"schema": "tiny"
}
session = Session.builder.configs(connection_parameters).create()
Using SQL
from pystarburst import Session
session = Session.builder.configs({ ... }).create()
session.sql("SELECT 1 as a").show()
Querying a table
from pystarburst import Session
session = Session.builder.configs({ ... }).create()
df = session.table("nation")
print(df.schema)
df.show()
Filtering a data frame
from pystarburst import Session
session = Session.builder.configs({ ... }).create()
df = session.table("nation")
df.filter(df.col("regionkey") == 0).show()
Joining data frames
from pystarburst import Session
session = Session.builder.configs({ ... }).create()
df = session.table("nation")
df.filter(df.col("regionkey") == 0).show()
Aggregation
from pystarburst import Session
from pystarburst.functions import col
session = Session.builder.configs({ ... }).create()
df = session.table("nation")
df.agg((col("regionkey"), "max"), (col("regionkey"), "avg")).show()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
pystarburst-0.6.2-py3-none-any.whl
(126.4 kB
view hashes)
Close
Hashes for pystarburst-0.6.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b570c96c15411b09e2cc9bbec48116964f9fb11a813eaaf9c7f6e7682e38f9e7 |
|
MD5 | fcb311166b0fc87dadc8c2383c77bb73 |
|
BLAKE2b-256 | dbb65766609bb3d887551ddd46e2396c2348df414b9f414c21d30d4cfd0a5aed |