A set of helpers to extend snowpark functionality
Project description
snowpark-extensions-py
Snowpark by itself is a powerful library, but still some utility functions can always help.
Currently provided extensions:
SessionBuilder extensions
Name | Description |
---|---|
SessionBuilder.from_snowsql | can read the information from the snowsql config file by default at ~/snowsql/config or at a given location |
SessionBuilder.env | reads settings from SNOW_xxx or SNOWSQL_xxx variables |
SessionBuilder.appName | Sets a query tag with the given appName |
SessionBuilder.append_tag | Appends a new tag to the existing query tag |
You can the create your session like:
from snowflake.snowpark import Session
import snowpark_extensions
new_session = Session.builder.from_snowsql().appName("app1").create()
from snowflake.snowpark import Session
import snowpark_extensions
new_session = Session.builder.env().appName("app1").create()
The appName
can use to setup a query_tag like APPNAME=tag;execution_id=guid
which can then be used to track job actions with a query like
You can then use a query like: To see all executions from an app or
select *
from table(information_schema.query_history())
whery query_tag like '%APPNAME=tag%'
order by start_time desc;
To see the executions for a particular execution:
select *
from table(information_schema.query_history())
whery query_tag like '%APPNAME=tag;execution_id=guid%'
order by start_time desc;
Column Extensions
Name | Description |
---|---|
Column.getItem | An expression that gets an item at position ordinal out of a list, or gets an item by key out of a dict. |
DataFrame Extensions
Name | Description |
---|---|
DataFrame.dtypes | returns the list of datatypes in the DataFrame |
DataFrame.map | provides an equivalent for the map function for example df.map(func,input_types=[StringType(),StringType()],output_types=[StringType(),IntegerType()],to_row=True) |
DataFrame.simple_map | if a simple lambda like lambda x: x.col1 + x.col2 is used this functions can be used like df.simple_map(lambda x: x.col1 + x.col2) |
DataFrame.groupby.applyInPandas | Maps each group of the current DataFrame using a pandas udf and returns the result as a DataFrame. |
Functions Extensions
Name | Description |
---|---|
functions.array_sort | sorts the input array in ascending order. The elements of the input array must be orderable. Null elements will be placed at the end of the returned array. |
functions.unix_timestamp | returns the UNIX timestamp of current time. |
functions.from_unixtimestamp | can be used to convert UNIX time to Snowflake timestamp |
functions.format_number | formats numbers using the specified number of decimal places |
functions.reverse | returns a reversed string |
functions.explode | returns a new row for each element in the given array |
functions.date_add | returns the date that is n days days after |
functions.date_sub | returns the date that is n days before |
Usage:
just import it at the top of your file and it will automatically extend your snowpark package
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for snowpark_extensions-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b67367dc006ab55e984419f06b0b8777ea0a5e6f840d55846c2521e5ba067273 |
|
MD5 | 02a30f5f7f23fcfaf83343a63bd2a3c7 |
|
BLAKE2b-256 | 6adae6ac761e278c53e3dc7b74bdba104aeda46d65483ecfb537a5a2593561a2 |
Close
Hashes for snowpark_extensions-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f087a0b935b3de6d82dc5fa0e73b2f05ddd98193d3f5efd7bb9743046e07bc11 |
|
MD5 | f004f8a7ed35c778f29d5a9d3fbb1a23 |
|
BLAKE2b-256 | c0780b895c176c4df32313a1fb1faee3acd0c8a7d1e91b7e06640a9d7b0713a5 |