finai-snowflake-transform

A small public package used to verify that a PyPI release, including YAML files bundled inside it, can be installed and executed inside Snowflake through both Snowpark and pandas on Snowflake.

Transformation

Parameters are read from a YAML file bundled in the package, so nothing depends on the working directory:

TRANSFORMED_VALUE = fill_null(VALUE, null_fill) * scale + offset

Install

pip install finai-snowflake-transform
# with the Snowflake runtime extra
pip install "finai-snowflake-transform[snowflake]"

Use

ID and VALUE are preserved and TRANSFORMED_VALUE is appended. Arithmetic stays native to the engine you pass in; the caller reads and writes tables.

from finai_snowflake_transform import (
    load_config,
    transform_pandas_on_snowflake,
    transform_snowpark,
)

load_config("defaults.yaml")
# {'scale': 2.0, 'offset': 1.0, 'null_fill': 0.0}

# pandas on Snowflake (also works with plain pandas)
import modin.pandas as pd
import snowflake.snowpark.modin.plugin  # noqa: F401

out = transform_pandas_on_snowflake(pd.read_snowflake("MY_INPUT"), "defaults.yaml")

# Snowpark
out = transform_snowpark(session.table("MY_INPUT"), "alternate.yaml")

Invalid configuration names, non-mapping YAML, missing parameters, booleans and non-finite numbers all raise immediately with a message naming the config file.

License

MIT