Skip to main content

A local client for conecting and working with Snowflake

Project description

shipyard-snowflake

The shipyard-snowflake Python Package is a set of utility classes designed to streamline interactions with Snowflake, a cloud-based data warehousing platform. This package offers two main client classes, SnowflakeClient and SnowparkClient, each catering to different use cases and functionalities within the Snowflake ecosystem.

Table of Contents

Installation

You can install shipyard-snowflake using pip:

python3 -m pip install shipyard-snowflake

Usage

SnowflakeClient

The SnowflakeClient class provides methods to interact with Snowflake using the Snowflake Python Connector. It supports functionalities such as authentication, data uploading, executing queries, and fetching results. Here's an example of how to use it:

import pandas as pd
from shipyard_snowflake import SnowflakeClient
from shipyard_snowflake.utils import utils
# Initialize the SnowflakeClient with credentials
client = SnowflakeClient(
    username="your_username",
    pwd="your_password",
    account="your_account_id",
    warehouse="your_warehouse",
    database="your_database",
    schema="your_schema",
    rsa_key="your_rsa_key",  # Optional, use either rsa_key or pwd
    role="your_role"
)

# Connect to Snowflake
client.connect()

Upload a file to Create or Replace a Table

# Upload a csv to Snowflake to replace a table
# create the table first
data_types = utils.infer_schema(file_name = "<file_path>")
sql = client._create_table_sql(table_name = "<table_name>", columns = data_types)
client.create_table(sql)
# load the file to the table
client.upload(file_path = "<file_path>", table_name = "<table_name>", insert_method = "replace")

Upload a file to append to an existing table

client.upload(file_path = "<file_path>", table_name = "<table_name>", insert_method = "append")

### Fetch the results of a query as a pandas dataframe
```python
query = "SELECT * FROM your_table_name"
result_df = client.execute_query(query=query)

Execute a Query

query = "DROP TABLE IF EXISTS SALES"
result_df = client.fetch(query = query)

Close the connection

# Close the connection
client.conn.close()

SnowparkClient

The SnowparkClient class allows interaction with Snowflake using Snowpark, a modern data processing and scripting framework. It enables data upload and some query execution capabilities. Here's an example of how to use it:

from shipyard_snowflake import SnowparkClient

# Initialize the SnowparkClient with credentials
client = SnowparkClient(
    username="your_username",
    pwd="your_password",
    account="your_account_url",
    warehouse="your_warehouse", # optional
    database="your_database", # optional
    schema="your_schema", # optional
    role="your_role" # optional
)

# Connect to Snowflake using Snowpark
session = client.connect()

# Upload a DataFrame to Snowflake
import pandas as pd
df = pd.DataFrame(...)  # Your DataFrame
client.upload(
    session=session,
    df=df,
    table_name="your_table_name",
    overwrite=True # False for append
)
# Close the session
session.close()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shipyard_snowflake-0.2.8.tar.gz (14.0 kB view details)

Uploaded Source

Built Distribution

shipyard_snowflake-0.2.8-py3-none-any.whl (18.9 kB view details)

Uploaded Python 3

File details

Details for the file shipyard_snowflake-0.2.8.tar.gz.

File metadata

  • Download URL: shipyard_snowflake-0.2.8.tar.gz
  • Upload date:
  • Size: 14.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.9.16 Darwin/22.5.0

File hashes

Hashes for shipyard_snowflake-0.2.8.tar.gz
Algorithm Hash digest
SHA256 21ab66c715605995c8c9e563e7c7c09ada115e23ce5c2702edfcb760f6b5e8d4
MD5 1cca5dbd41b74257292bbbb4c0b179ec
BLAKE2b-256 ad5a2daa10f34758318dda52127acf73d7cb2e7ada12942debfc28ebd60c1ad6

See more details on using hashes here.

File details

Details for the file shipyard_snowflake-0.2.8-py3-none-any.whl.

File metadata

File hashes

Hashes for shipyard_snowflake-0.2.8-py3-none-any.whl
Algorithm Hash digest
SHA256 4ab6b5eccc6643a0fbf92a4b9d018953aa9ee58daf25cc1e2fbad36bc7ad5714
MD5 fcf2a69a6edd687f66f6c54d578e7dae
BLAKE2b-256 4cac040aaa45f7da54bea2d0be6776554828de5b2af249b4a90da767ceaeeb1d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page