Snowflake Spark Connect Python Client
pip install snowflake-spark-connect
Usage
from snowflake.sparkconnect import SparkSession
spark = SparkSession.builder.getOrCreate()
spark.sql("SELECT 1").show()
spark.stop()
getOrCreate() starts a Spark Connect session in Snowflake, returning once the session is ready for use. spark.stop() ends the Spark Connect session in Snowflake, releasing server-side resources.
Builder options
Authentication
Use exactly one of the following authentication options when creating a session to customize the connection.
| Method | What it does |
|---|---|
| (nothing) | getOrCreate() returns the default connection in the environment |
.snowpark_session(session) |
Use an existing Snowpark session object |
.connection("prod") |
Named profile from ~/.snowflake/connections.toml |
.remote(account=..., ...) |
Snowflake connection parameters (see below) |
Please refer to Snowflake connection configuration docs for how to use connections.toml.
Once a Spark Connect session is established, token refresh happens in the background to ensure the session stays authenticated.
Session lifetime
| Option | Meaning |
|---|---|
.idlettl(timedelta(...)) |
Stop the Spark server after this much idle time |
.sessionttl(timedelta(...)) |
Hard cap on total session lifetime |
.tokenttl(timedelta(...)) |
Lifetime of each Spark Connect JWT used for authentication |
spark_idlettl / spark_sessionttl / spark_tokenttl inside .remote(...) |
Same TTLs, passed as Snowflake kwargs |
Database and schema
Spark Connect sessions must be started with a database and schema. .database(name) and .schema(name) on the builder can be used to specify a database and schema combination to use. An exception is raised if only one of database and schema are provided by any source (e.g explicitly or via a connection profile). When neither is set, the user's personal database USER$<current user>.PUBLIC is used by default.
| Option | Meaning |
|---|---|
.database(name) |
Use this Snowflake database for the Spark Connect server |
.schema(name) |
Use this Snowflake schema for the Spark Connect server |
Explicit .database(...) and .schema(...) values always override database/schema from a connection, configuration, or supplied/active Snowpark session.
Artifact repositories and Python packages
To use third-party Python packages in UDFs, start the Spark Connect session with an Artifact Repository attached and a listing of all desired packages.
| Option | Meaning |
|---|---|
.artifact_repositories([...]) |
Artifact Repository names, bound as ARTIFACT_REPOSITORIES |
.python_packages([...]) |
Pip specs, bound as PYTHON_PACKAGES |
spark_artifact_repositories / spark_python_packages inside .remote(...) |
Same values, passed as Snowflake kwargs |
.remote(**kwargs) Snowflake parameters
The same parameters as snowpark-python are accepted. Additional, Spark-specific
parameters are as follows:
| Parameter | Notes |
|---|---|
spark_idlettl / spark_sessionttl / spark_tokenttl |
Same as .idlettl() / .sessionttl() / .tokenttl() |
spark_artifact_repositories / spark_python_packages |
Same as .artifact_repositories() / .python_packages() |
3 maintainers
Metadata
3 maintainers
Release files for snowflake-spark-connect 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| snowflake_spark_connect-1.0.0.tar.gz | 1.9 MB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| snowflake_spark_connect-1.0.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 4.4 MB
Release files / snowflake_spark_connect-1.0.0.tar.gz
| Download URL | snowflake_spark_connect-1.0.0.tar.gz |
|---|---|
| Size | 1.9 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c282b72ff57187af38132039bbc509bb2b046eb68d5e2c11676b8fd034ffa430
|
|
BLAKE2b-256 checksum How to use checksums |
c94b53bde0561e1c90f85829043af81ae7dc9eb0e63ef9448a648e95fddf1b0a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.13 {"installer":{"name":"uv","version":"0.9.13"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / snowflake_spark_connect-1.0.0-py3-none-any.whl
| Download URL | snowflake_spark_connect-1.0.0-py3-none-any.whl |
|---|---|
| Size | 2.5 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b0349ac67cb4a4e993c0a5bb437d1506bf8b5f180967cb64968fe97a6ed3210b
|
|
BLAKE2b-256 checksum How to use checksums |
d08472e43027afe210df9431a5bf1cacd8481042aeb27a0e52116bcce9e916aa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.13 {"installer":{"name":"uv","version":"0.9.13"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for snowflake-spark-connect 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
