Skip to main content

sparkless-server-duckdb

Run the Sparkless server with trusted, read-only DuckDB table sources. Install the connect extra for the pinned pyspark-client==4.0.5.

python -m pip install "sparkless-server-duckdb[connect]==4.20.0"
from sparkless_server_duckdb import SparklessConnect

with SparklessConnect() as spark:
    assert spark.range(10).filter("id > 5").count() == 4

This example uses the default Polars backend. Configure SPARKLESS_DUCKDB_SOURCES on the server to expose read-only table aliases. See the DuckDB source guide.

Run Spark Connect plans in DuckDB

Set these variables before starting the server. Use a fresh database for this example.

export SPARKLESS_EXECUTION_BACKEND=duckdb
export SPARKLESS_DUCKDB_EXECUTION_OWNER='{"mode":"local","database_path":"./events.duckdb"}'
from sparkless_server_duckdb import SparklessConnect

with SparklessConnect() as spark:
    spark.sql("CREATE TABLE events (id BIGINT)").collect()
    spark.sql("INSERT INTO events VALUES (1)").collect()
    assert spark.table("events").collect()[0].id == 1

The execution owner holds writable data separately from the read-only source aliases. In DuckDB mode, use Spark Connect for SQL.

See the DuckDB execution guide for a local database quickstart, shared Quack ownership, and supported operations.

Metadata

Release files for sparkless-server-duckdb 4.20.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for sparkless-server-duckdb 4.20.0
File Interpreter ABI Platform Actions
sparkless_server_duckdb-4.20.0-py3-none-manylinux_2_28_x86_64.whl Python 3 none Linux glibc 2.28+ x86-64 Details for sparkless_server_duckdb-4.20.0-py3-none-manylinux_2_28_x86_64.whl
sparkless_server_duckdb-4.20.0-py3-none-macosx_14_0_arm64.whl Python 3 none macOS 14.0+ ARM64 Details for sparkless_server_duckdb-4.20.0-py3-none-macosx_14_0_arm64.whl

Total release size: 145.1 MB

Release files / sparkless_server_duckdb-4.20.0-py3-none-manylinux_2_28_x86_64.whl

Download URL sparkless_server_duckdb-4.20.0-py3-none-manylinux_2_28_x86_64.whl
Size 78.5 MB
Tags Linux glibc 2.28+ x86-64 Python 3
SHA-256 checksum
How to use checksums
5dbc2d09fdb540ea07fec5ef4ee286aa045a0fd73e3826cd363391ba842f99be
BLAKE2b-256 checksum
How to use checksums
82d20c59c8356fa2cdb567bad043a14727e9ec1aa4cbab74b09d3d1c16a45995
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / sparkless_server_duckdb-4.20.0-py3-none-macosx_14_0_arm64.whl

Download URL sparkless_server_duckdb-4.20.0-py3-none-macosx_14_0_arm64.whl
Size 66.6 MB
Tags Python 3 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
c4dca240cc32bb8d3d67dc3a5fa0d40e6ba1f609d28b4eddd3d940d08d8d506d
BLAKE2b-256 checksum
How to use checksums
aee0eb0a18a138f717f24c6969b871f565be162726c2f6e66c4017a1a2a2b9d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for sparkless-server-duckdb 4.20.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.20.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page