Skip to main content

Sparkless

PySpark-style DataFrames in Python. No JVM required.

PyPI Python versions Documentation

Sparkless brings familiar DataFrame and SQL APIs to local development and testing. Its Rust/Polars engine runs your transformations without Java or a Spark cluster.

Install

Install in a virtual environment with Python 3.11 or newer. Sparkless installs the matching sparkless-native engine automatically. Check the platform table for native wheel targets and source build requirements.

python -m pip install "sparkless==4.20.0"

Quick start

from sparkless.sql import SparkSession, functions as F

spark = SparkSession.builder.app_name("example").get_or_create()
df = spark.createDataFrame([("Alice", 25), ("Bob", 30)], ["name", "age"])
df.filter(F.col("age") >= 30).select("name").show()
spark.stop()

The example prints Bob. Use the same session for joins, aggregations, windows, SQL, and file I/O. Sparkless runs on one machine and implements a subset of PySpark. See compatibility and limitations.

Workflow Install
Run the same pytest suite with Sparkless and local PySpark sparkless[testing]==4.20.0
Start a local Polars server and a Spark Connect session sparkless-server[connect]==4.20.0
Use the DuckDB server product sparkless-server-duckdb[connect]==4.20.0
Connect to a server you already run sparkless-connect[client]==4.20.0

The server packages have their own wheel targets. Check the installation matrix before adding a server dependency.

Learn more

Guide What you'll find
Getting started Installation, extras, and examples
Python API DataFrames, functions, SQL, and Arrow export
Testing Run the same tests with Sparkless and PySpark
Spark Connect Native server setup and supported plans

Full documentation · Source and contributing · Issues · MIT license.

Metadata

Release files for sparkless 4.20.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sparkless 4.20.0
File Size Uploaded Provenance Actions
sparkless-4.20.0.tar.gz 43.3 kB With provenance Details for sparkless-4.20.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sparkless 4.20.0
File Interpreter ABI Platform Actions
sparkless-4.20.0-py3-none-any.whl Python 3 none any Details for sparkless-4.20.0-py3-none-any.whl

Total release size: 93.3 kB

Release files / sparkless-4.20.0.tar.gz

Download URL sparkless-4.20.0.tar.gz
Size 43.3 kB
Tags Source
SHA-256 checksum
How to use checksums
4857dc070fa1acdf8b5bdf0d3fb1d613007a4430f06b6239051be55bcc952921
BLAKE2b-256 checksum
How to use checksums
44932dc186abd165f7648d828443595f8b155aeea3450a6efe1bb290fe14532c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

Release files / sparkless-4.20.0-py3-none-any.whl

Download URL sparkless-4.20.0-py3-none-any.whl
Size 50.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
389ad4aa1c5c3390bc3cd1e999c06c44a7a56994a5722f48823fec9a9938f3c6
BLAKE2b-256 checksum
How to use checksums
007c69fc64e83a0761356fa0de71844f19494ef2e28acb2569979e6d39c3ade3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

Security for sparkless 4.20.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for sparkless-4.20.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.20.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page