Skip to main content

Sparkless

PySpark-style DataFrames in Python. No JVM required.

PyPI Python versions Documentation

Sparkless brings familiar DataFrame and SQL APIs to local development and testing. Its Rust/Polars engine runs your transformations without Java or a Spark cluster.

Install

Requires Python 3.11+:

python -m pip install "sparkless>=4,<5"

Quick start

from sparkless.sql import SparkSession, functions as F

spark = SparkSession.builder.app_name("example").get_or_create()
df = spark.createDataFrame([("Alice", 25), ("Bob", 30)], ["name", "age"])
df.filter(F.col("age") >= 30).select("name").show()
spark.stop()

Use common transformations, joins, aggregations, windows, SQL, and file I/O. For tests, install sparkless[testing] to get the backend-independent sparkless_testing helpers. They let one pytest suite run with Sparkless or real PySpark.

Sparkless runs on one machine and implements a subset of PySpark behavior. See Before you adopt for compatibility and limitations.

Supported platforms

Prebuilt wheels cover Linux, macOS, and Windows. See the platform guide and PyPI files for your release.

Learn more

Guide What you'll find
Getting started Installation, extras, and examples
Python API DataFrames, functions, SQL, and Arrow export
Testing Run the same tests with Sparkless and PySpark
Spark Connect Native server setup and the 4.18 supported subset

Full documentation · Source and contributing · Issues · MIT license.

Metadata

Release files for sparkless 4.19.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sparkless 4.19.0
File Size Uploaded Provenance Actions
sparkless-4.19.0.tar.gz 43.1 kB With provenance Details for sparkless-4.19.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sparkless 4.19.0
File Interpreter ABI Platform Actions
sparkless-4.19.0-py3-none-any.whl Python 3 none any Details for sparkless-4.19.0-py3-none-any.whl

Total release size: 92.9 kB

Release files / sparkless-4.19.0.tar.gz

Download URL sparkless-4.19.0.tar.gz
Size 43.1 kB
Tags Source
SHA-256 checksum
How to use checksums
783220028e49575fc0559600acac259acf062a63a0236e948359e741f2e92664
BLAKE2b-256 checksum
How to use checksums
709be56d720430b728a0d85714802341c11d00708fa455ebf4cd20a915c266be
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.

Transparency log

Release files / sparkless-4.19.0-py3-none-any.whl

Download URL sparkless-4.19.0-py3-none-any.whl
Size 49.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
41d42336502b482fc42bad97c99bd1a70cf35a3db97c42e9dc6c2cc57e3a67fd
BLAKE2b-256 checksum
How to use checksums
68157d1e52adde3229b4cdc944ec0a565e314a85dc8ef808ae578840fd0324ca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.

Transparency log

Security for sparkless 4.19.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for sparkless-4.19.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.19.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page