Skip to main content

Sparkless

PySpark-style DataFrames in Python. No JVM required.

PyPI Python versions Documentation

Sparkless brings familiar DataFrame and SQL APIs to local development and testing. Its Rust/Polars engine runs your transformations without Java or a Spark cluster.

Install

Requires Python 3.11+:

python -m pip install "sparkless>=4,<5"

Quick start

from sparkless.sql import SparkSession, functions as F

spark = SparkSession.builder.app_name("example").get_or_create()
df = spark.createDataFrame([("Alice", 25), ("Bob", 30)], ["name", "age"])
df.filter(F.col("age") >= 30).select("name").show()
spark.stop()

Use common transformations, joins, aggregations, windows, SQL, and file I/O. For tests, sparkless.testing lets you run one pytest suite with Sparkless or real PySpark.

Sparkless runs on one machine and implements a subset of PySpark behavior. See Before you adopt for compatibility and limitations.

Supported platforms

Prebuilt wheels cover Linux, macOS, and Windows. See the platform guide and PyPI files for your release.

Learn more

Guide What you'll find
Getting started Installation, extras, and examples
Python API DataFrames, functions, SQL, and Arrow export
Testing Run the same tests with Sparkless and PySpark
Spark Connect Native server setup and the 4.18 supported subset

Full documentation · Source and contributing · Issues · MIT license.

Metadata

Release files for sparkless 4.18.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sparkless 4.18.1
File Size Uploaded Provenance Actions
sparkless-4.18.1.tar.gz 62.4 kB With provenance Details for sparkless-4.18.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sparkless 4.18.1
File Interpreter ABI Platform Actions
sparkless-4.18.1-py3-none-any.whl Python 3 none any Details for sparkless-4.18.1-py3-none-any.whl

Total release size: 131.9 kB

Release files / sparkless-4.18.1.tar.gz

Download URL sparkless-4.18.1.tar.gz
Size 62.4 kB
Tags Source
SHA-256 checksum
How to use checksums
3dcea020b5bf0e0a9004c76547fcb153defb2d0c2e8c5c3e887765e2122c2cd2
BLAKE2b-256 checksum
How to use checksums
a017ce35439a3703f49c80708ab9965207ba32a2c522d3eda2ff3b16fb36462e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 31, 2026.

Transparency log

Release files / sparkless-4.18.1-py3-none-any.whl

Download URL sparkless-4.18.1-py3-none-any.whl
Size 69.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b1cb7331b2b554505c6f0be8a9aa9d89ccd0427b4c2e9db1569ed754924c719c
BLAKE2b-256 checksum
How to use checksums
82c870d4df000454ce167634dfbc4dd1bbd040ec39b6892d6e23e06c6705238d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 31, 2026.

Transparency log

Security for sparkless 4.18.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for sparkless-4.18.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 31, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.18.1 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page