Skip to main content

Sparkless

PySpark-style DataFrames in Python. No JVM required.

PyPI Python versions Documentation

Sparkless brings familiar DataFrame and SQL APIs to local development and testing. Its Rust/Polars engine runs your transformations without Java or a Spark cluster.

Install

Requires Python 3.11+:

python -m pip install "sparkless>=4,<5"

Quick start

from sparkless.sql import SparkSession, functions as F

spark = SparkSession.builder.app_name("example").get_or_create()
df = spark.createDataFrame([("Alice", 25), ("Bob", 30)], ["name", "age"])
df.filter(F.col("age") >= 30).select("name").show()
spark.stop()

Use common transformations, joins, aggregations, windows, SQL, and file I/O. For tests, sparkless.testing lets you run one pytest suite with Sparkless or real PySpark.

Sparkless runs on one machine and implements a subset of PySpark behavior. See Before you adopt for compatibility and limitations.

Supported platforms

Prebuilt wheels cover Linux, macOS, and Windows. See the platform guide and PyPI files for your release.

Learn more

Guide What you'll find
Getting started Installation, extras, and examples
Python API DataFrames, functions, SQL, and Arrow export
Testing Run the same tests with Sparkless and PySpark
Spark Connect Native server setup and the 4.18 supported subset

Full documentation · Source and contributing · Issues · MIT license.

Metadata

Release files for sparkless 4.18.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for sparkless 4.18.0
File Actions
sparkless-4.18.0-cp311-abi3-musllinux_1_2_x86_64.whl CPython 3.11 abi3 Linux musl 1.2+ x86-64 Details for sparkless-4.18.0-cp311-abi3-musllinux_1_2_x86_64.whl
sparkless-4.18.0-cp311-abi3-musllinux_1_2_aarch64.whl CPython 3.11 abi3 Linux musl 1.2+ ARM64 Details for sparkless-4.18.0-cp311-abi3-musllinux_1_2_aarch64.whl
sparkless-4.18.0-cp311-abi3-manylinux_2_28_aarch64.whl CPython 3.11 abi3 Linux glibc 2.28+ ARM64 Details for sparkless-4.18.0-cp311-abi3-manylinux_2_28_aarch64.whl
sparkless-4.18.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.11 abi3 Linux glibc 2.17+ x86-64 Details for sparkless-4.18.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
sparkless-4.18.0-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for sparkless-4.18.0-cp311-abi3-macosx_11_0_arm64.whl
sparkless-4.18.0-cp311-abi3-macosx_10_12_x86_64.whl CPython 3.11 abi3 macOS 10.12+ x86-64 Details for sparkless-4.18.0-cp311-abi3-macosx_10_12_x86_64.whl

Total release size: 253.3 MB

Release files / sparkless-4.18.0-cp311-abi3-musllinux_1_2_x86_64.whl

Download URL sparkless-4.18.0-cp311-abi3-musllinux_1_2_x86_64.whl
Size 42.1 MB
Tags CPython 3.11 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
d41a56ca9fbd1b131ee7da0a6a4ab2a35ab000ff8496d49e453715a249edb20f
BLAKE2b-256 checksum
How to use checksums
b298135b2ddad1b128c0826e20f4663907eda87c82f0dd25294bd00dec09198c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

Release files / sparkless-4.18.0-cp311-abi3-musllinux_1_2_aarch64.whl

Download URL sparkless-4.18.0-cp311-abi3-musllinux_1_2_aarch64.whl
Size 39.2 MB
Tags CPython 3.11 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
7833d8ecb07a3ca53c6c059d5addbaa5dcdef2beeb9111d612c4093c61533434
BLAKE2b-256 checksum
How to use checksums
b91ad3b677d2aae1bf659dfce026d6d6dde896872455ca0f708730289eab21ae
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

Release files / sparkless-4.18.0-cp311-abi3-manylinux_2_28_aarch64.whl

Download URL sparkless-4.18.0-cp311-abi3-manylinux_2_28_aarch64.whl
Size 39.2 MB
Tags CPython 3.11 Linux glibc 2.28+ ARM64 abi3
SHA-256 checksum
How to use checksums
74979c6fd9d8d5e7179183a823efd5a706e8aba3d4f55cdeceafe30e02f73187
BLAKE2b-256 checksum
How to use checksums
43af26d00c2dc2be5ccf9c837a590356bdd593777bd5f12d0311679ba9c35fa0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

Release files / sparkless-4.18.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL sparkless-4.18.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 42.1 MB
Tags CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
adbdf5cd40caf0a6e2edca96972ecf6af3f3bf79c3d42c32b9cdb9dae5de7d1a
BLAKE2b-256 checksum
How to use checksums
2b0ba099381ab87e891c9991e180726f2082ee227d195c9d0f11912729c9b1de
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

Release files / sparkless-4.18.0-cp311-abi3-macosx_11_0_arm64.whl

Download URL sparkless-4.18.0-cp311-abi3-macosx_11_0_arm64.whl
Size 44.1 MB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
268cfe4efc6a8d4f664a9fd807f2028dbca9518a75ba2597d06576e1abf542c4
BLAKE2b-256 checksum
How to use checksums
725d76cd684e0e42cec1f8ceb4a1d5dfc95a74b8e3b390d07000de45bc44f213
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

Release files / sparkless-4.18.0-cp311-abi3-macosx_10_12_x86_64.whl

Download URL sparkless-4.18.0-cp311-abi3-macosx_10_12_x86_64.whl
Size 46.7 MB
Tags CPython 3.11 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
9e68e521d547d59653209af0fb375379aa4fdb9f1e9378d172dda314618d6fc2
BLAKE2b-256 checksum
How to use checksums
7f9e906a89f5be4fb3b89c98055f764406555fefe825d47467292ffacba2f170
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

Security for sparkless 4.18.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 6 files in this release have been uploaded with provenance metadata.

Select from 6 files

Attestation for sparkless-4.18.0-cp311-abi3-musllinux_1_2_x86_64.whl

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.18.0 This release

6 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page