singlespark 0.1.3

PySpark emulator — Polars for DataFrames, DuckDB for SQL, fake HDFS on local filesystem

singlespark

Singlenode spark — no JVM required. Use the best powerful APIs for Data on top.

• Polars for DataFrame operations
• DuckDB for Spark SQL
• Local filesystem for fake HDFS

Installation

pip install singlespark

Quick start

from singlespark import SparkSession
from singlespark.sql import functions as F

spark = SparkSession.builder.appName("MyApp").getOrCreate()
df = spark.createDataFrame([("Alice", 30), ("Bob", 25)], ["name", "age"])
df.filter(F.col("age") > 26).show()
spark.stop()