rivetsql-pyspark 0.1.16

PySpark plugin for Rivet SQL

rivetsql-pyspark

PySpark engine plugin for Rivet.

---

Distributed processing on Apache Spark. Supports both local Spark sessions and remote Spark Connect clusters for large-scale workloads.

---

⚡ Install

pip install rivetsql[pyspark]

---

🔧 Configuration

# profiles.yaml
default:
  engines:
    - name: spark
      type: pyspark
      options:
        master: "local[*]"
        app_name: rivet
        config:
          spark.sql.adaptive.enabled: "true"
          spark.sql.shuffle.partitions: "200"
      catalogs: [local]

---

✨ Capabilities

Feature	Supported
Compute engine	✅
Read	✅
Local mode	✅
Spark Connect	✅

---

📚 Documentation

Full docs at rivetsql.github.io/rivet/plugins/pyspark

---

Part of the Rivet framework.