Skip to main content

Spark SQL magic command for Jupyter notebooks

Project description


Spark SQL magic command for Jupyter notebooks.



  • Python >= 3.6
  • PySpark >= 2.3.0
  • IPython >= 7.4.0


pip install sparksql-magic



%load_ext sparksql_magic


%config SparkSql.max_num_rows=<INT>
Option Default Description
SparkSql.max_num_rows 20 The maximum number of rows to display


%%sparksql [-c] [-v VIEW] [variable]
Parameter Description
-c --cache Cache dataframe
-e --eager Cache dataframe with eager load
-v VIEW --view VIEW Create or replace temporary view
variable Capture dataframe in a local variable

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for sparksql-magic, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size sparksql_magic-0.0.2-py36-none-any.whl (4.2 kB) File type Wheel Python version py36 Upload date Hashes View hashes
Filename, size sparksql-magic-0.0.2.tar.gz (3.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page