Skip to main content

Spark SQL magic command for Jupyter notebooks

Project description


Spark SQL magic command for Jupyter notebooks.



  • Python >= 3.6
  • PySpark >= 2.3.0
  • IPython >= 7.4.0


pip install sparksql-magic



%load_ext sparksql_magic


%config SparkSql.limit=<INT>
Option Default Description
SparkSql.limit 20 The maximum number of rows to display


%%sparksql [-c|--cache] [-e|--eager] [-v|--view VIEW] [-l|--limit LIMIT] [variable]
Parameter Description
-c --cache Cache dataframe
-e --eager Cache dataframe with eager load
-v VIEW --view VIEW Create or replace temporary view
-l LIMIT --limit LIMIT The maximum number of rows to display (Default: SparkSql.limit)
variable Capture dataframe in a local variable

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for sparksql-magic, version 0.0.3
Filename, size File type Python version Upload date Hashes
Filename, size sparksql_magic-0.0.3-py36-none-any.whl (4.3 kB) File type Wheel Python version py36 Upload date Hashes View
Filename, size sparksql-magic-0.0.3.tar.gz (3.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page