Spark SQL magic command for Jupyter notebooks
Project description
sparksql-magic
Spark SQL magic command for Jupyter notebooks.
Prerequisites
- Python >= 3.6
- PySpark >= 2.3.0
- IPython >= 7.4.0
Install
pip install sparksql-magic
Usage
Load
%load_ext sparksql_magic
Config
%config SparkSql.max_num_rows=<INT>
Option | Default | Description |
---|---|---|
SparkSql.max_num_rows |
20 | The maximum number of rows to display |
Parameter
%%sparksql [-c] [-v VIEW] [variable]
<QUERY>
Parameter | Description |
---|---|
-c --cache |
Cache dataframe |
-v VIEW --view VIEW |
Create or replace temporary view |
variable |
Capture dataframe in a local variable |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
sparksql-magic-0.0.1.tar.gz
(2.9 kB
view hashes)
Built Distribution
Close
Hashes for sparksql_magic-0.0.1-py36-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 03c3ac4c2af16c2e87cb0f0313c4db3913bd2b0482208ac207e0f90cc7be56fa |
|
MD5 | 066be40ef0a81ade6f0b5d7f1a6ba22a |
|
BLAKE2b-256 | 4cd4e0665f8455aafa538a4d03adb3dab8a5455344d2ab9ec6791c87a25aae4a |