Spark SQL magic command for Jupyter notebooks
Project description
sparksql-magic
Spark SQL magic command for Jupyter notebooks.
Prerequisites
- Python >= 3.6
- PySpark >= 2.3.0
- IPython >= 7.4.0
Install
pip install sparksql-magic
Usage
Load
%load_ext sparksql_magic
Config
%config SparkSql.limit=<INT>
Option | Default | Description |
---|---|---|
SparkSql.limit |
20 | The maximum number of rows to display |
Parameter
%%sparksql [-c|--cache] [-e|--eager] [-v|--view VIEW] [-l|--limit LIMIT] [variable]
<QUERY>
Parameter | Description |
---|---|
-c --cache |
Cache dataframe |
-e --eager |
Cache dataframe with eager load |
-v VIEW --view VIEW |
Create or replace temporary view |
-l LIMIT --limit LIMIT |
The maximum number of rows to display (Default: SparkSql.limit ) |
variable |
Capture dataframe in a local variable |
Project details
NoneRelease history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
sparksql-magic-0.0.3.tar.gz
(3.0 kB
view hashes)
Built Distribution
Close
Hashes for sparksql_magic-0.0.3-py36-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec0cd21bf0964444b8c573245d91d588529b230748b7d7b7c541a35f4703da77 |
|
MD5 | 274366fa3e7e8384f6cd14f7eee3b26e |
|
BLAKE2b-256 | edced6dd2f262abce2cfc9112c41d610475f6b08662d8f153ead921fe6ed7f9e |