sparksql-magic
Spark SQL magic command for Jupyter notebooks.
Prerequisites
- Python >= 3.6
- PySpark >= 2.3.0
- IPython >= 7.4.0
Install
pip install sparksql-magic
Usage
Load
%load_ext sparksql_magic
Config
%config SparkSql.limit=<INT>
| Option | Default | Description |
|---|---|---|
SparkSql.limit |
20 | The maximum number of rows to display |
Parameter
%%sparksql [-c|--cache] [-e|--eager] [-v|--view VIEW] [-l|--limit LIMIT] [variable]
<QUERY>
| Parameter | Description |
|---|---|
-c --cache |
Cache dataframe |
-e --eager |
Cache dataframe with eager load |
-v VIEW --view VIEW |
Create or replace temporary view |
-l LIMIT --limit LIMIT |
The maximum number of rows to display (Default: SparkSql.limit) |
variable |
Capture dataframe in a local variable |
1 maintainer
Release files for sparksql-magic 0.0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sparksql-magic-0.0.3.tar.gz | 3.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| sparksql_magic-0.0.3-py36-none-any.whl | Python 3.6 | none | any | Details |
Total release size: 7.3 kB
Release files / sparksql-magic-0.0.3.tar.gz
| Download URL | sparksql-magic-0.0.3.tar.gz |
|---|---|
| Size | 3.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8827b0094fa6019d21da5894e483f892889c3918fc356b562dedbc8b360d7fb8
|
|
BLAKE2b-256 checksum How to use checksums |
137a50cb4b79e86bafb530c392efd1d439dcf0b662679f05e7fef2f649fa70c0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.6.4
|
Release files / sparksql_magic-0.0.3-py36-none-any.whl
| Download URL | sparksql_magic-0.0.3-py36-none-any.whl |
|---|---|
| Size | 4.3 kB |
| Tags | Python 3.6 |
|
SHA-256 checksum How to use checksums |
ec0cd21bf0964444b8c573245d91d588529b230748b7d7b7c541a35f4703da77
|
|
BLAKE2b-256 checksum How to use checksums |
edced6dd2f262abce2cfc9112c41d610475f6b08662d8f153ead921fe6ed7f9e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.6.4
|
