sparksql-jupyter
Spark SQL magic command for Jupyter notebooks.
Prerequisites
- Python >= 3.6
- PySpark >= 2.3.0
- IPython >= 7.4.0
- Jinja2 >= 3.0.0
Install
pip install sparksql-jupyter
Usage
Load
%load_ext sparksql_jupyter
Config
%config SparkSql.limit=<INT>
| Option | Default | Description |
|---|---|---|
SparkSql.limit |
20 | The maximum number of rows to display |
Parameter
%%sparksql [-c|--cache] [-e|--eager] [-v|--view VIEW] [-l|--limit LIMIT] [-j|--jinja] [variable]
<QUERY>
| Parameter | Description |
|---|---|
-c --cache |
Cache dataframe |
-e --eager |
Cache dataframe with eager load |
-v VIEW --view VIEW |
Create or replace temporary view |
-l LIMIT --limit LIMIT |
The maximum number of rows to display (Default: SparkSql.limit) |
variable |
Capture dataframe in a local variable |
-j --jinja |
Capture dataframe in a local variable in jinja2 template |
Release Note
- 2024/05/09
- stringify timestamp with timezone columns before seriealization
1 maintainer
Release files for sparksql-jupyter 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sparksql-jupyter-0.0.5.tar.gz | 4.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| sparksql_jupyter-0.0.5-py36-none-any.whl | Python 3.6 | none | any | Details |
Total release size: 8.5 kB
Release files / sparksql-jupyter-0.0.5.tar.gz
| Download URL | sparksql-jupyter-0.0.5.tar.gz |
|---|---|
| Size | 4.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a923d23cf9e7e823713fd81bc30fc0d5cd5274823ddd945677e10381fbf2dcf7
|
|
BLAKE2b-256 checksum How to use checksums |
02662ee17cbdc2534c01ec0c11897b91ac29a9ac03a98b5bb8e75805d486c6bf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.10
|
Release files / sparksql_jupyter-0.0.5-py36-none-any.whl
| Download URL | sparksql_jupyter-0.0.5-py36-none-any.whl |
|---|---|
| Size | 4.5 kB |
| Tags | Python 3.6 |
|
SHA-256 checksum How to use checksums |
24e88d3ac7ead0f73535aa11c3cd82e532d6fc1455a920fb03b7fd35f34679a0
|
|
BLAKE2b-256 checksum How to use checksums |
91ffdab5f2c08c6937f39f76e6f0295b4a80bbce0ab205f99fda1d87839ebb8f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.10
|
