This is an extremely early proof of concept for using the metakernel in combination with py4j to make a simpler kernel for scala.
Installation
On python 3.5+
pip install .
Installing the jupyter kernel
To install the jupyter kernel install it using
python -m spylon_kernel install
Using the kernel
The scala spark metakernl prodived a scala kernel by default. At the first scala cell that is run a spark session will be constructed so that a user can interact with the interpreter.
Customizing the spark context
The launch arguments can be customized using the %%init_spark magic as follows
%%init_spark
launcher.jars = ["file://some/jar.jar"]
launcher.master = "local[4]"
launcher.conf.spark.executor.cores = 8
Other languages
Since this makes use of metakernel you can evaluate normal python code using the %%python magic. In addition once the spark context has been created the spark variable will be added to your python ernvironment.
%%python
df = spark.read.json("examples/src/main/resources/people.json")
To get completions for python, make sure that you have installed jedi
Using as a magic
Spylon-kernel can be used as a magic in an existing ipykernel. This is the recommended solution when you want to write relatively small blocks of scala.
from spylon_kernel import register_ipython_magics
register_ipython_magics()
%%scala
val x = 8
x
Using as a library
If you just want to send a string of scala code to the interpreter and evaluate it you can do that too.
from spylon_kernel import get_scala_interpreter
interp = get_scala_interpreter()
# Evaluate the result of a scala code block.
interp.interpret("""
val x = 8
x
""")
interp.last_result()
Metadata
Release files for spylon-kernel 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| spylon-kernel-0.1.1.tar.gz | 30.9 kB | Without provenance | Details for spylon-kernel-0.1.1.tar.gz |
Release files / spylon-kernel-0.1.1.tar.gz
| Download URL | spylon-kernel-0.1.1.tar.gz |
|---|---|
| Size | 30.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4647473e02341f2d236eed541733dfbd07cb492526a1755e3443c677bae6d0c9
|
|
BLAKE2b-256 checksum How to use checksums |
4aaef68fde5cb20538da3e169b3fe49d7c8db711df899bb5ed8458165d7a535c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for spylon-kernel 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.