This is an extremely early proof of concept for using the metakernel in combination with py4j to make a simpler kernel for scala.
Installation
On python 3.5+
pip install .
Installing the jupyter kernel
To install the jupyter kernel install it using
python -m spylon_kernel install
Using the kernel
The scala spark metakernl prodived a scala kernel by default. At the first scala cell that is run a spark session will be constructed so that a user can interact with the interpreter.
Customizing the spark context
The launch arguments can be customized using the %%init_spark magic as follows
%%init_spark
launcher.jars = ["file://some/jar.jar"]
launcher.master = "local[4]"
launcher.conf.spark.executor.cores = 8
Other languages
Since this makes use of metakernel you can evaluate normal python code using the %%python magic. In addition once the spark context has been created the spark variable will be added to your python ernvironment.
%%python
df = spark.read.json("examples/src/main/resources/people.json")
To get completions for python, make sure that you have installed jedi
Using as a magic
Spylon-kernel can be used as a magic in an existing ipykernel. This is the recommended solution when you want to write relatively small blocks of scala.
from spylon_kernel import register_ipython_magics
register_ipython_magics()
%%scala
val x = 8
x
Using as a library
If you just want to send a string of scala code to the interpreter and evaluate it you can do that too.
from spylon_kernel import get_scala_interpreter
interp = get_scala_interpreter()
# Evaluate the result of a scala code block.
interp.interpret("""
val x = 8
x
""")
interp.last_result()
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file spylon-kernel-0.1.1.tar.gz.
File metadata
- Download URL: spylon-kernel-0.1.1.tar.gz
- Upload date:
- Size: 30.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4647473e02341f2d236eed541733dfbd07cb492526a1755e3443c677bae6d0c9
|
|
| MD5 |
a691a125327e7465e790f58914dffd1c
|
|
| BLAKE2b-256 |
4aaef68fde5cb20538da3e169b3fe49d7c8db711df899bb5ed8458165d7a535c
|