Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Utilities to work with Scala/Java code with py4j

Project Description

A set of compatibility routines for making it easier to interact with Scala from Python.

Occasionally Python-focused data shops need to use JVM languages for performance reasons. Generally this necessitates throwing away whole repositories of Python code and starting over or resorting to service architectures (e.g., Apache thrift) which increase system complexity.

You don’t have to.

Using py4j and Spylon you can readily interact with Scala code for more performance critical sections of your code whilst leaving the rest unmodified.

Alternatively you can use it as a bridge to allow building wrappers for a Scala/Java codebase.

Installation

Spylon can be installed either from pip or conda-forge.

When installing from pip and the desire is to use it with Apache Spark, you should run

` pip install spylon[spark] `

Usage

The simplest way to use spylon is to use it to help with writing PySpark jobs. If you want to supply your own jars to load for usage as Spark user defined functions, you’d want to supply the jar with the udf implementation to spark via spark-submit.

For an easier interactive experience you can make use of the supplied Apache Spark launcher to make it simpler to instantiate a PySpark application from inside a python Jupyter notebook.

Extensions

Spylon is designed as an easy to extend toolkit. Since Apache Spark is a major user or Py4J, some special use cases have been implemented for that and its an example of some use cases for Spylon.

Release History

Release History

This version
History Node

0.3.0

History Node

0.2.12

History Node

0.2.11

History Node

0.2.10

History Node

0.2.9

History Node

0.2.8

History Node

0.2.7

History Node

0.2.6

History Node

0.2.5

History Node

0.2.4

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

History Node

0.1.4

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

History Node

0.1.0

History Node

0.0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
spylon-0.3.0.tar.gz (140.4 kB) Copy SHA256 Checksum SHA256 Source May 26, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting