sparkonda 0.2.0

Minimalistic utility library to manage conda environments for pyspark jobs on yarn clusters

Minimalistic utility library to manage conda environments for PySpark jobs on yarn clusters.

Features

• Manage conda environments on PySpark executors to use specific packages on the remote workers without involving admins to install needed software on hadoop cluster.

Docs

http://sparkonda.readthedocs.org/en/latest/index.html

History

0.1.0 (2015-01-11)

Initial version to manage the conda environments on pyspark cluster workers without involving your cluster admins too heavily.