Skip to main content

Misc helpers for Spark Python Notebook

Project description

# pixiedust

Pixiedust is an open source Python helper library that works as an add-on to Jupyter notebooks to improve the user experience of working with data. It also provides extra capabilities that fill a gap when the notebook is hosted on the cloud and the user has no access to configuration files.

Its current capabilities include:

- **packageManager** lets you install spark packages inside a python notebook. This is something that you can't do today on hosted Jupyter notebooks, which prevents developers from using a large number of spark package add-ons.
The following code installs the GraphFrames spark package into your IPython notebook
```
from pixiedust.packageManager import PackageManager
pkg=PackageManager()
pkg.installPackage("graphframes:graphframes:0")
```

- **visualizations.** One single API called `display` lets you visualize your spark object in different ways: table, charts, maps, etc.... This module is designed to be extensible, providing an API that lets anyone easily contribute a new visualization plugin.

This sample visualization plugin uses d3 to show the different flight routes for each airport:

![graph map](http://developer.ibm.com/clouddataservices/wp-content/uploads/sites/47/2016/07/pd_graphmap.png)

- **service integration.** Stash data into a variety of back-end data sources, like Cloudant, dashDB, GraphDB, etc...

![save as options](http://developer.ibm.com/clouddataservices/wp-content/uploads/sites/47/2016/07/pd_download.png)

_**Note:** Pixiedust currently works only on a Python Notebook hosted on IBM Bluemix._

Project details


Release history Release notifications | RSS feed

This version

0.26

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pixiedust-0.26.tar.gz (42.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page