Jupyter kernels for working with dapla services
Project description
dapla-ipython-kernels
Python module for use within Jupyter notebooks. It contains kernel extensions for integrating with Apache Spark, Google Cloud Storage and custom dapla services.
Getting Started
Install the module from pip:
# pip
pip install dapla-ipython-kernels
Now the module is ready to use with a single import:
import dapla as dp
This module is targeted to python kernels in Jupyter, but it may work in any IPython environment. It also depends on a number of custom services, e.g. the custom auth service
To test, simply create any Pandas dataframe. This can be stored in Google Cloud Storage at a specific path:
import pandas as pd
import dapla as dp
data = {
'apples': [3, 2, 0, 1],
'oranges': [0, 3, 7, 2]
}
# Create pandas DataFrame
purchases = pd.DataFrame(data, index=['June', 'Robert', 'Lily', 'David'])
# Write pandas DataFrame to parquet
dp.write_pandas(purchases, '/testfolder/python/purchases', valuation='INTERNAL', state= 'INPUT')
Conversely, parquet files can be read from a path directly into a pandas DataFrame.
import dapla as dp
# Read path into pandas dataframe
purchases = dp.read_pandas('/testfolder/python/purchases')
Other functions
Since the python module integrates with Google Cloud Storage and custom dapla services, some other functions exist as well:
import dapla as dp
# List path by prefix
dp.show('/testfolder/python')
Path | Timestamp |
---|---|
/testfolder/python/purchases | 1593120298095 |
/testfolder/python/other | 1593157667793 |
import dapla as dp
# Show file details
dp.details('/testfolder/python/purchases')
Size | Name |
---|---|
2908 | 42331105444c9ca0ce049ef6de7160.parquet |
See also the example notebook written for Jupyter.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ssb-ipython-kernels-0.1.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2eaf4ce54f8b3e84ecab01666cda26c084169ea15b547cb80d8c105f6219d0c5 |
|
MD5 | bb6e31149f746d68daa25bb0cec06f44 |
|
BLAKE2b-256 | 94d76e23c9f35ad6feaeb9932e22c99c9d5a578ab618ac7d66865d4932845487 |
Hashes for ssb_ipython_kernels-0.1.6-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4c00a1521020e4c893ebdfc836f80239afb0d2087ce0a7ef88e84f912550c954 |
|
MD5 | 04aa19a79e87d26e091f7ee66b4d5d05 |
|
BLAKE2b-256 | d1c0aabd57f03505309ffd6ef6f047b97a019a8cd9db6a8e48769b32a532c44f |