Skip to main content

Python client library for Staroid cloud platform

Project description

Open data studio python library


pip install ods

Quick start


  1. Login and get an access token. And set STAROID_ACCESS_TOKEN environment variable. See here for more detail.
  2. Login and create a SKE (Star Kubernetes engine) cluster.
import ods
# 'ske' is the name of kubernetes cluster created from Alternatively, you can export 'STAROID_SKE' environment variable.


Create spark session

Create spark session with default configuration

import ods
spark = ods.spark("spark-1") # 'spark-1' is name of spark-serverless instance to create.
df = spark.createDataFrame(....)

Configurue initial number of worker nodes

import ods
spark = ods.spark("spark-1", worker_num=3).session()
df = spark.createDataFrame(....)

detal=True to automatically download & configure delta lake

import ods
spark = ods.spark("spark-delta", delta=True)"delta").load(....)

pass spark_conf dictionary for additonal configuration

import ods
spark = ods.spark(spark_conf = {
    "spark.hadoop.fs.s3a.access.key": "...",
    "spark.hadoop.fs.s3a.secret.key" : "..."

Check tests/ as well.


Coming soon 🚛

import ods
cluster = ods.dask("dask-1", worker_num=10)

from dask.distributed import Client
client = Client(cluster)


Coming soon 🚛

import ods

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ods-0.0.3.tar.gz (8.3 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page