Skip to main content

Tools for using and

Project description Documentation Status

Tools for using and



pip install rhg_compute_tools


Kubernetes tools

  • easily spin up a preconfigured cluster with get_cluster(), or flavors with get_micro_cluster(), get_standard_cluster(), get_big_cluster(), or get_giant_cluster().
>>> import rhg_compute_tools.kubernetes as rhgk
>>> cluster, client = rhgk.get_cluster()

Google cloud storage utilities

  • Utilities for managing google cloud storage directories in parallel from the command line or via a python API
>>> import rhg_compute_tools.gcs as gcs
>>> gcs.sync_gcs('my_data_dir', 'gs://my-bucket/my_data_dir')



  • Add remote scheduler deployment (part of dask_kubernetes 0.10)
  • Remove extraneous GCSFUSE_TOKENS env var no longer used in new worker images
  • Set library thread limits based on how many cpus are available for a single dask thread
  • Change formatting of the extra env_items passed to get_cluster to be a list rather than a list of dict-like name/value pairs


  • Add CLI tools . See rctools gcs repdirstruc --help to start
  • Add new function rhg_compute_tools.gcs.replicate_directory_structure_on_gcs to copy directory trees into GCS. Users can authenticate with cred_file or with default google credentials
  • Fixes to docstrings and metadata
  • Add new function rhg_compute_tools.gcs.rm to remove files/directories on GCS using the API
  • Store one additional environment variable when passing cred_path to rhg_compute_tools.kubernetes.get_cluster so that the API will be authenticated in addition to gsutil


  • Deployment fixes


  • Design tools: use RHG & CIL colors & styles
  • Plotting helpers: generate cmaps with consistent colors & norms, and apply a colorbar to geopandas plots with nonlinear norms
  • Autoscaling fix for kubecluster: switch to dask_kubernetes.KubeCluster to allow use of recent bug fixes


  • Add rhg_compute_tools.gcs.cp_gcs and rhg_compute_tools.gcs.sync_gcs utilities


  • need to figure out how to use this rever thing


  • Bug fix again in rhg_compute_tools.kubernetes.get_worker


  • Bug fix in rhg_compute_tools.kubernetes.get_worker


  • Add xarray from delayed methods in rhg_compute_tools.xarray
  • rhg_compute_tools.gcs.cp_to_gcs now calls gsutil in a subprocess instead of operations. This dramatically improves performance when transferring large numbers of small files
  • Additional cluster creation helpers


  • New google compute helpers (see rhg_compute_tools.gcs.cp_to_gcs, rhg_compute_tools.gcs.get_bucket)
  • New cluster creation helper (see rhg_compute_tools.kubernetes.get_worker)
  • Dask helpers (see rhg_compute_tools.utils submodule)


  • First release on PyPI.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for rhg-compute-tools, version 0.2.2
Filename, size File type Python version Upload date Hashes
Filename, size rhg_compute_tools-0.2.2-py2.py3-none-any.whl (18.9 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size rhg_compute_tools-0.2.2.tar.gz (27.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page