Skip to main content

A helper library for Hops that facilitates development by hiding the complexity of discovering services and setting up security.

Project description

hops-utily-py is a helper library for Hops that facilitates development by hiding the complexity of discovering services and setting up security for python programs interacting with Hops services.

It provides an API to scale out TensorFlow training on a Hops Hadoop cluster. Providing first-class support to Hyper-parameter search, Distributed TensorFlow and Horovod.

Moreover it provides an easy-to-use API for defining TLS-secured Kafka producers and consumers on the Hops platform.

Quick Start

To Install:

>>> pip install hops

Sample usage:

>>> from hops import experiment
>>> from hops import hdfs
>>> notebook = hdfs.project_path() + "Jupyter/Experiment/..." #path to your notebook
>>> experiment.launch(minimal_mnist, #minimal_mnist is your training function
>>>                   name='mnist estimator',
>>>                   description='A minimal mnist example with two hidden layers',
>>>                   versioned_resources=[notebook]

To build docs:

>>> cd docs
>>> sphinx-apidoc -f -o source/ ../hops ../hops/distribute/
>>> make html

Documentation

Tutorials and general documentation is available here: hops-examples

Example notebooks for doing deep learning and big data processing on Hops is available here: hops-io

API documentation is available here: API-docs

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
hops-2.7.4.tar.gz (27.9 kB) Copy SHA256 hash SHA256 Source None Oct 11, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page