Skip to main content

Livy-Submit enables you to send your python files to a spark cluster using Livy running on edge node

Project description

Livy-Submit

With Livy-Submit, you can easily send pyspark files to run on your spark cluster in the case you don't have access to spark-submit on an edge-node but you do have access to a Livy service on an edge-node.

You can find more information, including a sample session on https://www.guidodiepen.nl/2019/06/submitting-pyspark-jobs-to-livy-with-livy_submit/

Prerequisites

The application is build using the requests library. It also can make use of keyring to store the password for connecting to your edgenode (in order for you not to have it hardcoded anywhere else).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Livy-Submit-1.1.0.tar.gz (7.5 kB view details)

Uploaded Source

Built Distribution

Livy_Submit-1.1.0-py3-none-any.whl (8.6 kB view details)

Uploaded Python 3

File details

Details for the file Livy-Submit-1.1.0.tar.gz.

File metadata

  • Download URL: Livy-Submit-1.1.0.tar.gz
  • Upload date:
  • Size: 7.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.10

File hashes

Hashes for Livy-Submit-1.1.0.tar.gz
Algorithm Hash digest
SHA256 d63b9049766d0ad6726e8ddef2a8ff26198a4b43d2fa36056a07bd035aa179ae
MD5 5079406fcf4412e671caac03af776136
BLAKE2b-256 7d4abbc709c3d81c693d5c20397785f7ae3674386e29f30147240bb0a7d4c97d

See more details on using hashes here.

File details

Details for the file Livy_Submit-1.1.0-py3-none-any.whl.

File metadata

  • Download URL: Livy_Submit-1.1.0-py3-none-any.whl
  • Upload date:
  • Size: 8.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.10

File hashes

Hashes for Livy_Submit-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 91dd998af5fe1cac2d5700d2a9a82faa8e93003d91be10a1ecbd87c9b3310fd7
MD5 8496c9552247463661fd7cdb0fc6058e
BLAKE2b-256 b80844efaf4b5b301a49c043142b8dbf99d268ef571348c141d75fb9b271bd32

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page