Skip to main content

Backend implementation for running MLFlow projects on Hadoop/YARN

Project description

mlflow-yarn

Backend implementation for running MLFlow projects against a Hadoop/YARN backend).

To activate just do 'pip install mlflow-yarn' along your mlflow environment. It will register the plugin as an entrypoint with the YARN backend.

$ pip install mlflow-yarn

mlflow-yarn only supports Python ≥3.6.

Developement

Install from source

$ git clone https://github.com/criteo/cluster-pack
$ cd cluster-pack
$ pip install .

Examples

Example with pip only project

  • Dependencies are pulled from requirements.txt
$ git clone https://github.com/criteo/mlflow-yarn
$ pip install mlflow
mlflow run tests/resources/pip_project -e compute_intersection -P size=10000 --backend yarn

Example with conda project

  • Dependencies are pulled from conda.yaml
$ git clone https://github.com/criteo/mlflow-yarn
$ pip install mlflow
$ mlflow run tests/resources/conda_project -e compute_intersection -P size=10000 --backend yarn

More infos on setting up a project file.

Docker container environment is currently not supported.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

mlflow_yarn-0.0.1-py3-none-any.whl (15.0 kB view details)

Uploaded Python 3

File details

Details for the file mlflow_yarn-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: mlflow_yarn-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 15.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/39.2.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.6.8

File hashes

Hashes for mlflow_yarn-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 2784c3de5abc40f2881e3eee1345de6eee1ffb9f5bd88694e8c8aaf13c38ce78
MD5 112b331974cf9a38010df8055927fc9f
BLAKE2b-256 ab80c678a7e93fdaadc03ad92a4276ed715e98dd40bb31fd15cfe09f2e5c772a

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page