Skip to main content

Tracking and config of machine learning runs

Project description

Build Status License PyPI version fury.io Documentation Code style: black GitHub commit activity GitHub release (latest SemVer) Join MLOps Live

MLRun logo

MLRun - The Open Source MLOps Orchestration Framework

MLRun enables production pipeline design using a modular strategy, where the different parts contribute to a continuous, automated, and far simpler path from research and development to scalable production pipelines, without refactoring code, adding glue logic, or spending significant efforts on data and ML engineering.

MLRun uses Serverless Function technology: write the code once, using your preferred development environment and simple “local” semantics, and then run it as-is on different platforms and at scale. MLRun automates the build process, execution, data movement, scaling, versioning, parameterization, outputs tracking, CI/CD integration, deployment to production, monitoring, and more.

Those easily developed data or ML “functions” can then be published or loaded from a marketplace and used later to form offline or real-time production pipelines with minimal engineering efforts.

mlrun-flow


Data preparation, model development, model and application delivery, and end to end monitoring are tightly connected: they cannot be managed in silos. This is where MLRun MLOps orchestration comes in. ML, data, and DevOps/MLOps teams collaborate using the same set of tools, practices, APIs, metadata, and version control.

MLRun simplifies & accelerates the time to production.

Architecture

pipeline

MLRun is composed of the following layers:

  • Feature Store — collects, prepares, catalogs, and serves data features for development (offline) and real-time (online) usage for real-time and batch data. See also Feature store: data ingestion and Feature store: data retrieval, as well as the Feature Store tutorials.
  • ML CI/CD pipeline — automatically trains, tests, optimizes, and deploys or updates models using a snapshot of the production data (generated by the feature store) and code from the source control (Git).
  • Real-Time Serving Pipeline — Rapid deployment of scalable data and ML pipelines using real-time serverless technology, including the API handling, data preparation/enrichment, model serving, ensembles, driving and measuring actions, etc.
  • Real-Time monitoring and retraining — monitors data, models, and production components and provides a feedback loop for exploring production data, identifying drift, alerting on anomalies or data quality issues, triggering re-training jobs, measuring business impact, etc.

Get started

It's easy to start using MLRun:

  1. Install the MLRun service locally using Docker or over Kubernetes Cluster. Alternatively, you can use Iguazio's managed MLRun service
  2. Set up your client environment to work with the service.
  3. Follow the Quick Start tutorial and Additional Tutorials and Examples to learn how to use MLRun to develop and deploy machine learning applications to production.

For hands-on learning, try the MLRun Katakoda Scenarios. And you can watch the Tutorial on Youtube to see the flow in action.

MLRun documentation

Read more in the MLRun documentation, including:

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

mlrun-1.1.3rc1-py3-none-any.whl (1.0 MB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page