Skip to main content

MLCFlow: Simplifying MLPerf Automations

License Downloads

MLC core actions test MLC script automation features test MLPerf inference resnet50

MLCFlow is a versatile CLI and Python interface developed by MLCommons in collaboration with a dedicated team of volunteers (see Contributors). It serves as a streamlined replacement for the CMind tool, designed to drive the automation workflows of MLPerf benchmarks more efficiently. You can use this tool for any of your workflow automation tasks.

The concept behind CMind originated from Grigori Fursin, while the MLPerf Automations project was created by Grigori Fursin and Arjun Suresh, whose collective contributions laid the foundation for modernizing MLPerf benchmarking tools. MLCFlow is a complete replacement for the CMind package with an entirely new implementation led by Arjun Suresh and Anandhu Sooraj, with support from the MLCFlow community. The script automation used in MLCFlow is inherited from the CM project but is now modified to support MLC and provides a simpler interface to the developers. Please see the documentation for more details.

On February 9, 2025, MLCFlow released its first stable version, 1.0.0.

Key Features

Building upon the core idea of CMind—wrapping native scripts with Python wrappers and YAML metadata—MLCFlow focuses exclusively on key automation components: Scripts, along with its complementary modules: Cache, Docker, and Experiments. This targeted design simplifies both implementation and interface, enabling a more user-friendly experience.

  • Typo detection — Mistyped actions or targets produce a "Did you mean …?" hint before the error message, so mistakes are quick to correct:
    $ mlc rune script
    Did you mean one of: 'prune', 'run'?
    mlc: error: argument command: invalid choice: 'rune' …
    
    See docs/typo_detection.md for details.

Status

MLCFlow is now fully equipped for workflow development, with complete support for all previously used CM scripts in MLPerf inference automation. If you're interested in discussions, join the MLCommons Benchmark Infra Discord channel, and check out the latest progress in Issues.


MLC CLI Overview

The MLC Command-Line Interface (CLI) enables users to perform actions on specified targets using a simple syntax:

mlc <action> <target> [options]

Key Components:

  • <action>: The operation to be performed.
  • <target>: The object on which the action is executed.
  • [options]: Additional parameters passed to the action.

Supported Targets and Actions

1. Repo

  • Actions related to repositories, such as cloning or updating.

2. Script

  • Manage or execute automation scripts.

3. Cache

  • Handle cached data, including cleanup or inspection.

Each target has its own set of specific actions to tailor automation workflows as specified below.

Target Action
script run, find/search, rm, mv, cp, add, test, docker-run, show, experiment, doc
cache find/search, rm, show, list, prune, mark-tmp
repo pull, search, rm, list, find/search , add

MLC started with a compatibility layer where by it supported MLCommons CM automations - Script, Cache and Experiment. Now, MLCFLow has just the Script Automation which is an extension of the Script Automation from CM but with a cleaner integration of Cache Automation and Docker and Test extensions. The old CM scripts are now updated with the latest MLCFlow scripts in the MLPerf Automations repository.

Architectural Diagram

classDiagram
    class Action {
        +access(options)
        +find_target_folder(target)
        +load_repos_and_meta()
        +load_repos()
        +conflicting_repo(repo_meta)
        +register_repo(repo_meta)
        +unregister_repo(repo_path)
        +add(i)
        +rm(i)
        +save_new_meta(i, item_id, item_name, target_name, item_path, repo)
        +update(i)
        +is_uid(name)
        +cp(run_args)
        +copy_item(source_path, destination_path)
        +search(i)
    }
    class RepoAction {
        +find(run_args)
        +github_url_to_user_repo_format(url)
        +pull_repo(repo_url, branch, checkout)
        +pull(run_args)
        +list(run_args)
        +rm(run_args)
    }
    class ScriptAction {
        +search(i)
        +rm(i)
        +dynamic_import_module(script_path)
        +call_script_module_function(function_name, run_args)
        +docker(run_args)
        +run(run_args)
        +test(run_args)
        +list(args)
    }
    class CacheAction {
        +search(i)
        +find(i)
        +rm(i)
        +show(run_args)
        +list(args)
    }
    class ExperimentAction {
        +show(args)
        +list(args)
    }
    class Index {
        +add(meta, folder_type, path, repo)
        +get_index(folder_type, uid)
        +update(meta, folder_type, path, repo)
        +rm(meta, folder_type, path)
        +build_index()
    }
    class Item {
        +meta
        +path
        +repo
        +_load_meta()
    }
    class Repo {
        +path
        +meta
        +_load_meta()
    }
    class Automation {
        +action_object
        +automation_type
        +meta
        +path
        +_load_meta()
        +search(i)
    }

    Action <|-- RepoAction
    Action <|-- ScriptAction
    Action <|-- CacheAction
    RepoAction o-- Repo
    ScriptAction o-- Automation
    CacheAction o-- Index
    ExperimentAction o-- Index
    CfgAction o-- Index
    Index o-- Repo
    Index o-- Item
    Item o-- Repo
    Automation o-- Action

Metadata

Release files for mlcflow 1.3.9

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mlcflow 1.3.9
File Size Uploaded Provenance Actions
mlcflow-1.3.9.tar.gz 239.4 kB With provenance Details for mlcflow-1.3.9.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for mlcflow 1.3.9
File Interpreter ABI Platform Actions
mlcflow-1.3.9-py3-none-any.whl Python 3 none any Details for mlcflow-1.3.9-py3-none-any.whl

Total release size: 453.1 kB

Release files / mlcflow-1.3.9.tar.gz

Download URL mlcflow-1.3.9.tar.gz
Size 239.4 kB
Tags Source
SHA-256 checksum
How to use checksums
16d916583490a35d018b3cefd9d89d3b4af5876af8b7ea2d218d45a7420dbdd5
BLAKE2b-256 checksum
How to use checksums
d1705739b49bd698bb42c73918c812e76d9b782766b840e6ba0a17069696a78e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

Release files / mlcflow-1.3.9-py3-none-any.whl

Download URL mlcflow-1.3.9-py3-none-any.whl
Size 213.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
58108e07d4a0e48e9fd5c14529d151d0e6004e4d4d1dc5f08f4372917af1a911
BLAKE2b-256 checksum
How to use checksums
df6599d3cb1c625a8183133bfd810bc9b464990035a293bbe4ed4512c002f6d4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

Security for mlcflow 1.3.9

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for mlcflow-1.3.9.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 7, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.3.9 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page