Skip to main content

Common Metadata eXchange framework (CMX) and Collective Mind automation framework (CM)

Project description

PyPI version Python Version License Downloads

arXiv CM test CM script automation features test

Common Metadata eXchange (CMX)

The CMX framework facilitates the decomposition of complex software systems and benchmarks such as MLPerf into portable, reusable, and interconnected automation recipes for MLOps and DevOps. These recipes are developed and continuously improved by the community.

Starting in 2025, CMX V4.0.0 serves as drop-in, backward-compatible replacement for the earlier Collective Mind framework (CM) and other MLCommons automation prototypes, while providing a simpler and more robust interface.

CMX is a lightweight, Python-based toolset that provides a unified command-line interface (CLI), a Python API, and minimal dependencies. It is designed to help researchers and engineers automate repetitive, time-consuming tasks such as building, running, benchmarking, and optimizing AI, machine learning, and other applications across diverse and constantly evolving models, data, software, and hardware.

CMX is continuously enhanced through public and private Git repositories, providing automation recipes and artifacts that are seamlessly accessible via its unified interface.

CMX is a part of Collective Knowledge (CK) - an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse and ever-evolving systems using the MLPerf benchmarking methodology.

Legacy Collective Mind automation framework (CM)

MLOps and MLPerf automations

We have developed a collection of portable, extensible and technology-agnostic automation recipes with a common CLI and Python API (CM scripts) to unify and automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications on diverse platforms with any software and hardware.

The two key automations are script and cache: see online catalog at CK playground, online MLCommons catalog.

CM scripts extend the concept of cmake with simple Python automations, native scripts and JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are continuously extended by the community and MLCommons members to run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux and any other operating system, in a cloud or inside automatically generated containers while keeping backward compatibility.

See the online documentation at MLCommons to run MLPerf inference benchmarks across diverse systems using CMX. Just install pip install cmind and substitute the following commands and flags:

  • cm -> cmx
  • mlc -> cmx run mlc
  • mlcr -> cmxr
  • -v -> --v

MLPerf results visualization

CM4MLPerf-results powered by CM - a simplified and unified representation of the past MLPerf results in the CM format for further visualization and analysis using CK graphs.

Collective Knowledge Playground

Collective Knowledge Playground - a unified and open-source platform designed to index all CM scripts similar to PYPI, assist users in preparing CM commands to:

These initiatives aim to help academia and industry collaboratively enhance the efficiency and cost-effectiveness of AI systems.

Artifact Evaluation and Reproducibility Initiatives

Artifact Evaluation automation - a community-driven initiative leveraging the Collective Mind framework to automate artifact evaluation and support reproducibility efforts at ML and systems conferences.

License

Apache 2.0

Copyright

Copyright (c) 2021-2025 MLCommons

Grigori Fursin, the cTuning foundation and OctoML donated this project to MLCommons to benefit everyone.

Copyright (c) 2014-2021 cTuning foundation

Author

Maintainers

Long-term vision

To learn more about the motivation behind this project, please explore the following presentations:

  • "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ ArXiv ]
  • ACM REP'23 keynote about the MLCommons CM automation framework: [ slides ]
  • ACM TechTalk'21 about Collective Knowledge project: [ YouTube ] [ slides ]
  • Journal of Royal Society'20: [ paper ]

Documentation

TBD

Acknowledgments

This open-source project was created by Grigori Fursin and sponsored by cTuning.org, OctoAI and HiPEAC. Grigori donated this project to MLCommons to modularize and automate MLPerf benchmarks, benefit the community, and foster its development as a collaborative, community-driven effort.

We thank MLCommons, FlexAI and cTuning for supporting this project, as well as our dedicated volunteers and collaborators for their feedback and contributions!

If you found the CM automations helpful, kindly reference this article: [ ArXiv ], [ BibTex ].

Project details


Release history Release notifications | RSS feed

This version

4.0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cmind-4.0.2.tar.gz (80.4 kB view details)

Uploaded Source

Built Distribution

cmind-4.0.2-py3-none-any.whl (85.5 kB view details)

Uploaded Python 3

File details

Details for the file cmind-4.0.2.tar.gz.

File metadata

  • Download URL: cmind-4.0.2.tar.gz
  • Upload date:
  • Size: 80.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.10.11

File hashes

Hashes for cmind-4.0.2.tar.gz
Algorithm Hash digest
SHA256 6820ddafb1aff669844f57f507df32a65f7614d1a0d3da6092823b961622b4ae
MD5 a423e60a0f4b03df3ede8e4d6272295c
BLAKE2b-256 c754858d16be0d07b730027d5ce6adc954710500eb4ab72ff6d4796afd2da243

See more details on using hashes here.

File details

Details for the file cmind-4.0.2-py3-none-any.whl.

File metadata

  • Download URL: cmind-4.0.2-py3-none-any.whl
  • Upload date:
  • Size: 85.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.10.11

File hashes

Hashes for cmind-4.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 324d45284c70b0c504c4ec6d6f4ac57925f22a0621d3784ae9434f7e53ef6656
MD5 06c1ce8acd92ce730a8cb169bf51b8c5
BLAKE2b-256 dd0ed9c3d75bd314ba03e3e81ae6cab4bc9e7a5f7e0a1111c27185678633ae08

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page