cmind 1.2.1

cmind

About

Collective Mind scripting language (MLCommons CM) is a part of the MLCommons Collective Knowledge project. It is motivated by the feedback from researchers and practitioners when reproducing experiments from more than 150 research papers and validating them in the real world - there is a need for a common, human-readable and technology-agnostic interface to manage and run any software project on any platform with any software, hardware, and data.

CM is being developed by the public MLCommons task force on automation and reproducibility as a simple, intuitive, technology-agnostic, and English-like scripting language that provides a universal interface to any software project and transforms it into a database of portable and reusable CM scripts in a transparent and non-intrusive way.

CM is powered by Python, JSON and/or YAML meta descriptions, and a unified CLI. Is helps to solve the "dependency hell" for ML and AI systems while automatically generating unified README files and synthesize unified containers with a common API. It is also used to automate reproducibility initiatives and artifact evaluation at AI, ML and Systems conferences while reducing all the tedious, manual, repetitive, and ad-hoc efforts to reproduce research projects and validate them in production.

CM powers the Collective Knowledge platform (MLCommons CK playground) to aggregate reproducible experiments, connect academia and industry to organize reproducibility and optimization challenges, and help developers and users select Pareto-optimal end-to-end applications and systems based on their requirements and constraints (cost, performance, power consumption, accuracy, etc).

See a few real-world examples of using the CM scripting language:

• README to reproduce published IPOL'22 paper
• README to reproduce MLPerf RetinaNet inference benchmark at Student Cluster Competition'22
• Auto-generated READMEs to reproduce official MLPerf BERT inference benchmark v3.0 submission with a model from the Hugging Face Zoo
• Auto-generated Docker containers to run and reproduce MLPerf inference benchmark

Documentation and the Getting Started Guide

Table of contents

Collaborative development

This open-source technology is being developed by the open MLCommons task force on automation and reproducibility led by Grigori Fursin and Arjun Suresh:

• Join our public Discord server.
• Join our public conf-calls.
• Check our news.
• Check our presentation and Forbes article about our development plans.
• Read about our CK concept (previous version before MLCommons).

Copyright

2021-2023 MLCommons

License

Apache 2.0

Acknowledgments

This project is currently supported by MLCommons, cTuning foundation, cKnowledge and individual contributors. We thank HiPEAC and OctoML for sponsoring initial development.