cmind 0.7.4

cmind

Collective Mind toolkit

The Collective Mind toolkit (CM) transforms Git repositories, Docker containers, Jupyter notebooks, zip/tar files and any local directory into a collective database of reusable artifacts and automation scripts with a unified interface and extensible meta descriptions.

Our goal is to help researchers and engineers exchange their artifacts, knowledge, experience and best practices in a more automated, reusable, portable and unified way across rapidly evolving software and hardware.

CM is motivated by our tedious experience reproducing 150+ ML and Systems papers when our colleagues have spent many frustrating months communicating with each other and trying to understand numerous technical reports, README files, specifications, dependencies, ad-hoc scripts, tools, APIs, models and data sets of all shared projects to be able to validate experimental results and adapt ad-hoc projects to the real world with very diverse and continuously changing software, hardware, user environments, settings and data.

The Collective Mind toolkit is based on the Collective Knowledge concept (CK) that was successfully validated in the past few years to provide a simple, common and extensible format and API for shared projects and make it easier for researchers and engineers to communicate, collaborate and innovate. The CK prototype was used to enable collaborative ML and Systems R&D, connect MLOps and DevOps by treating models, datasets and other artifacts as "code" packages, automate the MLPerf inference benchmark, and automate the development and deployment of Pareto-efficient ML Systems in the real world. We are desiging the CM toolkit based on all the feedback we have received from these projects.

See related slides about our motivation and a related article about "MLOps Is a Mess But That's to be Expected" (March 2022).

License

Apache 2.0

How it works

• Check this getting started tutorial to understand the Collective Mind concepts and try this toolkit.

Community meetings

• Public notes
• Regular conf-calls

News

• 2022 April 20: Join us at the public MLCommons community meeting. Register here.

• 2022 April 3: We presented our approach to bridge the growing gap between ML Systems research and production at the HPCA'22 workshop on benchmarking deep learning systems.

• 2022 March: We presented our concept to enable collaborative and reproducible ML Systems R&D at the SIAM'22 workshop on "Research Challenges and Opportunities within Software Productivity, Sustainability, and Reproducibility"

• 2022 March: we've released the first prototype of the Collective Mind toolkit (CK2) based on your feedback and our practical experience reproducing 150+ ML and Systems papers and validating them in the real world.

Documentation

• User guide and API (under development)

Development

CM core

We use GitHub tickets to improve and enhance the CM core that manages shared projects as a collective database of reusable artifacts and automations. Please don't hesitate to share your ideas and report encountered issues!

Reusable CM components

We are developing reusable CM components to bridge the gap between MLOps and DevOps and make it easier to co-design, benchmarking, optimize and deploy AI and ML system across continuously changing software and hardware stacks: https://github.com/octoml/cm-mlops .

Modular CM-based projects

TBA

Resources

• MLOps

Acknowledgments

We thank the users and partners of the original CK framework, OctoML, MLCommons and all our colleagues for their valuable feedback and support!

Contacts

• Grigori Fursin - author and coordinator
• Arjun Suresh - coordinator and maintainer