cmind 1.6.0

cmind

About

Collective Mind (CM) is a human-friendly interface to run a growing number of ad-hoc MLPerf, MLOps, and DevOps scripts from MLCommons projects and research papers in a unified way on any operating system with any software and hardware as portable, reusable and extensible automation recipes (CM scripts):

pip install cmind

cm pull repo mlcommons@ck

cm run script "python app image-classification onnx"

cm run script "download file _wget" --url=https://cKnowledge.org/ai/data/computer_mouse.jpg --verify=no --env.CM_DOWNLOAD_CHECKSUM=45ae5c940233892c2f860efdf0b66e7e

cm run script "python app image-classification onnx" --input=computer_mouse.jpg

cm docker script "python app image-classification onnx" --input=computer_mouse.jpg
cm docker script "python app image-classification onnx" --input=computer_mouse.jpg -j -docker_it

cm run script "get generic-python-lib _package.onnxruntime"
cm run script "get coco dataset _val _2014"
cm run script "get ml-model stable-diffusion"
cm run script "get ml-model huggingface zoo _model-stub.alpindale/Llama-2-13b-ONNX" --model_filename=FP32/LlamaV2_13B_float32.onnx --skip_cache

cm show cache
cm show cache "get ml-model stable-diffusion"

cm run script "run common mlperf inference" --implementation=nvidia --model=bert-99 --category=datacenter --division=closed
cm find script "run common mlperf inference"

cm pull repo ctuning@cm-reproduce-research-projects
cmr "reproduce paper micro-2023 victima _install_deps"
cmr "reproduce paper micro-2023 victima _run" 

...

import cmind
output=cmind.access({'action':'run', 'automation':'script',
                     'tags':'python,app,image-classification,onnx',
                     'input':'computer_mouse.jpg'})
if output['return']==0: print (output)

Collective Mind is a community project being developed by the MLCommons Task Force on Automation and Reproducibility with great help from MLCommons (70+ AI organizations), research community and individual contributors - we want to have a common, non-intrusive, technology-agnostic, portable and easily-extensible interface that requires minimal learning curve to start automating all manual and repetitive tasks including downloading artifacts, installing tools, resolving dependencies, running experiments, processing logs, and reproducing results.

That is why we implemented CM as a small Python library with minimal dependencies (Python 3.7+, git, wget), simple Python API and human-friendly command line that simply searches for CM scripts by tags in all pulled Git repositories, automatically generates command lines for a given script or tool on a given platform, updates all paths and environment variables, runs a given automation either natively or inside automatically-generated containers and unifies input and output as a Python dictionary or JSON/YAML file.

Our goal is to make it easier to prototype, build, run, benchmark, optimize and manage complex AI/ML applications across diverse and rapidly evolving models, data sets, software and hardware simply by chaining these unified CM scripts into portable, human-readable and reusable workflows.

Please go to this GitHub page to learn more about this community project and join public Discord server to participate in collaborative developments.

License

Apache 2.0

Copyright

2022-2024 MLCommons