Collective Knowledge - a lightweight knowledge manager to organize, cross-link, share and reuse artifacts and workflows
Project description
Collective Knowledge framework (CK)
News
- We are very excited to join forces with OctoML.ai! Contact Grigori Fursin for more details!
- See our ACM TechTalk about the CK technology, reproducible research, FAIR principles and MLPerf (February 2021).
- The overview of the CK technology has appeared in the Philosophical Transactions A, the world's longest-running journal where Newton published: DOI, ArXiv.
- We are preparing a major redesign of the CK framework to make it more Pythonic and easier to use (CK2) - feel free to contact Grigori Fursin for more details and subscribe for our newsletter!
- We have finished prototyping the CK concept and successfully used it to automate MLPerf inference benchmark submissions for edge devices and the design space exploration of efficient ML/SW/HW stacks in terms of speed, accuracy, energy and costs.
Overview
Collective Knowledge framework (CK) helps to organize software projects as a database of reusable components with common automation actions and extensible meta descriptions based on FAIR principles (findability, accessibility, interoperability and reusability) as described in this article.
Our goal is to help researchers and practitioners share, reuse and extend their knowledge in the form of portable workflows, automation actions and reusable artifacts with a common API, CLI, and meta description. See how CK supports collaborative and reproducible research:
- MLPerf automation workflows (GitHub);
- Real-world use cases from MLPerf, Arm, General Motors, IBM, the Raspberry Pi foundation, ACM, dividiti and other great partners;
- Reddit discussion about reproducing 150 papers;
- Our reproducibility initiatives: methodology, checklist, events.
Documentation
Installation
Follow this guide to install CK framework on your platform.
CK supports the following platforms:
As a host platform | As a target platform | |
---|---|---|
Generic Linux | ✓ | ✓ |
Linux (Arm) | ✓ | ✓ |
Raspberry Pi | ✓ | ✓ |
MacOS | ✓ | ± |
Windows | ✓ | ✓ |
Android | ± | ✓ |
iOS | TBD | TBD |
Bare-metal (edge devices) | - | ± |
Example (without Docker)
Here we show how to pull a GitHub repo in the CK format and use a unified CK interface to compile and run any program (image corner detection in our case) with any compatible data set on any compatible platform:
python3 -m pip install ck
ck pull repo --url=https://github.com/ctuning/ck-crowdtuning
ck ls program:*susan*
ck search dataset --tags=jpeg
ck detect soft --tags=compiler,gcc
ck detect soft --tags=compiler,llvm
ck show env --tags=compiler
ck compile program:cbench-automotive-susan --speed
ck run program:cbench-automotive-susan --cmd_key=corners --repeat=1 --env.MY_ENV=123 --env.TEST=xyz
You can check output of this program in the following directory:
cd `ck find program:cbench-automotive-susan`/tmp
ls -l
tmp-output.tmp - image with detected corners (rename to ppm to view it)
Check CK docs for further details.
Example (with Docker)
We have prepared a CK container with AI and ML components: [Docker], [CK meta]
You can start it as follows:
docker run --rm -it ctuning/ck-ai:ubuntu-20.04
You can then prepare and run portable AI/ML workflows and program pipelines.
More examples of CK workflows, automation actions and reusable artifacts for
- Showroom (public projects powered by CK):
- MLPerf automation
- Student Cluster Competition automation: SCC18, digital artifacts
- Jupyter notebooks: ML/SW/HW DSE for edge devices
- ML-based autotuning project: reproducible paper demo, MILEPOST
- Stable Docker containers with CK workflows: MLPerf example, cKnowledge.io, Docker Hub
- Quantum hackathons
- ACM SW/HW co-design tournaments for Pareto-efficient deep learning
- Portable CK workflows and components for:
- GUI to automate ML/SW/HW benchmarking with MLPerf example (under development)
- Reproduced papers
- Live scoreboards for reproduced papers
- Examples of CK components (automations, API, meta descriptions):
- program : image-classification-tflite-loadgen [cKnowledge.io] [GitHub]
- program : image-classification-tflite [GitHub]
- soft : lib.mlperf.loadgen.static [GitHub]
- package : lib-mlperf-loadgen-static [GitHub]
- package : model-onnx-mlperf-mobilenet [GitHub]
- package : lib-tflite [cKnowledge.io] [GitHub]
- docker : object-detection-tf-py.tensorrt.ubuntu-18.04 [cKnowledge.io]
- docker [GitHub]
- docker : speech-recognition.rnnt [GitHub]
- package : model-tf-* [GitHub]
- script : mlperf-inference-v0.7.image-classification [cKnowledge.io]
- jnotebook : object-detection [GitHub]
CK portal
cKnowledge.io: organizing ML and Systems knowledge in the form of portable CK workflows, automation actions and reusable components:
- All CK ML&systems components
- CK compatible repositories
- CK portable solutions (including MLPerf inference on edge)
- CK-based adaptive containers
- CK modules
- Automation actions
- Portable program workflows
- Meta packages
- Software detection (code, data, models)
- Platform detection
- Shared experiments
- Reproduced results from papers
Contributions
Users can extend the CK functionality via CK modules or external GitHub reposities in the CK format as described here.
Please check this documentation if you want to extend the CK core functionality and modules.
Note, that we plan to redesign the CK core to be more pythonic (we wrote the first prototype without OO to be able to port to bare-metal devices in C but we decided not to do it at the end). We also plan to relicense the framework to Apache 2.0.
Author
Acknowledgments
We would like to thank all contributors and collaborators for their support, fruitful discussions, and useful feedback! See more acknowledgments in the CK journal article.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.