ck 2.5.5

Collective Knowledge - a lightweight knowledge manager to organize, cross-link, share and reuse artifacts and workflows based on FAIR principles

Collective Knowledge framework (CK)

Linux/MacOS: Windows:

Versions and licenses

• V3 (planning) : Apache 2.0
• V2+ (development) : Apache 2.0
• V1.x (maintenance and bug fixes) : BSD 3-clause

News

• Project website
• CK-powered MLPerf™ benchmark automation
• Community projects to improve and redesign CK
• AI/ML repository with all aggregated CK components
• OctoML's CK-based MLOps/MLPerf repository

Overview

Collective Knowledge framework (CK) helps to organize software projects as a database of reusable components with common automation actions and extensible meta descriptions based on FAIR principles (findability, accessibility, interoperability and reusability) as described in our journal article (shorter pre-print).

Our goal is to help researchers and practitioners share, reuse and extend their knowledge in the form of portable workflows, automation actions and reusable artifacts with a common API, CLI, and meta description. See how CK helps to automate benchmarking, optimization and design space exploration of AI/ML/software/hardware stacks, simplifies MLPerf™ inference benchmark submissions and supports collaborative, reproducible and reusable ML Systems research:

• ACM TechTalk
• AI/ML/MLPerf™ automation workflows and components from the community;
• Reddit discussion about reproducing 150 papers;
• Our reproducibility initiatives: methodology, checklist, events.

Current projects

• Developing a platform to automate SW/HW co-design for ML Systems across diverse models, data sets, frameworks and platforms based on user constraints in terms of speed, accuracy, energy and costs: OctoML.ai & cKnowledge.io
• Automating MLPerf(tm) inference benchmark and packing ML models, data sets and frameworks as CK components with a unified API and meta description
• Providing a common format to share artifacts at ML, systems and other conferences: video, Artifact Evaluation
• Redesigning CK together with the community based on user feedback
• Other real-world use cases from MLPerf™, Qualcomm, Arm, General Motors, IBM, the Raspberry Pi foundation, ACM and other great partners;

Documentation

• Online CK documentation
  • Why CK?
  • CK Basics
  • Try CK
• Publications

Installation

Follow this guide to install CK framework on your platform.

CK supports the following platforms:

	As a host platform	As a target platform
Generic Linux	✓	✓
Linux (Arm)	✓	✓
Raspberry Pi	✓	✓
MacOS	✓	±
Windows	✓	✓
Android	±	✓
iOS	TBD	TBD
Bare-metal (edge devices)	-	±

Examples

Portable CK workflow (native environment without Docker)

Here we show how to pull a GitHub repo in the CK format and use a unified CK interface to compile and run any program (image corner detection in our case) with any compatible data set on any compatible platform:

python3 -m pip install ck

ck pull repo:octoml@mlops

ck ls program:*susan*

ck search dataset --tags=jpeg

ck detect soft --tags=compiler,gcc
ck detect soft --tags=compiler,llvm

ck show env --tags=compiler

ck compile program:image-corner-detection --speed

ck run program:image-corner-detection --repeat=1 --env.MY_ENV=123 --env.TEST=xyz

You can check output of this program in the following directory:

cd `ck find program:image-corner-detection`/tmp
ls

processed-image.pgm

You can now view this image with detected corners.

Check CK docs for further details.

MLPerf™ benchmark workflows

• Image classification
• Object detection

Portable CK workflow (with Docker)

We have prepared m CK containers with ML Systems components:

• https://github.com/ctuning/ai/tree/main/docker
• https://github.com/octoml/mlops/tree/main/docker

You can run them as follows:

ck pull repo:octoml@mlops
ck build docker:ck-template-mlperf --tag=ubuntu-20.04
ck run docker:ck-template-mlperf --tag=ubuntu-20.04

Portable workflow example with virtual CK environments

You can create multiple virtual CK environments with templates to automatically install different CK packages and workflows, for example for MLPerf™ inference:

ck pull repo:octoml@venv
ck create venv:test --template=mlperf-inference-main
ck ls venv
ck activate venv:test

ck pull repo:octoml@mlops
ck install package --ask --tags=dataset,coco,val,2017,full
ck show env

Integration with web services and CI platforms

All CK modules, automation actions and workflows are accessible as a micro-service with a unified JSON I/O API to make it easier to integrate them with web services and CI platforms as described here.

More examples

• See docs

CK portal

We have developed the cKnowledge.io portal to help the community organize and find all the CK workflows and components similar to PyPI:

• Search CK components
• Browse CK components
• Find reproduced results from papers
• Test CK workflows to benchmark and optimize ML Systems

Contributions

Users can extend the CK functionality via CK modules or external GitHub reposities in the CK format as described here.

Please check this documentation if you want to extend the CK core functionality and modules.

Note, that we plan to redesign the CK core to be more pythonic (we wrote the first prototype without OO to be able to port it to bare-metal devices in C but eventually we decided to drop this idea).

Author

• Grigori Fursin

Sponsors

• OctoML.ai
• cTuning foundation

Acknowledgments

We would like to thank all contributors and collaborators for their support, fruitful discussions, and useful feedback! See more acknowledgments in the CK journal article.