modelco 1.1.1

MoCo: A One-Stop Shop for Model Collaboration Research

MoCo is a toolkit for Model Collaboration research, where multiple language models collaborate and complement each other for compositional AI systems.

Technical report: paper

Quick Start

Environment Setup

We use uv for package management.

uv venv --python 3.10 moco
source moco/bin/activate
uv pip install -r requirements.txt

The nvidia-*-cu12 entries in requirements.txt are bundled CUDA wheels; skip them if your system already has CUDA drivers installed, or install torch via the PyTorch index instead (uv pip install torch==2.7.1 --index-url https://download.pytorch.org/whl/cu126).

uv pip install --no-build-isolation --no-deps git+https://github.com/Dao-AILab/causal-conv1d.git
uv pip install --no-build-isolation --no-deps git+https://github.com/state-spaces/mamba.git

Running Your First Collaboration

If you don't have 3 GPUs, go to model_collaboration/test_config.json and set "gpu_ids": [0], [0,1], or whatever you have; if your GPU is nice, increase batch_size.

python -m model_collaboration.main -c model_collaboration/test_config.json

You will see the outputs and evaluation results in the model_collaboration/logs/ folder.

Supported Methods

MoCo currently supports the following model collaboration algorithms, across API-level, text-level, logit-level, and weight-level collaboration. We provide a sample config for each method in examples/ and please check out docs/user_readme.md for more details about writing configs and the different collaboration methods implemented.

Please note that MoCo does not aim to be a reproducibility study: we adapt the core ideas behind related papers and employ what works flexibly.

Supported Data

MoCo comes with a lot of evaluation datasets built-in, and you are free to bring your own datasets, or even just generate responses only and take evaluation elsewhere. Essentially, change the task and task_type in the config to use diverse datasets. Check out link for more details.

Contributing to MoCo

We welcome contributions to MoCo!

If you are interested in contributing new model collaboration methods, check out link.

If you are interested in contributing new datasets, check out link.

If you have any suggestions, please open an issue.

MoCo-supported projects

Scaling participation in modular AI systems: diverse participants contribute small specialized models that collaborate together, outperforming monolithic LLMs. link

Safety of model collaboration systems: what if one of the models is malicious? link

The single-multi evolution loop: multiple LMs collaborate, distill the collaborative system back into each individual model, and repeat for multi-LLM self-evolution. link

Citation

If MoCo is helpful for you, please consider citing:

@inproceedings{feng2026one,
  title={When one llm drools, multi-llm collaboration rules},
  author={Feng, Shangbin and Ding, Wenxuan and Liu, Alisa and Wang, Zifeng and Shi, Weijia and Wang, Yike and Shen, Shannon Zejiang and Han, Xiaochuang and Lang, Hunter and Lee, Chen-Yu and others},
  booktitle={Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  pages={17048--17063},
  year={2026}
}

@article{feng2026moco,
  title={MoCo: A One-Stop Shop for Model Collaboration Research},
  author={Feng, Shangbin and Bai, Yuyang and Yang, Ziyuan and Wang, Yike and Tan, Zhaoxuan and Yan, Jiajie and Lei, Zhenyu and Ding, Wenxuan and Shi, Weijia and Wang, Haojin and others},
  journal={arXiv preprint arXiv:2601.21257},
  year={2026}
}

Also, please cite the related papers for the methods you employed, as listed in docs/user_readme.md.

Have a nice day.