transformers-mic

A Model Integration Contract (MIC) is a proposed standard: next to a Hub model, an executable check that it works in a framework, run by that framework's CI. Each MIC has three parts:

• a fixture: a shrunk model built from the real checkpoint's config, with its tokenizer and processor files copied unchanged
• a usage path: runnable Markdown that does what users do, such as generating text or running a training step
• baselines: the outputs that usage produced, reviewed by a human

Fixtures run on framework PRs (CPU, seconds); real checkpoints run after merge, nightly and before release. A slim pass is a fast regression signal, not a pass for the real model.

This repository holds the tmic CLI, the fixture builder and the runner. MICs are published in huggingface-mic and pinned by the fleet dataset; coverage is in the coverage space. In Transformers, the contracts job runs on PRs and reports failures without blocking (status).

Try it locally

pip install "transformers-mic[runtime] @ git+https://github.com/huggingface/transformers-mic.git" \
  "hf-doc-builder @ git+https://github.com/huggingface/doc-builder.git"
hf auth login
tmic run org/model --local-only --fixtures-only --no-history --bill-to ''

This drafts a contract for org/model, builds its fixture and records baselines on your machine, asks you to review each step, and uploads nothing. See Try it locally for the flags. tmic run --mode agent gives agents a JSON protocol for the same questions.

Cloud baselines and publication are for maintainers: CLI guide, cloud baselines (dashboard activation is still pending).

Documents

Start with:

• CLI guide: tmic, from a checkpoint to a published MIC.
• The standard: why MICs, the repository layout, manifest, checks and pins.
• Small fixtures: the shrink recipe and the weight experiments.
• Evidence: replays, backfills, stability and cost.
• Pilot: the proposed adoption pilot and the results so far.

Details: full example, technical design, Hub layout, slim models in Transformers CI, Transformers CI proof, improving contracts, smoke vs assertions, problems and open questions, cloud baselines, end-to-end tests, agent walkthroughs, maintainer slides, changelog, releasing.