midigpt

A transformer model for computer-assisted multitrack music composition.

• Fill in missing bars while preserving your existing arrangement
• Generate new tracks from scratch, conditioned on musical attributes
• Steer the output with attribute controls (note density, polyphony, note duration, and on prism/expressive also key, pitch range, pitch classes and genre), globally or per bar
• Run a MIDI-GPT API server over HTTP, with streaming, cancellation and several models in one process
• One-line setup — load pretrained models from HuggingFace Hub, no compiler needed

Installation

pip install "midigpt[inference]"

Pre-built wheels for CPython 3.10–3.14 on Linux (x86_64), macOS (x86_64 + arm64), and Windows (AMD64). No compiler needed.

On Intel Macs, midigpt[inference] needs Python 3.12 or older: PyTorch stopped publishing macOS x86_64 wheels after 2.2, so it installs with torch<2.3 and numpy<2.

Quick start

Load a pretrained model from HuggingFace Hub and generate music in four lines:

from midigpt import Score, Track, Bar
from midigpt.inference import InferenceEngine, GenerationRequest, InferenceConfig, TrackPrompt

engine = InferenceEngine.from_pretrained("yellow_medium")

# 4-bar score with one empty melodic track
score = Score(tracks=[Track(bars=[Bar() for _ in range(4)])])

result = engine.session(
    score,
    GenerationRequest(
        tracks=[TrackPrompt(id=0, bars=[0, 1, 2, 3])],
        config=InferenceConfig(model_dim=4, mask_mode="attention"),
    ),
).run()

total = sum(len(b.notes) for t in result.tracks for b in t.bars)
print(f"Generated {total} notes")
result.to_midi("output.mid")

The model is downloaded once and cached by huggingface_hub in ~/.cache/huggingface/hub/. When the model repo is gated, accept its license on Hugging Face and run hf auth login (or set HF_TOKEN) first; see access tokens.

Models

InferenceEngine.from_pretrained(name) downloads the current checkpoint for that model name.

model_dim in InferenceConfig is the context window in bars, not a vocabulary dimension — pass a value from the model's num_bars_map. expressive_medium additionally encodes sub-grid timing via delta tokens and supports switchable velocity/microtiming controls. yellow has no MaskBar token, so use mask_mode="attention" (or "attention_approx", "attention_skip", "remove") with it. See docs/models.md for the full breakdown.

The weights are licensed separately from the code, under the Metacreation Non-Commercial Research License: non-commercial research (including research-creation) and education. They were trained on GigaMIDI; see the model card for details.

Inference API

Load a model

# By name (downloads from Metacreation/MIDI-GPT on HuggingFace Hub)
engine = InferenceEngine.from_pretrained("yellow_medium")   # or "prism_medium", "expressive_medium"

# From a local checkpoint (.safetensors or .pt bundle)
engine = InferenceEngine.from_checkpoint("path/to/model.safetensors")

Infill existing bars

score = Score.from_midi("my_song.mid")

request = GenerationRequest(
    tracks=[
        TrackPrompt(id=0, bars=[4, 5, 6, 7]),   # bars to regenerate
        TrackPrompt(id=1, bars=[], ignore=True), # leave track 1 unchanged
    ],
    config=InferenceConfig(temperature=1.0, top_p=0.95, model_dim=8, mask_mode="attention"),
)

result = engine.session(score, request).run()
result.to_midi("output.mid")

Autoregressive generation from scratch

score = Score(tracks=[Track(bars=[Bar() for _ in range(8)])])   # one empty 8-bar track

request = GenerationRequest(
    tracks=[
        TrackPrompt(
            id=0,
            bars=list(range(8)),                   # bars to generate, in order
            autoregressive=True,
            attributes={"max_polyphony": 3},      # quantized attribute level
            controls={"time_signature": 0},        # index into encoder TS list
        ),
    ],
    config=InferenceConfig(temperature=1.0, model_dim=8, mask_mode="attention", polyphony_hard_limit=4),
)
result = engine.session(score, request).run()

Every track in the score needs a TrackPrompt; use ignore=True for tracks to leave out.

Key types

TrackPrompt fields

Sampling filters

InferenceConfig exposes a four-stage logit-filtering pipeline (top_k → top_p → mask_k → mask_p):

A small mask_k=1 or mask_p=0.3 pushes the model off its most-confident picks — useful for getting diverse outputs when novelty_check=True.

Generation options

A few more InferenceConfig fields (see docs/api.md for all of them):

Mask modes

Control how future bars appear in the context window:

Set via InferenceConfig(mask_mode="attention"). "attention" works on all encoders; "token" requires the encoder vocab to include a MaskBar domain.

Attribute controls

Introspect available controls at runtime:

engine._analyzer.attribute_sizes()         # {"note_density": 10, "min_polyphony": 10, ...}
engine._analyzer.attribute_value_labels()  # {"note_density": ["very sparse", ...], ...}
engine._analyzer.attribute_track_types()   # {"note_density": "drum", "min_polyphony": "melodic", ...}

Pass quantized levels (integers in [0, size)) in TrackPrompt.attributes. Each attribute applies to melodic tracks, drum tracks or both (attribute_track_types()); on yellow, note_density is drum-only and polyphony and note duration are melodic-only.

Training

1. Preprocess parquet shards

python -m midigpt.training.preprocess \
    --parquet /data/train/*.parquet \
    --checkpoint models/yellow_medium-final.safetensors

Builds a valid-index cache so dataset initialization is instant on subsequent runs. Cached in ~/.midigpt/ (override with MIDIGPT_CACHE).

2. Launch training

python -m midigpt.training.trainer \
    --config     models/train_config.json \
    --train-data /data/train/*.parquet \
    --eval-data  /data/valid/*.parquet \
    --output-dir checkpoints/run_001

3. Python API

from midigpt.training.trainer import TrainConfig, train

config = TrainConfig.from_file("models/train_config.json")
train(config, train_path="/data/train/00000.parquet", eval_path="/data/valid/00000.parquet")

train() uses PyTorch Lightning and writes a packed .safetensors bundle at the end of training containing weights, architecture config, and encoder config.

Key TrainConfig fields

HTTP server

pip install "midigpt[http]"

# From a local checkpoint (.safetensors or .pt)
midigpt-http --ckpt checkpoints/run_001/model_final.safetensors --port 8000

# From HuggingFace Hub, by checkpoint name (optionally from another repo with --hf-repo)
midigpt-http --pretrained yellow_medium --port 8000

# Several models in one server
midigpt-http --config models.yaml

A stateless REST API — every request carries the full score and generation parameters. The interactive API docs are available at http://localhost:8000/docs.

A generation whose client disconnects is cancelled; --max-generation-seconds caps how long one may run. See docs/http.md for the config file, response fields and streaming.

# Score: 1 melodic track, 4 empty bars — generate all 4 from scratch
curl -s -X POST http://localhost:8000/generate \
  -H "Content-Type: application/json" \
  -d '{
    "score": {
      "resolution": 480, "tempo": 500000,
      "tracks": [{
        "instrument": 0, "track_type": "melodic",
        "bars": [
          {"ts_numerator": 4, "ts_denominator": 4, "notes": []},
          {"ts_numerator": 4, "ts_denominator": 4, "notes": []},
          {"ts_numerator": 4, "ts_denominator": 4, "notes": []},
          {"ts_numerator": 4, "ts_denominator": 4, "notes": []}
        ]
      }]
    },
    "request": {
      "tracks": [{"id": 0, "bars": [0, 1, 2, 3]}],
      "config": {"model_dim": 4, "mask_mode": "attention"}
    }
  }' | jq .score

The server runs on the CPU by default. Add --device cuda for an NVIDIA GPU (with a CUDA build of PyTorch), --device mps for Apple Silicon, or --device auto to pick CUDA, then MPS, then CPU. See docs/http.md.

Real-time OSC server

pip install "midigpt[realtime]"
midigpt-server --ckpt models/yellow_medium-final.safetensors --port 7400

Listens for OSC messages on a UDP port and streams generated notes back in real time. Generation is triggered bar-by-bar via /midigpt/bar/end on a background thread.

Selected OSC addresses:

Development

Setup

git clone https://github.com/Metacreation-Lab/MIDI-GPT.git
cd MIDI-GPT
pip install -e ".[inference,dev]"   # compiles the C++ extension in-place

Prerequisites: Python 3.10+, CMake 3.21+, a C++20 compiler.

Tests

# Python
pytest tests/python/
pytest tests/python -m "not slow and not inference"   # CI subset (no model needed)

# C++
cmake -S . -B build_cpp -DCMAKE_BUILD_TYPE=Release
cmake --build build_cpp -j
ctest --test-dir build_cpp --output-on-failure

Linting

ruff check src/ tests/    # lint
ruff format src/ tests/   # format

pre-commit runs both automatically on commit:

pip install pre-commit && pre-commit install

Release

Tag a commit vX.Y.Z → .github/workflows/wheels.yml builds wheels on Linux / macOS / Windows × Python 3.10–3.14, drafts a GitHub Release, and publishes to PyPI via OIDC Trusted Publishing.

Logging

Set MIDIGPT_LOG_LEVEL=DEBUG (or a numeric level) before importing. Accepts both string names (DEBUG, INFO, WARNING) and integers.

Citation

@misc{pasquier2025midigptcontrollablegenerativemodel,
      title={MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition},
      author={Philippe Pasquier and Jeff Ens and Nathan Fradet and Paul Triana and Davide Rizzotti and Jean-Baptiste Rolland and Maryam Safi},
      year={2025},
      eprint={2501.17011},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2501.17011},
}

License

The code is released under the MIT License — Copyright (c) 2026 Metacreation Lab. See LICENSE.

The pretrained weights are not covered by the MIT License: they are under the Metacreation Non-Commercial Research License, for non-commercial research and education. For other uses, contact pasquier@sfu.ca.