chirality 0.1.0

Piano fingering from ergonomic rules, statistical models, and reinforcement learning

Chirality — an RL pipeline for piano fingering

Chirality learns and computes piano fingerings (which finger plays which key) for musical scores. It models the keyboard as a lattice, encodes ergonomic knowledge from the piano-fingering literature as cost rules, learns statistical fingering models from human data, and offers several solvers on top of the same representation:

• a linear agent — exact dynamic programming (Dijkstra over a fingering network), the same approach Parncutt et al. used in the original paper;
• a VNS solver — variable neighborhood search over whole-piece fingerings (after Balliauw et al.), the oracle for any rule set;
• statistical models — the Nakamura fingering HMMs (orders 1–3, Viterbi decoding) and a learned-weight rule model (path-difference learning), both trained on the PIG dataset of human fingerings — currently the systems that match human fingerings best;
• an RL agent — PPO (via skrl) with optional Behaviour Cloning and RND exploration, trained on the Piano-v0 Gymnasium environment, with the HMM log-likelihood available as a learned reward.

Both hands are supported: the left hand is the ergonomic mirror of the right, handled once at the rule-aggregation boundary (see docs/design.md).

Headline numbers (full tables in docs/experiments_summary.md): on the PIG benchmark, human annotators agree with each other on 71.4% of notes; our learned-weight rule model reaches 65.4%, the order-3 HMM 64.4%, the rule solvers ~59–60%, and the best PPO agent 44.8% (right hand).

Rule sets

The cost rules live in src/chirality/rules/ and are validated against the source papers:

Module	Paper	Status
parncutt97	Parncutt, Sloboda, Clarke, Raekallio & Desain (1997), An ergonomic model of keyboard fingering for melodic fragments	All 12 rules; reproduces the paper's Table 2 and all 88 transcribed Table 4 rows plus all 7 playable-fingering counts exactly (docs/table_4.md)
jacobs01	Jacobs (2001), Refinements to the ergonomic model for keyboard fingering	Physical-distance mapping (Rule A), unified large-span (Rule C), finger-4-only weak rule, rule 7 disabled
balliauw17	Balliauw, Herremans, Palhazi Cuervo & Sörensen (ITOR), A VNS algorithm to generate piano fingerings for polyphonic sheet music	15 rules over a 14-key distance metric; soft practical limits; native chord rules (14, 15)
alkasimi07	Al Kasimi, Nichols & Raphael (ISMIR 2007), A simple algorithm for automatic generation of polyphonic piano fingerings	Vertical (within-chord) + horizontal (slice-to-slice, averaged) bathtub costs; crossover penalties

The plan is to keep layering rule sets from further papers on the same lattice representation. Each new rule set should follow the same pattern: pure right-hand rule functions, mirroring at the aggregate, and a reproduction test against the paper's own published numbers (tests/test_table_repro.py is the template).

Statistical models

The statistical branch (src/chirality/hmm/, src/chirality/rules/weight_learning.py) is validated with the same paper-reproduction rigor as the rule sets:

Module	Paper	Status
hmm	Nakamura, Saito & Yoshii (2020), Statistical learning and estimation of piano fingering	Orders 1–3 with Viterbi decoding on our lattice; decoder is note-for-note identical to the authors' reference binary (0/10,225 mismatches, all orders); training on PIG recovers their released parameters to file precision; the paper's Table 2 match rates reproduce to 0.1
hmm.metrics	ibid., Sec. 6.2 + App. A	All four multi-ground-truth match rates (M_gen, M_high, M_soft, M_rec); agrees with the reference evaluator to print precision; the human inter-annotator row reproduces the paper's exactly (71.4/79.1/90.8/84.3)
rules.weight_learning	Radisavljevic & Driessen (ICMC 2004), Path difference learning for guitar fingering problem	Structured perceptron over the 12 Parncutt rule point-sums with an exact weighted Viterbi; learned weights are the best non-human system on the PIG benchmark (65.4 M_gen) and interpretable (thumb-passing ×4, weak-finger rules ≈ off)

The models double as RL machinery: the first-order HMM log-likelihood is registered as the hmm_nll reward (mixable with the rule penalties), and it is the first reward the attention policies were able to follow — see docs/reward_roadmap.md.

chirality fetch-models downloads the pretrained artifacts from the Hugging Face model repo. To retrain from scratch instead: the training data is the PIG dataset (registration required; research and non-profit use) — unpack it to data/PIG/, then chirality hmm-train and chirality learn-weights fit everything.

Installation

pip install chirality
chirality fetch-models     # pretrained models -> ./data (add --ppo for checkpoints)
chirality annotate --input score.mei --output ./annotated --weights --both

fetch-models downloads the pretrained statistical models (learned rule weights + fingering HMMs) from vibetuned/chirality-models into ./data, where every command looks by default — no account needed.

For development the project uses uv and Python ≥ 3.14:

uv sync
uv run pytest          # 334 tests: unit rules + paper reproductions

Paper-reproduction tests against the Nakamura reference artifacts and PIG skip automatically unless scripts/fetch_nakamura_ref.sh has been run and data/PIG/ exists.

CLI

The chirality command (defined in src/chirality/cli.py):

# Sanity-check the Gymnasium env (add --render to watch random actions)
uv run chirality verify-env [--render]

# Watch the linear (Dijkstra) agent play, with live rendering
uv run chirality linear [--rules parncutt|jacobs] [--hand 1|2] \
    [--score-type scale|chord|arpeggio|mei] [--mei-path score.mei] [--debug]

# Watch the whole-piece oracles play (VNS search / HMM Viterbi)
uv run chirality vns [--rules parncutt] [--score-type mixed]
uv run chirality hmm [--order 1|2|3] [--params param_FHMM1.txt]

# Fit the statistical models from the PIG dataset (needs data/PIG)
uv run chirality hmm-train          # HMM params -> data/hmm_params/
uv run chirality learn-weights      # path-difference rule weights

# Train the PPO agent (defaults from conf/base.yaml)
uv run chirality train [--steps 100000] [--use-bc] [--use-rnd] \
    [--score-type scale] [--resume runs/.../checkpoint.pt]

# Test a trained checkpoint with rendering
uv run chirality test --checkpoint runs/piano_ppo/.../best_agent.pt

# Annotate MEI scores with fingerings (see "Annotating scores" below)
uv run chirality annotate --input ./scores/ --output ./annotated \
    --weights --both

Notes:

• --rules selects the ergonomic model (jacobs is the default).
• --hand: 1 = right hand (treble staff), 2 = left hand (bass staff). annotate --both does both staves in one pass.
• --debug on linear saves the fingering-network graph and the env render for each episode under debug_linear/.
• --video clip.gif on linear, vns, hmm, and test records the performance to an animated GIF (3 episodes by default; --episodes overrides). hmm needs the reference parameters staged by scripts/fetch_nakamura_ref.sh (or --params for your own).
• Defaults (hand, score generator, policy architecture, BC/RND settings) come from conf/base.yaml; CLI flags override them.

Annotating scores

chirality annotate writes fingering numbers (<fing> elements) into MEI scores, one method per run — listed here strongest human match first (numbers from docs/experiments_summary.md):

# Learned-weight rule model (best system, 65.4% human match).
# Uses data/learned_weights/parncutt.json when present
# (from `chirality learn-weights` or the Hugging Face repo),
# uniform Parncutt weights otherwise.
uv run chirality annotate --input score.mei --output ./annotated \
    --weights --both

# Nakamura fingering HMM (orders 1-3; order 2/3 ~ 64.4% human match).
# Uses data/hmm_params/ (from `chirality hmm-train`) or the
# reference parameters staged by scripts/fetch_nakamura_ref.sh.
uv run chirality annotate --input score.mei --output ./annotated \
    --hmm --order 2 --both

# Rule-based Dijkstra agent (no trained parameters needed at all)
uv run chirality annotate --input score.mei --output ./annotated \
    --linear --rules jacobs --both

# A trained PPO checkpoint
uv run chirality annotate --input score.mei --output ./annotated \
    --model runs/.../best_agent.pt

--both fingers both staves (treble = right hand, bass = left); --staff 1|2 picks one. Statistical methods decode each staff as a note sequence directly (no environment), so they are fast and handle whole directories of .mei files.

The environment and its renderer

Piano-v0 (src/chirality/envs/piano_env.py) models an 88-key piano as a 52×2 lattice: 52 columns (one per white key) × 2 rows (row 0 = naturals, row 1 = the sharp of that column). Hand position is derived automatically from the pressed fingers, so the agent's action is purely a fingering decision:

• Action: fingers (MultiBinary 5 — which finger presses) and fingers_black (MultiBinary 5 — black-key modifier per finger).
• Observation: the score lookahead as a 2×52×10 grid, the current hand position, the relative distance to the next target, and an action_mask (key colours, note count, allowed fingers).
• Rewards: composable RewardComponents (envs/rewards.py), including the ergonomic rule penalties (envs/parncutt_rewards.py, envs/jacobs_rewards.py).

With render_mode="human" the env opens a matplotlib window with two panels: the score lookahead grid (upcoming targets scrolling toward the keyboard, with episode/step/reward in the title) and the keyboard view showing the current hand span and pressed fingers. render_mode="rgb_array" returns frames for capture instead.

Scores come from generators in src/chirality/scores/: random scales, chord progressions, arpeggios, or real mei files (MEIScoreGenerator parses one staff of an MEI score).

Repository layout

conf/base.yaml               training / env configuration
docs/                        rule write-ups + paper reproduction notes
src/chirality/
  rules/                     ergonomic rule engine (paper-validated)
  envs/                      Piano-v0 env, reward components, renderer
  scores/                    score generators (scales, chords, MEI, demos)
  linear/                    Dijkstra fingering agent
  policies/                  PPO architectures (simple, CNN, transformer, decoder)
  exploration/               BC + RND trainers
  annotate.py                MEI fingering annotation
  cli.py                     typer CLI
tests/                       unit tests + paper reproduction tests
test_data/                   MEI corpus + annotated outputs (gitignored)

Roadmap (next sessions)

• Geometric rule module (rules/geometry.py) — whole-hand costs computable from the lattice: hand-angle regression, frame-shift vs drift, grip-shape reuse, per-finger travel, hand-size-parametric spans (docs/reward_roadmap.md §3).
• Counterpoint generator — reuse the VNS skeleton in linear/vns.py (Herremans & Sörensen used VNS for counterpoint composition): the neighbourhoods mutate notes instead of fingers and the pluggable sequence_cost objective becomes a counterpoint rule set, with fingering cost as a playability term.
• Adversarial imitation — GAIL-style discriminator rewards from VNS or PIG demonstrations; MaxEnt-IRL over the rule features as the principled version of the path-difference weights.
• TiPiano (pending release) — 340K fingertip-key contact events with hand motion capture: fingering supervision at scale plus empirical grounding for the geometry rules.
• Experiment log lives in docs/experiments.md (abridged: docs/experiments_summary.md); the training launchers append their results there automatically.

Documentation

• docs/design.md — architecture and technical choices
• docs/experiments_summary.md — the two headline result tables
• docs/experiments.md — full experiment results log
• docs/reward_roadmap.md — reward-model roadmap and status
• docs/table_4.md — Parncutt Table 4 reproduction and findings
• docs/table_2.md — combined span-rule table (rules 1–3)
• docs/rules_paslclrade97.md, docs/rules_jacobs01.md — rule write-ups

References

Rule-based models:

• Parncutt, R., Sloboda, J. A., Clarke, E. F., Raekallio, M. & Desain, P. (1997). An ergonomic model of keyboard fingering for melodic fragments. Music Perception, 14(4), 341–382.
• Jacobs, J. P. (2001). Refinements to the ergonomic model for keyboard fingering of Parncutt, Sloboda, Clarke, Raekallio, and Desain. Music Perception, 18(4), 505–511.
• Al Kasimi, A., Nichols, E. & Raphael, C. (2007). A simple algorithm for automatic generation of polyphonic piano fingerings. ISMIR.
• Balliauw, M., Herremans, D., Palhazi Cuervo, D. & Sörensen, K. (2017). A variable neighborhood search algorithm to generate piano fingerings for polyphonic sheet music. International Transactions in Operational Research, 24(3), 509–535.

Statistical models and learning:

• Yonebayashi, Y., Kameoka, H. & Sagayama, S. (2007). Automatic decision of piano fingering based on hidden Markov models. IJCAI.
• Nakamura, E., Ono, N. & Sagayama, S. (2014). Merged-output HMM for piano fingering of both hands. ISMIR.
• Nakamura, E., Saito, Y. & Yoshii, K. (2020). Statistical learning and estimation of piano fingering. Information Sciences, 517, 68–85. (arXiv:1904.10237; the PIG dataset)
• Ramoneda, P., Jeong, D., Nakamura, E., Serra, X. & Miron, M. (2022). Automatic piano fingering from partially annotated scores using autoregressive neural networks. ACM Multimedia.
• Radisavljevic, A. & Driessen, P. (2004). Path difference learning for guitar fingering problem. ICMC.

License and acknowledgments

Copyright © 2026 Pedro Fillastre / Vibetuned (non-profit). Licensed under the GNU AGPL v3.0 or later.

This project was developed in close collaboration with Claude (Anthropic) — the paper reproductions, the statistical-model ports and their validation against the reference implementations, the evaluation harness, and large parts of the training pipeline grew out of that pairing.

The statistical models are trained on the PIG dataset (Nakamura, Saito & Yoshii), used under its research/non-profit terms — thanks to the authors for releasing both the data and their reference implementation.