MAPSS

MAPSS (Manifold-based Assessment of Perceptual Source Separation) is an ICLR 2026 metric package for source-separation systems. It reports two complementary frame-level measures in [0, 1], where higher is better:

• Perceptual Separation (PS): separation of an estimated source from competing references.
• Perceptual Match (PM): perceptual match between an estimated source and its assigned reference.

The public interface accepts N ordered reference waveforms and the corresponding N ordered system outputs. It supports WAV paths, NumPy arrays, and PyTorch tensors.

Install

MAPSS supports Python 3.10-3.12.

python -m pip install "mapss-measures==1.1.1"

Install the optional plotting command as well:

python -m pip install "mapss-measures[plot]==1.1.1"

The default pretrained backbone is downloaded from Hugging Face on first use. A CUDA GPU is recommended; CPU execution is supported but slower.

Python quick start

from mapss import mapss

references = [
    "reference_source_1.wav",
    "reference_source_2.wav",
]
outputs = [
    "estimated_source_1.wav",
    "estimated_source_2.wav",
]

result = mapss(
    reference=references,
    output=outputs,
    source_names=["source_1", "source_2"],
    model="wav2vec2",
    layer=2,
    alpha=1.0,
    seed=42,
)

print(result.summary)
result.save("mapss_results")

Input contract: output[i] must estimate reference[i]. MAPSS requires at least two sources and does not silently find or change the source assignment.

Inputs are downmixed to mono and resampled to the paper's 16 kHz operating rate. By default, unequal signal durations are rejected; use length_policy="trim" only when shortest-length alignment is intentional.

Command line

The same evaluation can be run without writing a Python script:

python -m mapss \
  --reference reference_1.wav reference_2.wav \
  --output estimate_1.wav estimate_2.wav \
  --source-name source_1 --source-name source_2 \
  --model wav2vec2 --layer 2 --alpha 1.0 --seed 42 \
  --results-dir mapss_results

Windows PowerShell uses the same module command with backtick line continuations:

python -m mapss `
  --reference "C:\data\reference_1.wav" "C:\data\reference_2.wav" `
  --output "C:\data\estimate_1.wav" "C:\data\estimate_2.wav" `
  --model wav2vec2 --layer 2 `
  --results-dir mapss_results

Add more paths to both lists for mixtures with more sources. Keep the two lists the same length and order.

Results and plotting

result.save(...) and --results-dir write:

• ps_scores.csv: frame timestamps and PS for every source;
• pm_scores.csv: frame timestamps and PM for every source;
• confidence.csv: paper-derived error quantities when add_ci=True;
• summary.csv: convenient per-source means and valid-frame counts.

Plot all sources after installing the plot extra:

python -m mapss.plotting mapss_results

The figure is saved as mapss_results/ps_pm_over_time.png.

Inactive frames are stored as NaN and excluded from convenience means. Do not replace them with zero. The paper's formal PS utterance aggregation is defined in Appendix B.4 and is not a plain frame mean.

Important keyword arguments

Supported representations are wav2vec2, wavlm, hubert, their _base variants, wav2vec2_xlsr, and raw. The raw model is only for fast installation tests, not scientific reporting.

Documentation

• Clean-laptop quick start
• Python API reference
• Interpreting PS and PM
• Troubleshooting
• Reproducible evaluation
• Grand-challenge integration
• Examples

Citation

@inproceedings{ivry2026mapss,
  title     = {MAPSS: Manifold-based Assessment of Perceptual Source Separation},
  author    = {Ivry, Amir and Cornell, Samuele and Watanabe, Shinji},
  booktitle = {International Conference on Learning Representations},
  year      = {2026}
}

License

MAPSS is released under the MIT License.