hunterFormsBS 0.1.0

Flexible frequency-band splitter for music source separation. The BandSplitRoformer module supports BSRoformer, MelBandRoformer, and custom overlapping or non-overlapping band configurations. Fully typed, modular, and documented, including migration help, usage, and paper references. PyTorch; CUDA-accelerated.

hunterFormsBS

A flexible frequency-band splitter for music source separation, organized around a single separator family that can express BS-style, mel-style, and custom layouts.

Instead of treating BSRoformer and MelBandRoformer as separate architectures, this package treats them as different band-layout configurations of one core design centered on BandSplitRotator.

The codebase is implemented in PyTorch, fully typed (py.typed), and designed for modular reuse so research ideas (for example PoPE or custom filter banks) can be integrated without splitting into parallel implementations.

Quick fix: size mismatch when loading a checkpoint

If loading a BSRoFormer checkpoint raises a size-mismatch error, check mask_estimator_depth in the configuration.

Some upstream configurations effectively used mask_estimator_depth=1 even when set to 2 because a later subtraction was applied. This package removes that subtraction, so the direct equivalent is:

• set mask_estimator_depth=1

Updating that value resolves the most common mismatch quickly.

Why this architecture helps in practice

• Forward-looking architecture: A single model family makes it easier to adopt new ideas, such as PoPE or custom band-split definitions, while keeping interfaces aligned with established ecosystems.
• Universal configuration: Configurable backward compatibility with existing standards.
• Rich tooling & Ecosystem: The package provides strong typing (py.typed), modular APIs, and rich docstrings focusing on usage, literature citations, and migration paths.

Easy to migrate

Transitioning from other standard implementations is straightforward because most identifiers are exactly the same and the data flow is highly similar.

If you're changing from an existing codebase, you can use the transition modules: simply keep using the BSRoformer and MelBandRoformer namespaces and APIs as a bridge, unify your other classes, and then switch to BandSplitRotator when you're ready.

What is unified here

The key design idea is that the difference between the BS-style front end and the mel-band front end is treated as a band-layout problem, not as a reason to maintain two unrelated model families.

• hunterFormsBS.bandSplitRotator.BandSplitRotator is the new universal entry point.
• hunterFormsBS.bs_roformer.BSRoformer and hunterFormsBS.mel_band_roformer.MelBandRoformer serve as transition modules, keeping familiar APIs, upstream names, and defaults.
• hunterFormsBS.bandSplit.BandSplit, hunterFormsBS.bandSplit.MaskEstimator, and hunterFormsBS.attend.Transformer hold the reusable typed building blocks shared across those entry points.

At the band level, the model only needs a band-membership map, called mask_filter_bank in the codebase. You can think of that map as a Boolean matrix

$$ F \in {0, 1}^{B \times N_f} $$

where $B$ is the number of bands and $N_f$ is the number of STFT frequency bins.

• In a non-overlapping BS-style layout, each frequency bin belongs to exactly one band, so

  $$ \forall f,; \sum_b F_{b,f} = 1. $$

• In an overlapping mel-style layout, some frequency bins belong to more than one band, so

  $$ \exists f \text{ such that } \sum_b F_{b,f} > 1. $$

When bands overlap, the reconstructed mask for a frequency bin is averaged across the contributing bands:

$$ \hat{M}{f,t} = \frac{1}{S_f} \sum{b : F_{b,f} = 1} \hat{M}^{(b)}{f,t}, \qquad S_f = \sum_b F{b,f}. $$

That is why this package makes it easy to move between overlapping and non-overlapping bands, and to change how bands are distributed across the frequency axis. The architectural difference lives in the filter bank, not in two separate theories of the model.

Which entry point you should use

Use this	When	Why
hunterFormsBS.BandSplitRotator	You are starting new work or want one separator that can cover BS-style, mel-style, and custom band layouts.	This is the unified model entry point.
hunterFormsBS.bs_roformer.BSRoformer	You want the familiar non-overlapping BS-style interface or a close comparison with upstream BS-RoFormer code.	The constructor keeps BS-oriented defaults and compatibility fields.
hunterFormsBS.mel_band_roformer.MelBandRoformer	You want the familiar mel-band interface or a close comparison with upstream mel-band code.	The constructor keeps mel-oriented defaults and automatic mel-band construction.
hunterFormsBS.*_experimental	You are testing research ideas such as value residual learning or hyper-connections.	These modules are exploratory and intentionally separate from the stable path.

Package map

Module	Main symbols	Purpose
hunterFormsBS.__init__	BandSplitRotator, BandSplit, MaskEstimator, Transformer, lossComputation, DEFAULT_FREQS_PER_BANDS, ComputeLoss, FlashAttentionConfig, KwargsOfAttention, KwargsSTFT, KwargsTransformer	Public top-level namespace for the stable typed API.
hunterFormsBS.bandSplitRotator	BandSplitRotator	Unified separator that can build BS-style, mel-style, or custom band layouts from one model family.
hunterFormsBS.bs_roformer	BSRoformer	Stable compatibility module for the non-overlapping BS-style variant.
hunterFormsBS.mel_band_roformer	MelBandRoformer	Stable compatibility module for the overlapping mel-band variant.
hunterFormsBS.bandSplit	BandSplit, MaskEstimator, MLP, lossComputation, DEFAULT_FREQS_PER_BANDS	Shared band projection, mask-estimation heads, BS-style default partition, and training-loss helper.
hunterFormsBS.attend	Attend, Attention, FeedForward, LinearAttention, Transformer	Stable attention, feedforward, linear-attention, and transformer building blocks.
hunterFormsBS.theTypes	ComputeLoss, FlashAttentionConfig, KwargsOfAttention, KwargsSTFT, KwargsTransformer	Typed configuration records used across the package.

Experimental module map

Module	Main symbols	Purpose
hunterFormsBS.attend_experimental	experimental Attention, experimental Transformer	Research-oriented attention blocks with value-residual mixing and hyper-connection support.
hunterFormsBS.bs_roformer_experimental	experimental BSRoformer	Experimental BS-style separator that uses the experimental attention stack.
hunterFormsBS.mel_band_roformer_experimental	experimental MelBandRoformer	Experimental mel-band separator that uses the experimental attention stack.

Architecture in one sentence

The stable separator path is

raw audio → STFT → band gathering → BandSplit → hierarchical attention → MaskEstimator → mask

followed by overlap-aware mask averaging when needed, complex masking in the STFT domain, and inverse STFT reconstruction back to waveform audio.

Top-level exports

The top-level package namespace currently re-exports the stable shared pieces that new users most often need:

• BandSplitRotator

The compatibility classes are intentionally available from their own modules so that imports can stay explicit during comparisons with upstream repos.

Reference materials

Gaussian Error Linear Units (GELUs)

• BibTeX citation. TeX Source with precise formulas for AI agents.
• eprint: arXiv.1606.08415
• Implementations:
  • hendrycks/GELUs
  • torch.nn.GELU

Language Modeling with Gated Convolutional Networks

• Common name: GLU (Gated Linear Units)
• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: proceedings.mlr.press

Attention Is All You Need

(^Which is why there are no other papers on this list.)

• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: proceedings.neurips.cc

Root Mean Square Layer Normalization

• Common name: RMSNorm
• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: proceedings.neurips.cc
• Implementations:
  • bzhangGo/rmsnorm
    • context7
  • hunterhogan/torch_einops_kit.scaleValues.RMSNorm
    • context7

RoFormer: Enhanced Transformer with Rotary Position Embedding

• Common name: RoPE
• BibTeX citation. TeX Source with precise formulas for AI agents.
• DOI: 10.1016/j.neucom.2023.127063
• Free pre-print: arXiv:2104.09864
• Implementations:
  • lucidrains/rotary-embedding-torch
    • context7
  • hunterhogan/rotary-embedding-torch

XCiT: Cross-Covariance Image Transformers

• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: proceedings.neurips.cc

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: proceedings.neurips.cc
• Implementations:
  • Dao-AILab/flash-attention
    • context7

Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation

• BibTeX citation.
• Proceedings: ICLR 2022 Conference
• Implementations:
  • arogozhnikov/einops
    • Official documentation
    • context7

Music Source Separation with Band-Split RoPE Transformer

• Common name: BS-RoFormer
• BibTeX citation. TeX Source with precise formulas for AI agents.
• DOI: 10.1109/ICASSP48485.2024.10446843
• Free pre-print: arXiv:2309.02612
• Implementations:
  • lucidrains/BS-RoFormer
    • context7

Mel-RoFormer for Vocal Separation and Vocal Melody Transcription

• Common name: MelBand-RoFormer
• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: 10.5281/zenodo.14877371
• Implementations:
  • lucidrains/BS-RoFormer
    • context7

Value Residual Learning

• Common name: ResFormer
• BibTeX citation. TeX Source with precise formulas for AI agents.
• Proceedings: 10.18653/v1/2025.acl-long.1375
• Implementations:
  • Zcchill/Value-Residual-Learning
    • context7

Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings

• Common name: PoPE
• BibTeX citation. TeX Source with precise formulas for AI agents.
• eprint: arXiv.2509.10534
• Implementations:
  • lucidrains/PoPE-pytorch
    • context7
  • hunterhogan/PoPE-pytorch
    • context7

Packages and documentation

• pytorch/pytorch
  • official documentation
  • context7
• ZFTurbo/Music-Source-Separation-Training
  • context7
• lucidrains/torch-einops-utils
  • context7
• hunterhogan/torch_einops_kit
  • context7

My recovery