morphoformer 2.3.1

MorphFormer: multilingual morphological reinflection via character-level Transformer

MorphFormer v2

Character-level Transformer for multilingual morphological reinflection.

Installation

pip install morphoformer

From source:

cd Morph_v2
pip install -e .

Requirements: Python >= 3.14, PyTorch >= 2.0

Quick Start

morphoformer download --lang rus,deu,fra --merge
morphoformer train --preset medium --data "data/collections/*_train.tsv" --device cuda
morphoformer infer --checkpoint checkpoints/morphformer_epoch50.pt \
  --word "бежать" --morph "V;IND;PRS;3;SG" --lang rus
morphoformer serve --checkpoint checkpoints/morphformer_epoch50.pt

Presets

Preset	d_model	Encoder	Decoder	dim_ff	~Params	VRAM
small	384	4 layers	3 layers	1024	~7M	< 4 GB
medium	512	8 layers	6 layers	1376	~45M	4–8 GB
large	768	10 layers	8 layers	2048	~120M	>= 8 GB

morphoformer init-config --preset medium --out config.toml

CLI

Command	Description
train	Train model
infer	Single inference from checkpoint
serve	Interactive REPL
download	Download SigMorphon/UniMorph data
modules	List registered modules
init-config	Generate TOML config template

morphoformer --help
morphoformer train --help

Data Format

TSV: lemma\tfeatures\tsurface_form\tlanguage

бежать	V;IND;PRS;3;SG	бежит	rus
gehen	V;IND;PRS;3;SG	geht	deu
aller	V;IND;PRS;3;SG	va	fra

Configuration

[model]
d_model = 512
num_heads = 8
num_kv_heads = 2
dim_ff = 1376
dropout = 0.15
num_languages = 50

[model.encoder]
num_layers = 8
conv = "local"
adapter = "language_conditioned"
adapter_bottleneck = 128

[model.decoder]
num_layers = 6

[model.morph_encoder]
type = "pooled"

[training]
epochs = 50
batch_size = 64
lr = 5e-4
warmup_steps = 4000
device = "auto"

Python API

import torch
from morphoformer import modules
from morphoformer.modules import MorphFormer
from morphoformer.data import CharVocab, FeatureVocab
from morphoformer.inference import greedy_decode

ckpt = torch.load("model.pt", map_location="cpu", weights_only=False)
char_vocab = CharVocab.from_dict(ckpt["char_vocab"])
feature_vocab = FeatureVocab.from_dict(ckpt["feature_vocab"])
# ... build model, load state_dict ...

result = greedy_decode(
    model, char_vocab, feature_vocab,
    "бежать", ["V", "IND", "PRS", "3", "SG"], lang_id=0,
    device=torch.device("cuda"), max_len=96, max_out=128,
)

Modules

$ morphoformer modules
[attention]  gqa, mha, cross
[feedforward]  swiglu, gelu
[norm]  rmsnorm, layernorm
[conv]  local, none
[adapter]  language_conditioned, bottleneck, none
[morph_encoder]  pooled, attention
[position]  rope

Custom module:

from morphoformer.modules.registry import register

@register("feedforward", "my_ffn")
class MyFFN(nn.Module):
    ...

Devices

morphoformer train --device auto    # CUDA -> XPU -> MPS -> CPU
morphoformer train --device cuda    # NVIDIA
morphoformer train --device rocm    # AMD
morphoformer train --device xpu     # Intel Arc
morphoformer train --device mps     # Apple Silicon

Project Structure

morphoformer/
├── cli.py                 # CLI entry point
├── config/
│   ├── schema.py          # Dataclass config schema
│   ├── loader.py          # TOML load/save
│   └── presets.py         # small / medium / large
├── data/
│   ├── vocab.py           # CharVocab
│   ├── feature_vocab.py   # FeatureVocab
│   ├── dataset.py         # MorphDataset + CUDA stream prefetch
│   └── download.py        # SigMorphon downloader
├── modules/
│   ├── registry.py        # @register / get / list_modules
│   ├── transformer.py     # MorphFormer (main model)
│   ├── encoder.py         # Encoder + EncoderLayer
│   ├── decoder.py         # Decoder + DecoderLayer + KV cache
│   ├── attention.py       # GQA, MHA, CrossAttention
│   ├── feedforward.py     # SwiGLU, GeLU
│   ├── norm.py            # RMSNorm, LayerNorm
│   ├── position.py        # RoPE
│   ├── conv.py            # Depthwise conv (Conformer-style)
│   ├── adapter.py         # Language-conditioned adapters
│   └── morph_encoder.py   # Pooled / Attention morph encoder
├── inference/
│   ├── decode.py          # Greedy decode + KV cache
│   └── cache.py           # KVCache dataclass
└── training/
    ├── trainer.py         # Training loop
    ├── scheduler.py       # Cosine LR + warmup
    └── memory.py          # VRAM profiling + auto batch planning

License

MIT