VectorAlign

Bilingual word alignment using multilingual embeddings — no training required!

For non-nerds: a word matching engine for any language pair using parallel data.

Installation

pip install vectoralign

Quick Start

from vectoralign import align

# Your parallel sentences
english = [
    "Hello world",
    "How are you today",
    "The weather is nice"
]
hindi = [
    "नमस्ते दुनिया",
    "आज आप कैसे हैं",
    "मौसम अच्छा है"
]

# Align and build dictionary
dictionary = align(english, hindi)

Features

• No training required — Uses pre-trained multilingual embeddings
• Batch processing — Batch processing with automatic CUDA detection
• Multiple models — Supports LaBSE, mBERT, and other HuggingFace models
• Bidirectional alignment — Intersection of forward and backward alignments

Version 0.2.0 added features:

• Memory management — Automatic garbage collection and CUDA cache management for efficient memory usage

Advanced Usage

from vectoralign import align

# Custom model and batch size
dictionary = align(
    src_sentences,
    tgt_sentences,
    model_name="setu4993/LaBSE",  # or "bert" for mBERT
    batch_size=64,
    threshold=0.6,  # Sentence similarity threshold
    output="my_dictionary.txt"
)

Supported Models

Output Format

The dictionary is saved as a TSV file:

word1    translation1    count
word2    translation2    count

Acknowledgments

This is a spiritual implementation of SimAlign by the Centre for Language and Information Processing, LMU Munich.

• Paper

License

MIT License