Minimal-SLT

Goal and Purpose

Minimal-SLT is a minimal and clean sign language translation (SLT) package developed for educational purposes based on Hugging Face 🤗 and PyTorch. It is developed for Phoenix14T dataset (DGS->de). But it can be adapted to other datasets easily.

Features

• Feature extraction (e.g., I3D and timm-supported models)
• Transformer model
• Wandb logging
• HF trainer
• BLEU, Chrf, ROUGE, BLEURT evaluation

Installation

Minimal-SLT uses UV. Make sure to install UV first.

uv venv .venv
source .venv/bin/activate
uv pip install minimal-slt
# for evaluation
git clone https://github.com/google-research/bleurt.git
cd bleurt
pip install .
uv pip install rouge_score

Usage

Feature extraction

# timm-supported
uv run minimal_slt feat --config configs/base_config.yaml --feat_type timm --timm_model "timm/vit_base_patch16_clip_224.openai"
# python -m minimal_slt feat --config configs/base_config.yaml --feat_type timm --timm_model "timm/vit_base_patch16_clip_224.openai"

For I3D, first download WLASL pre-trained weights from WLASL repo.

# I3D
uv run minimal_slt feat --config configs/base_config.yaml
# python -m minimal_slt feat --config configs/base_config.yaml

Training

uv run minimal_slt train --config configs/base_config.yaml --outpath your_minimal_slt_path
# python -m minimal_slt train --config configs/base_config.yaml --outpath your_minimal_slt_path

Inference

uv run minimal_slt test --config configs/base_config.yaml --model_path your_minimal_slt_path --output_dir ./artifacts/results
# python -m minimal_slt test --config configs/base_config.yaml --model_path your_minimal_slt_path --output_dir ./artifacts/results

Evaluation

uv run minimal_slt eval -i ./artifacts/results
# python -m minimal_slt eval -i ./artifacts/results

Contact

Please leave an issue if you have questions or issues with the code.

For general questions, email me at shakibyzn <at> gmail.com.