hyperion-voxlect 0.1.1

example_audios/Tianjin_1002906_0_0.wav

Using the Voxlect, we obtain the following probability

Dialect: Jiang-Huai       Probability: 0.002
Dialect: Jiao-Liao        Probability: 0.051
Dialect: Ji-Lu            Probability: 0.169 (Target dialect)
Dialect: Lan-Yin          Probability: 0.001
Dialect: Mandarin         Probability: 0.765
Dialect: Southwestern     Probability: 0.000 
Dialect: Zhongyuan        Probability: 0.013
Dialect: Yue              Probability: 0.000

If you are a Mandarin speaker, you will notice that the first 2 words are spoken likely to be Tianjin dialect, but after that, the generated speech sounds more like standard Mandarin, and our model captures this correctly.

Quick Example 2 - MMS-LID-256 Spanish Dialect

# Load libraries
import torch
import torch.nn.functional as F
from src.model.dialect.mms_dialect import MMSWrapper

# Label List
spanish_dialect_list = [
    "Andino-Pacífico", 
    "Caribe and Central", 
    "Chileno",
    "Mexican", 
    "Penisular", 
    "Rioplatense", 
]
    
# Find device
device = torch.device("cuda") if torch.cuda.is_available() else "cpu"

# Load model from Huggingface
mms_model = MMSWrapper.from_pretrained("tiantiaf/voxlect-spanish-dialect-mms-lid-256").to(device)
mms_model.eval()

# Load data, here just zeros as the example
# Our training data filters output audio shorter than 3 seconds (unreliable predictions) and longer than 15 seconds (computation limitation)
# So you need to prepare your audio to a maximum of 15 seconds, 16kHz and mono channel
max_audio_length = 15 * 16000
data = torch.zeros([1, 16000]).float().to(device)[:, :max_audio_length]
mms_logits, mms_embeddings = mms_model(data, return_feature=True)
    
# Probability and output
mms_prob = F.softmax(mms_logits, dim=1)
print(spanish_dialect_list[torch.argmax(mms_prob).detach().cpu().item()])

Given that the Voxlect Benchmark paper is still under peer-review, we provide limited set of models and model weights before the review is concluded. But below are the models we currently put out.

MMS-LID-256 Models

Whisper-Large V3 Models

Whisper-small Models

Responsible Use: Users should respect the privacy and consent of the data subjects, and adhere to the relevant laws and regulations in their jurisdictions when using Voxlect.

❌ Out-of-Scope Use

• Clinical or diagnostic applications
• Surveillance
• Privacy-invasive applications
• No commercial use

The source code remains distributed under the complete Responsible AI Source Code License v1.1 (RAIL), included in LICENSE. The license contains specific use restrictions, notice requirements, and termination provisions. Read it before using or redistributing this software; this README does not replace the license. Model checkpoints, pretrained models, datasets, and external assets may have separate licenses and terms.

If you like our work or use the models in your work, kindly cite the following. We appreciate your recognition!

@article{feng2025voxlect,
  title={Voxlect: A Speech Foundation Model Benchmark for Modeling Dialects and Regional Languages Around the Globe},
  author={Feng, Tiantian and Huang, Kevin and Xu, Anfeng and Shi, Xuan and Lertpetchpun, Thanathai and Lee, Jihwan and Lee, Yoonjeong and Byrd, Dani and Narayanan, Shrikanth},
  journal={arXiv preprint arXiv:2508.01691},
  year={2025}
}

Maintainer publishing setup

The GitHub Actions workflow publishes only when a GitHub Release is published from main. It uses PyPI Trusted Publishing (OIDC), not a long-lived API token.

For the first release, configure a PyPI GitHub Actions Trusted Publisher with:

• PyPI project: hyperion-voxlect
• Owner: hyperion-ml
• Repository: voxlect
• Workflow filename: .github/workflows/python-publish.yml
• GitHub environment: pypi

Create the pypi environment in GitHub and restrict deployment to authorized maintainers; required reviewers are recommended. Do not add a PyPI token secret. Then create a version tag and a published GitHub Release targeting main. Ordinary pushes, pull requests, and draft releases do not publish.

After the workflow succeeds, verify the wheel and source archive, metadata, dependencies, Python requirement, and included LICENSE on PyPI.

Related work

@article{feng2025vox,
  title={Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits},
  author={Feng, Tiantian and Lee, Jihwan and Xu, Anfeng and Lee, Yoonjeong and Lertpetchpun, Thanathai and Shi, Xuan and Wang, Helin and Thebaud, Thomas and Moro-Velazquez, Laureano and Byrd, Dani and others},
  journal={arXiv preprint arXiv:2505.14648},
  year={2025}
}