Hyperion VoxProfile

This repository is a Hyperion-maintained fork and Python packaging of Vox-Profile, a speech foundation-model benchmark for characterizing diverse speaker and speech traits. It keeps the original Python import name, vox_profile, while the published distribution is named hyperion-voxprofile.

Hyperion's changes are focused on dependency constraints and packaging compatibility with the Hyperion toolkit. This fork is not an official release, endorsement, or representation of the original authors.

Installation

Python 3.10 or newer is required:

python -m pip install hyperion-voxprofile

The installable distribution name and import name intentionally differ:

from vox_profile.model.accent.whisper_accent import WhisperWrapper

The package contains model architecture and adapter code only. It does not include model weights, datasets, audio, caches, or other large artifacts.

Basic use

Models are downloaded separately from Hugging Face when from_pretrained(...) is called. For example, with an appropriate model checkpoint and its dependencies:

import torch
from vox_profile.model.accent.whisper_accent import WhisperWrapper

device = "cuda" if torch.cuda.is_available() else "cpu"
model = WhisperWrapper.from_pretrained(
    "tiantiaf/whisper-large-v3-narrow-accent"
).to(device)
model.eval()

Prepare input audio as mono, 16 kHz audio no longer than 15 seconds. Inputs shorter than 3 seconds may produce unreliable predictions according to the original project. See the examples and the available model collection.

Dependencies and compatibility

The fork retains the upstream runtime dependencies while constraining their versions so it can coexist with Hyperion. PyTorch and TorchAudio installation may need to follow the official command for the target CPU/CUDA/ROCm platform; the package metadata provides the general PyPI requirements. No Git or VCS dependency is declared. If a platform-specific PyTorch build is needed, install it separately before installing this package.

Models, external assets, and limitations

Model weights are external Hugging Face assets and are not redistributed by this package. Review the license and terms for each checkpoint, the underlying pre-trained model, and any dataset used to train it before downloading or redistributing them. The model collection and the Vox-Profile paper provide the current checkpoint and research details.

These models produce statistical predictions about speech and speaker traits; they are research tools, not ground truth, identity verification, medical advice, employment decisions, or safety-critical systems. Consider accuracy, bias, consent, privacy, jurisdictional requirements, and the possibility of harm before using outputs. The original project is under peer review and model coverage and performance may change.

Attribution and license

Vox-Profile was created by Tiantian Feng and collaborators. Please cite the original work:

@article{feng2025vox,
  title={Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits},
  author={Feng, Tiantian and Lee, Jihwan and Xu, Anfeng and Lee, Yoonjeong and Lertpetchpun, Thanathai and Shi, Xuan and Wang, Helin and Thebaud, Thomas and Moro-Velazquez, Laureano and Byrd, Dani and others},
  journal={arXiv preprint arXiv:2505.14648},
  year={2025}
}

The source code remains distributed under the complete Responsible AI Source Code License v1.1 (RAIL), including its restrictions, notice requirements, and termination provisions. The full license file is included in source distributions and wheels. Read it before use or redistribution; this README does not replace the license and does not grant permission for restricted uses.

Development

python -m pip install -e .
python -m build
python -m twine check dist/*

Maintainer publishing setup

The repository workflow publishes only when a GitHub Release is published. It expects the release target branch to be main and uses PyPI Trusted Publishing (OIDC), not an API token.

Before the first release:

1. Sign in to PyPI and search for hyperion-voxprofile. If it is unregistered, create the project by publishing through the configured trusted publisher; PyPI creates the project on the first successful upload. If the name is already registered by someone else, do not attempt to impersonate it: choose an available name or contact PyPI only if you have a legitimate naming/trademark claim.
2. In PyPI account settings, add a Trusted Publisher for owner hyperion-ml, repository vox-profile-release, workflow filename .github/workflows/python-publish.yml, and environment pypi.
3. In GitHub, create the pypi environment. Restrict deployment to the maintainers who may publish and, if desired, require reviewers or other environment protection rules. Do not add a PyPI token secret; the workflow needs only contents: read and id-token: write.
4. From main, create a version tag and a GitHub Release for that tag. Set the release target to main and publish it; drafts and ordinary pushes or pull requests do not publish.
5. Verify the workflow run, then inspect the PyPI project page for the wheel and source archive, metadata, Python requirement, dependencies, project URLs, and included LICENSE file. Test installation in a clean Python 3.10+ environment.

See the GitHub workflow for the exact Trusted Publisher values used by this repository.