SpeechMix
Explore different way to mix speech model(wav2vec2, hubert) and nlp model(BART,T5,GPT) together.
Implementation of:
Large-Scale Self- and Semi-Supervised Learning for Speech Translation - ACL2021
Multilingual Speech Translation with Efficient Finetuning of Pretrained Models - ACL2021
Lightweight Adapter Tuning for Multilingual Speech Translation - Interspeech 2021
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task - ACL2021
A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks - ICASSP 2021
Installation
pip install
pip install speechmix
Build from source
git clone and cd into this project.
pip install -e .
Name the project(!important)
WANDB_PROJECT=amazing
base
python train.py --speech_model_config wav2vec2 \
--nlp_model_config facebook/bart-base \
--SpeechMixEED \
--dataset librispeech_asr \
--field clean \
--train_split train.100 \
--test_split validation \
--batch 3 \
--grad_accum 20 \
--epoch 30 \
--worker 15 \
--share_layer_ratio 0 \
--down_scale 2 \
--lr 4e-5 \
--warmup_steps 500 \
--wandb \
--notes base
1 maintainer
Metadata
1 maintainer
Release files for speechmix 0.0.32
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| speechmix-0.0.32.tar.gz | 18.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| speechmix-0.0.32-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 31.3 kB
Release files / speechmix-0.0.32.tar.gz
| Download URL | speechmix-0.0.32.tar.gz |
|---|---|
| Size | 18.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6095e3e5b392c90b68d5bf5b7258374d795879ca2ac92b74f19cb0e81460e058
|
|
BLAKE2b-256 checksum How to use checksums |
04e08453be39c4d6e137addaa530d3074ff8de2111a826dc82d88df46b4942a7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/33.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.9
|
Release files / speechmix-0.0.32-py3-none-any.whl
| Download URL | speechmix-0.0.32-py3-none-any.whl |
|---|---|
| Size | 13.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
93b82a6e19e82f2faf3f4378c68d64c0856aeeaa002b0934bb7101235215152a
|
|
BLAKE2b-256 checksum How to use checksums |
388b58a8493898f1f2d1900ecb0175f62f09780886ff4181ae13d6dc8f05c334
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/33.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.9
|
