e2-tts-pytorch 0.0.9

E2-TTS in Pytorch

E2 TTS - Pytorch (wip)

Implementation of E2-TTS, Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS, in Pytorch

You can chat with other researchers about this work here

Install

$ pip install e2-tts-pytorch

Usage

import torch

from e2_tts_pytorch import (
    E2TTS,
    DurationPredictor
)

duration_predictor = DurationPredictor(
    transformer = dict(
        dim = 512,
        depth = 2,
    )
)

mel = torch.randn(2, 1024, 512)
text = torch.randint(0, 256, (2, 512)) # use -1 for padding

loss = duration_predictor(mel, text = text)
loss.backward()

e2tts = E2TTS(
    duration_predictor = duration_predictor,
    transformer = dict(
        dim = 512,
        depth = 4,
        skip_connect_type = 'concat'
    ),
)

loss = e2tts(mel, text = text)
loss.backward()

sampled = e2tts.sample(mel[:, :5], text = text)

Citations

@inproceedings{Eskimez2024E2TE,
    title   = {E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS},
    author  = {Sefik Emre Eskimez and Xiaofei Wang and Manthan Thakker and Canrun Li and Chung-Hsien Tsai and Zhen Xiao and Hemin Yang and Zirun Zhu and Min Tang and Xu Tan and Yanqing Liu and Sheng Zhao and Naoyuki Kanda},
    year    = {2024},
    url     = {https://api.semanticscholar.org/CorpusID:270738197}
}