Textual Echo Cancellation (TEC)

This is not an officially supported Google product.

Introduction

This repository provides a standalone, open-source Python reproduction of Textual Echo Cancellation (TEC) based on the IEEE SLT 2021 paper:

Textual Echo Cancellation Shaojin Ding, Ye Jia, Ke Hu, Quan Wang Paper: https://arxiv.org/pdf/2008.06006 | Interactive Demo: Hugging Face Space (wq2012/tec) | Audio Samples: https://google.github.io/speaker-id/publications/TEC/

When a user speaks to a smart speaker or voice-enabled device while the device is playing back a Text-to-Speech (TTS) response, the microphone captures a reverberant mixture of the user's speech and the device's TTS playback. Classical Acoustic Echo Cancellation (AEC) requires streaming the full high-bandwidth TTS reference waveform to the echo canceller. Textual Echo Cancellation (TEC) instead uses the lightweight text transcript of the interfering TTS prompt (< 1 KB) as a side input to a multi-source attention sequence-to-sequence neural network, canceling the interfering TTS echo and reconstructing the clean user speech spectrogram and waveform.

   

---

Features

• Complete Model Suite:
  • TecModel: Multi-source sequence-to-sequence model taking noisy/reverberant speech (SpeechEncoderV1) and interfering TTS text (TtsEncoderV2) with MultiSourceFbeDecoderV1 (GmmMonotonicAttention or AdditiveAttention).
  • AecModel (AEC-Seq2seq): Neural baseline taking noisy/reverberant speech and clean reference TTS audio with dual speech encoders and multi-source attention.
  • VanillaSeq2SeqModel (NoSideInput): Single-source sequence-to-sequence speech enhancement baseline without side input.
  • NlmsAec (AEC-NLMS): Classical Normalized Least Mean Squares adaptive filter baseline.
• Standalone Lingvo Implementation: Built on open-source lingvo and tensorflow, with zero dependencies on proprietary internal libraries.

• End-to-End Pipelines & CLI Scripts:
  • Dataset Preparation (scripts/prepare_data.py): Pairs clean speech (LibriTTS) with longer interfering TTS utterances (LJSpeech / VCTK), simulates room impulse response (RIR) reverberation, mixes at a target SNR (default 0 dB), pads trailing zeros, and writes TFRecord datasets.
  • Model Training (scripts/train.py): Trains any registered Lingvo configuration (TecSingleInterfering, TecMultiInterfering, AecSingleInterfering, NoSideInputSingleInterfering, etc.).
  • Inference & Waveform Synthesis (scripts/inference.py): Predicts enhanced log-Mel spectrograms and synthesizes 24 kHz waveforms via Griffin-Lim phase reconstruction (WaveformProcessor).
  • Evaluation (scripts/evaluate.py): Computes 13-MFCC Mel Cepstral Distortion (MCD) with Dynamic Time Warping (DTW), punctuation-normalized Word Error Rate (WER), and model FLOPS / side-input bandwidth.
  • TFLite Export (scripts/export_tflite.py): Exports trained models to .tflite FlatBuffer format (with optional dynamic range quantization) and validates on-device execution with tf.lite.Interpreter.

---

Installation

Install from PyPI:

pip3 install textual-echo-cancellation

Or install from source:

git clone https://github.com/wq2012/tec.git
cd tec
pip3 install -r requirements.txt
pip3 install -e .

---

Quickstart

1. Prepare Training & Evaluation Datasets

You can prepare a TFRecord dataset from CSV manifests (utt_id,wav_path,transcript) of clean speech (e.g., LibriTTS) and interfering TTS speech (e.g., LJSpeech or VCTK), or generate a synthetic dataset for testing:

# Generate a synthetic TFRecord dataset for quick testing:
python3 scripts/prepare_data.py \
  --generate_synthetic \
  --num_synthetic 16 \
  --snr_db 0.0 \
  --reverb_rt60 0.25 \
  --output_tfrecord /tmp/tec_data/train.tfrecord

# Or prepare from LibriTTS + LJSpeech CSV manifests:
python3 scripts/prepare_data.py \
  --clean_manifest_csv /path/to/libritts_train.csv \
  --interfering_manifest_csv /path/to/ljspeech_train.csv \
  --snr_db 0.0 \
  --reverb_rt60 0.25 \
  --output_tfrecord /tmp/tec_data/train.tfrecord

2. Train the Model

python3 scripts/train.py \
  --model TecSingleInterfering \
  --train_file_pattern "/tmp/tec_data/train.tfrecord" \
  --logdir /tmp/tec_checkpoints \
  --max_steps 100 \
  --batch_size 4 \
  --learning_rate 1e-3

Available --model configurations:

• TecSingleInterfering: TEC model (speech + TTS text) for single-speaker TTS interference (LibriTTS + LJSpeech).
• TecMultiInterfering: TEC model (speech + TTS text) for multi-speaker TTS interference (LibriTTS + VCTK).
• AecSingleInterfering / AecMultiInterfering: AEC-Seq2seq baseline (speech + TTS reference audio).
• NoSideInputSingleInterfering / NoSideInputMultiInterfering: Vanilla-Seq2seq baseline (speech mixture only).

3. Run Inference

python3 scripts/inference.py \
  --model TecSingleInterfering \
  --checkpoint_path /tmp/tec_checkpoints/model.ckpt-100 \
  --mixed_wav /path/to/mixed_input.wav \
  --interfering_text "currently in mountain view it is 72 degrees" \
  --output_wav /tmp/enhanced_clean.wav

4. Evaluate MCD, WER, and Model Complexity

python3 scripts/evaluate.py \
  --ref_wav /path/to/clean_reference.wav \
  --pred_wav /tmp/enhanced_clean.wav \
  --ref_transcript "Turn off the bedroom lights!" \
  --hyp_transcript "turn off the bedroom lights" \
  --print_complexity

5. Export to TensorFlow Lite (.tflite)

python3 scripts/export_tflite.py \
  --model TecSingleInterfering \
  --checkpoint_path /tmp/tec_checkpoints/best.ckpt \
  --output_tflite /tmp/tec_model.tflite \
  --num_frames 200 \
  --text_length 100 \
  --decode_steps 200 \
  --quantize \
  --verify

---

Published Paper Reference Results

For reference, Table 3 of the original paper (arXiv:2008.06006v4) reported the following results using Google's internal speech infrastructure on 24 kHz LibriTTS mixed at 0 dB SNR with reverberant LJ Speech (single interfering voice) and VCTK (multiple interfering voices):

Condition	Method	WER (%) test-clean ↓	WER (%) test-other ↓	MCD (dB) test-clean ↓	MCD (dB) test-other ↓	MOS test-clean ↑	MOS test-other ↑	Side input (KB) ↓	GFLOPS ↓
Ground-truth LibriTTS	-	2.30	4.50	0.00	0.00	4.43 ± 0.04	3.82 ± 0.06	-	-
Single interfering voice (LibriTTS + LJ Speech)	Microphone signal	89.9	120.5	18.83	21.44	-	-	-	-
	AEC-NLMS	48.6	60.1	12.26	12.57	1.95 ± 0.10	1.28 ± 0.09	310	0
	Vanilla-Seq2seq	25.4	54.0	7.85	8.84	1.99 ± 0.06	1.47 ± 0.05	0	6.32
	AEC-Seq2seq	8.30	24.3	6.38	7.07	2.77 ± 0.07	1.90 ± 0.06	310	9.51
	TEC (proposed)	15.5	39.8	7.51	8.54	2.20 ± 0.07	1.65 ± 0.06	0.10	7.27
Multiple interfering voices (LibriTTS + VCTK)	Microphone signal	29.7	44.6	10.75	12.88	-	-	-	-
	AEC-NLMS	15.5	35.5	6.57	8.13	2.06 ± 0.11	1.60 ± 0.08	230	0
	Vanilla-Seq2seq	19.7	38.7	7.53	8.87	2.16 ± 0.07	1.50 ± 0.05	0	6.32
	AEC-Seq2seq	6.90	19.8	5.04	5.72	2.90 ± 0.07	2.03 ± 0.07	230	8.62
	TEC (proposed)	14.8	32.5	6.46	7.71	2.39 ± 0.07	1.70 ± 0.06	0.06	6.90

⋆ Note (Table 3 & Section 3.4–3.5 of the paper):

• The side input size and GFLOPS in the two conditions are different since the average lengths of the echo signal are different in the two conditions (~7 seconds per utterance in LJ Speech vs. ~2 seconds per utterance in VCTK).
• In the paper, models were trained on 2×2 TPU slices with a global batch size of 32 using the Adam optimizer (beta1=0.9, beta2=0.999, eps=1e-6) and an initial learning rate of 1e-4 exponentially decaying to 1e-5 after 50,000 iterations.

Interactive Demo & Pretrained Models on Hugging Face

• Interactive Demo (Hugging Face Space): https://huggingface.co/spaces/wq2012/tec — Upload any microphone mixture audio file and enter the interfering TTS text to run Textual Echo Cancellation and compare Automatic Speech Recognition (ASR) results before and after TEC.

We also provide pretrained checkpoints (best.ckpt.*) and quantized TensorFlow Lite models (model.tflite) on Hugging Face for both conditions and all neural architectures (TEC, AEC-Seq2seq, and Vanilla-Seq2seq):

• Single Interfering Voice (LibriTTS + LJSpeech):
  • TEC (proposed): wq2012/tec_single_interfering
  • AEC-Seq2seq: wq2012/aec_single_interfering
  • Vanilla-Seq2seq: wq2012/vanilla_seq2seq_single_interfering
• Multiple Interfering Voices (LibriTTS + VCTK):
  • TEC (proposed): wq2012/tec_multi_interfering
  • AEC-Seq2seq: wq2012/aec_multi_interfering
  • Vanilla-Seq2seq: wq2012/vanilla_seq2seq_multi_interfering

import os
import numpy as np
from huggingface_hub import snapshot_download
from tec import inference

# Download pretrained TecSingleInterfering model from Hugging Face
model_dir = snapshot_download(repo_id="wq2012/tec_single_interfering")
result = inference.run_inference_on_wav(
    model_name="TecSingleInterfering",
    mixed_wav_path="/path/to/mixed_input.wav",
    interfering_text="currently in mountain view it is 72 degrees",
    checkpoint_path=os.path.join(model_dir, "best.ckpt"),
    output_wav_path="/tmp/enhanced_clean.wav",
)

# Or run on-device inference with the quantized TensorFlow Lite model
tflite_out = inference.run_tflite_inference(
    tflite_path=os.path.join(model_dir, "model.tflite"),
    mixed_mel=np.zeros((100, 128), dtype=np.float32),
    interfering_text="currently in mountain view it is 72 degrees",
)

Using the standalone data preparation (scripts/prepare_data.py), training (scripts/train.py), and evaluation (scripts/evaluate.py) pipelines in this repository on the open-source LibriTTS (train-clean-100, dev-clean, test-clean, test-other), LJSpeech-1.1 (train/dev/test splits), and VCTK-0.92 (per-speaker train/dev/test splits across 109 speakers) datasets at 24 kHz (400 training utterances and 100 validation utterances per condition, mixed at 0 dB SNR with synthetic room impulse responses at RT60 = 0.25 s, trained for 500 steps with validation-loss early stopping best.ckpt, batch_size=8, learning_rate=1e-3, and evaluated with local Qwen3-ASR-0.6B-F16 via audio.cpp across N = 100 utterances per test split with 10,000-resample non-parametric bootstrap 95% confidence intervals):

Condition	Method	Hugging Face Model	WER (%) test-clean ↓	WER (%) test-other ↓	MCD (dB) test-clean ↓	MCD (dB) test-other ↓	Side Input test-clean (KB) ↓	Avg Complexity (GFLOPS) ↓
Ground-truth LibriTTS	GroundTruth	—	3.86 (41/1062) [2.43, 5.49]	6.62 (67/1012) [4.66, 8.78]	0.00	0.00	0.000	0.00
Single interfering voice (LibriTTS + LJSpeech)	MicrophoneSignal	—	80.32 (853/1062) [68.49, 93.55]	102.77 (1040/1012) [88.25, 119.32]	9.92 [9.56, 10.27]	10.74 [10.32, 11.16]	0.000	0.00
	NlmsAec (AEC-NLMS)	—	58.38 (620/1062) [47.37, 71.05]	85.77 (868/1012) [71.61, 102.19]	9.51 [9.16, 9.86]	10.33 [9.92, 10.75]	235.403	0.24
	NoSideInputSingleInterfering (Vanilla-Seq2seq)	wq2012/vanilla_seq2seq_single_interfering	39.08 (415/1062) [30.26, 48.91]	54.35 (550/1012) [44.66, 64.93]	7.07 [6.86, 7.29]	7.38 [7.13, 7.64]	0.000	4.17
	AecSingleInterfering (AEC-Seq2seq)	wq2012/aec_single_interfering	7.91 (84/1062) [5.68, 10.48]	17.29 (175/1012) [13.34, 21.76]	4.97 [4.60, 5.35]	4.96 [4.61, 5.31]	235.403	6.36
	TecSingleInterfering (TEC)	wq2012/tec_single_interfering	29.76 (316/1062) [23.09, 36.97]	50.59 (512/1012) [41.96, 59.66]	5.52 [5.16, 5.89]	5.55 [5.18, 5.93]	0.074	4.62
Ground-truth LibriTTS (Multi split)	GroundTruth	—	3.06 (30/979) [1.90, 4.35]	6.85 (65/949) [4.77, 9.16]	0.00	0.00	0.000	0.00
Multiple interfering voices (LibriTTS + VCTK)	MicrophoneSignal	—	50.77 (497/979) [40.86, 62.15]	67.33 (639/949) [56.40, 79.77]	7.50 [7.13, 7.87]	7.82 [7.41, 8.24]	0.000	0.00
	NlmsAec (AEC-NLMS)	—	34.22 (335/979) [25.72, 44.51]	48.58 (461/949) [39.15, 59.91]	7.01 [6.67, 7.37]	7.43 [7.04, 7.83]	203.138	0.21
	NoSideInputMultiInterfering (Vanilla-Seq2seq)	wq2012/vanilla_seq2seq_multi_interfering	63.33 (620/979) [55.27, 71.69]	69.86 (663/949) [61.89, 78.18]	8.26 [7.76, 8.76]	9.32 [8.77, 9.89]	0.000	3.59
	AecMultiInterfering (AEC-Seq2seq)	wq2012/aec_multi_interfering	6.84 (67/979) [4.98, 8.85]	14.86 (141/949) [11.25, 18.79]	4.65 [4.28, 5.02]	4.67 [4.28, 5.06]	203.138	5.48
	TecMultiInterfering (TEC)	wq2012/tec_multi_interfering	24.31 (238/979) [18.94, 30.47]	36.25 (344/949) [30.34, 42.52]	4.74 [4.42, 5.07]	4.91 [4.54, 5.30]	0.050	3.90

Text-Content Sensitivity Ablation (N = 100 per split)

To verify that TtsEncoderV2 actively exploits the semantic and phonetic content of the interfering TTS transcript rather than acting as a content-independent bias, we evaluate TecSingleInterfering and TecMultiInterfering under three text side-input conditions on the exact same audio mixtures:

Split	Side-Input Condition	WER (%) ↓	MCD (dB) ↓	ΔWER vs. matched	Paired p-value (Bootstrap / Wilcoxon)
single_test_clean (TecSingleInterfering)	matched (true TTS transcript)	29.76% (316/1062)	5.52 dB	—	—
	shuffled (mismatched transcript)	70.90% (753/1062)	9.26 dB	+41.15%	p < 0.0001 / p = 2.61e-12
	empty (no text)	71.94% (764/1062)	9.32 dB	+42.18%	p < 0.0001 / p = 4.89e-13
	Vanilla-Seq2seq (NoSideInputSingleInterfering)	39.08% (415/1062)	7.07 dB	+9.32%	p < 0.0001 / p = 6.69e-04
multi_test_clean (TecMultiInterfering)	matched (true TTS transcript)	24.31% (238/979)	4.74 dB	—	—
	shuffled (mismatched transcript)	48.52% (475/979)	7.92 dB	+24.21%	p < 0.0001 / p = 9.10e-09
	empty (no text)	47.29% (463/979)	7.83 dB	+22.98%	p < 0.0001 / p = 1.41e-07
	Vanilla-Seq2seq (NoSideInputMultiInterfering)	63.33% (620/979)	8.26 dB	+39.02%	p < 0.0001 / p = 1.99e-11

Input SNR Ablation on single_test_clean (N = 100 mixtures per SNR)

Input SNR	MicrophoneSignal WER / MCD	NlmsAec WER / MCD	Vanilla-Seq2seq WER / MCD	TecSingleInterfering WER / MCD	AecSingleInterfering WER / MCD
+5 dB	51.88% / 9.03 dB	40.58% / 8.72 dB	9.23% / 6.28 dB	8.47% / 4.69 dB	5.65% / 4.33 dB
0 dB	80.32% / 9.92 dB	58.38% / 9.51 dB	39.08% / 7.07 dB	29.76% / 5.52 dB	7.91% / 4.97 dB
-5 dB	119.02% / 10.73 dB	90.30% / 10.28 dB	96.80% / 7.82 dB	87.76% / 6.34 dB	17.23% / 5.73 dB
-10 dB	124.86% / 11.40 dB	121.28% / 10.95 dB	103.86% / 8.35 dB	101.04% / 6.91 dB	49.25% / 6.33 dB

---

Running Unit Tests

To run the full test suite locally:

bash run_tests.sh

---

Citation

If you find this library useful in your research, please cite the paper:

@inproceedings{ding2021textual,
  title={Textual Echo Cancellation},
  author={Ding, Shaojin and Jia, Ye and Hu, Ke and Wang, Quan},
  booktitle={2021 IEEE Spoken Language Technology Workshop (SLT)},
  pages={653--660},
  year={2021},
  organization={IEEE}
}