WeTextProcessing 0.0.3

WeTextProcessing, including TN & ITN

Text Normalization & Inverse Text Normalization

1. How To Use

1.1 pip install

pip install WeTextProcessing

# tn
from tn.chinese.normalizer import Normalizer
normalizer = Normalizer()
normalizer.normalize("2.5平方电线")
# itn
from itn.chinese.inverse_normalizer import InverseNormalizer
invnormalizer = InverseNormalizer()
invnormalizer.normalize("二点五平方电线")

1.2 source code compilation

git clone https://github.com/wenet-e2e/WeTextProcessing.git
cd WeTextProcessing

python normalize.py --text "2.5平方电线"
python inverse_normalize.py --text "二点五平方电线"

2. TN Pipeline

Please refer to TN.README

3. ITN Pipeline

Please refer to ITN.README

Acknowledge

1. Thank the authors of foundational libraries like OpenFst & Pynini.
2. Thank NeMo team & NeMo open-source community.
3. Thank Zhenxiang Ma, Jiayu Du, and SpeechColab organization.
4. Referred Pynini for reading the FAR, and printing the shortest path of a lattice in the C++ runtime.
5. Referred TN of NeMo for the data to build the tagger graph.
6. Referred ITN of chinese_text_normalization for the data to build the tagger graph.