Tacotron2

Model description

This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize timedomain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of 4.53 comparable to a MOS of 4.58 for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the input to WaveNet instead of linguistic, duration, and F_0 features. We further demonstrate that using a compact acoustic intermediate representation enables significant simplification of the WaveNet architecture.

Step 1: Installing packages

pip3 install -r requirements.txt 

Step 2: Preparing datasets

1.Download and extract the LJ Speech dataset in the current directory;

• wget -c https://data.keithito.com/data/speech/LJSpeech-1.1.tar.bz2;
• tar -jxvf LJSpeech-1.1.tar.bz2;

Step 3: Training

First, create a directory to save output and logs.

$ mkdir outdir logdir

On single GPU

$ python3 train.py --output_directory=outdir --log_directory=logdir --target_val_loss=0.5

Multiple GPUs on one machine

$ python3 -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True --target_val_loss=0.5

Multiple GPUs on one machine (AMP)

$ python3 -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True --target_val_loss=0.5

Results on BI-V100

Reference

https://github.com/NVIDIA/tacotron2