Tacotron 2 for Portuguese TTS

This is an adaptation of NVIDIA's tacotron2 repository to train and experiment with portuguese TTS models, following the work "A Corpus of Neutral Voice Speech in Brazilian Portuguese". More info about the data and a notebook to directly generate speech from portuguese pretrained models can be found at kaggle. To interact and contribute with with our data, models and code, please follow the instructions below.

---

Pre-requisites

1. NVIDIA GPU + CUDA cuDNN

Setup (Portuguese)

1. Download and extract the G Neutral Speech Male Dataset
2. Clone this repo: git clone https://github.com/mediatechlab/tacotron2.git
3. CD into this repo: cd tacotron2
4. Initialize submodule: git submodule init; git submodule update
5. Update .wav paths: sed -i -- 's,DUMMY,gneutral_speech_dataset_folder/wavs,g' filelists/*.txt
   • Alternatively, set load_mel_from_disk=True in hparams.py and update mel-spectrogram paths
6. Install PyTorch 1.0
7. Install Apex
8. Install python requirements or build docker image
   • Install python requirements: pip install -r requirements.txt

Training

1. python train.py --output_directory=outdir --log_directory=logdir
2. (OPTIONAL) tensorboard --logdir=outdir/logdir

Training using a pre-trained model

Training using a pre-trained model can lead to faster convergence
By default, the dataset dependent text embedding layers are ignored

1. Download our Tacotron 2 Portuguese Model.
2. python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start

Multi-GPU (distributed) and Automatic Mixed Precision Training

1. python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True

Inference demo

1. Download our published Tacotron 2 and Waveglow Portuguese Models
2. jupyter notebook --ip=127.0.0.1 --port=31337
3. Load inference.ipynb N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2 and the Mel decoder were trained on the same mel-spectrogram representation.

*NEW* Google Colab

Now you can try out tts in portuguese with studio quality in google colab with this notebook (subject to our terms of use).

Related repos

WaveGlow Faster than real time Flow-based Generative Network for Speech Synthesis

nv-wavenet Faster than real time WaveNet.

Tacotron2-MMI

Acknowledgements

This implementation uses code from the following repos: Keith Ito, Prem Seetharaman as described in our code.

We are inspired by Ryuchi Yamamoto's Tacotron PyTorch implementation.

We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang.