Is it possible to train any other language?

[thepyninja]

No description provided.

[thepyninja]

Hello, as you mentioned "Language-dependent frontend text processor for English and Japanese
".
What does it mean?
can we train any other language with it or not?

[r9y9]

It means you don't need to write text processing frontend for English and Japanese. For any other language, you need to write your own text processor.

[amilamad]

@r9y9
Hi
I have one question. Can we use transliteration to convert other language to English and use English frontend to train the model and for synthesizing ?

[r9y9]

@amilamad It may work but I have never tried it.

[pbaljeka]

You can use the SAMPA phoneset.. uses a lookup table to map from unicode characters to similar sounds...Its used to build grapheme based voices in Festival.
Lookup table:
https://github.com/festvox/festvox/blob/master/src/grapheme/unicode_sampa_mapping.scm
Phoneset:
https://github.com/festvox/festvox/blob/master/src/grapheme/sampa.table
I have trained a multilingual-multispeaker model with Bengali, Hindi, Marathi, Gujarathi, English.. results aren't too great.. but kind of understandable..

Unfactorised embedding: Example wavs files (each language-speaker pair has one id)
http://tts.speech.cs.cmu.edu/pbaljeka/www/test_wavs/mlms_test_135k+140k_0.168_all/

Factorized: Example wavs files (each language, speaker , gender has separate embedding)
http://tts.speech.cs.cmu.edu/pbaljeka/www/mlms_redo_wavs/

[amilamad]

@pbaljeka Did you use https://github.com/r9y9/deepvoice3_pytorch project for generating above samples?.
can I know the size of the data samples that was used for the training ?
Did you wrote your own frontend ?

[pbaljeka]

@amilamad Yes I used this repo for it. The data was a mix of IITM blizzard voices and festvox indic datasets.. so from about 9 hours of data to less than 20 mins for some speakers and languages. I used the Festvox frontend with sampa phones.

[amilamad]

@pbaljeka
Thank you :)

[tbornt]

@r9y9
Hi,
I think the text processor makes the dataset into a list of (spectrogram_filename, mel_filename, n_frames, text) format. And for some dataset, we need to do something like phoneme alignment, silence removing before text processing. Is this right?

[r9y9]

@tbornt You are right. See https://github.com/r9y9/deepvoice3_pytorch/tree/master/vctk_preprocess for example.

[tbornt]

@r9y9
Thanks! So it is not that difficult to train deepvoice for other languages. Maybe we can write a guide and then build a model zoo.

[r9y9]

Yes, I agree. Also #78 will help people work on their own datasets. Flexible custom dataset support is pretty cool. I'm reviewing now.

[fpanchoro]

Hello, I have not seen anything applied for Spanish language, do you know if there is any dataset that could help me to train the training?

[imdatceleste]

Hi @fpanchoro , you may want to checkout our free dataset at M-AILABS Speech Dataset...

[aishweta]

Can anyone share Indian language dataset link here. Would like to test this repo on indian language

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.