Fine-tuning for the Multilingual(Japanese, Chinese, Korean, English) Language

[kadirnar]

Hi there, instead of training separate models for each language, I trained 2 models with multilingual support. The results weren't perfect, but I got some good outcomes.

Models:

• https://huggingface.co/kadirnar/Orpheus-TTS-Starrail (I think that's the best)
• https://huggingface.co/kadirnar/Orpheus-TTS-Genshin

Datasets:

• https://huggingface.co/datasets/simon3000/starrail-voice
• https://huggingface.co/datasets/simon3000/genshin-voice

[stlohrey]

Hi,
did you try combined learning for other languages? Also, did you try introducing a language tag in the finetuning data? i Think this could be helpful. Maybe true multilingual training could help reducing the amount of data needed for new languages and help to get better results in low resource languages?

[kadirnar]

I haven't tried the language tag but I will. For this, I'll collect a large dataset and do pretraining. It's difficult to create high-quality voices with finetuning.

[shekharmeena2896]

Hi I am trying to finetune on African languages, most of them audio transcripts are in roman/latin, orignally they are itself written roman/latin, Is it possible to finetune for new language