Click here to flash read.
Recently, there has been a growing interest in text-to-speech (TTS) methods
that can be trained with minimal supervision by combining two types of discrete
speech representations and using two sequence-to-sequence tasks to decouple
TTS. To address the challenges associated with high dimensionality and waveform
distortion in discrete representations, we propose Diff-LM-Speech, which models
semantic embeddings into mel-spectrogram based on diffusion models and
introduces a prompt encoder structure based on variational autoencoders and
prosody bottlenecks to improve prompt representation capabilities.
Autoregressive language models often suffer from missing and repeated words,
while non-autoregressive frameworks face expression averaging problems due to
duration prediction models. To address these issues, we propose
Tetra-Diff-Speech, which designs a duration diffusion model to achieve diverse
prosodic expressions. While we expect the information content of semantic
coding to be between that of text and acoustic coding, existing models extract
semantic coding with a lot of redundant information and dimensionality
explosion. To verify that semantic coding is not necessary, we propose
Tri-Diff-Speech. Experimental results show that our proposed methods outperform
baseline methods. We provide a website with audio samples.
