Advancements in Speech Synthesis: From Text to Natural Speech

2. Formant Synthesis

Formant synthesis technique is a rule-based TTS technique. It produces speech segments by generating artificial signals based on a set of specified rules mimicking the formant structure and other spectral properties of natural speech. The synthesized speech is produced using an additive synthesis and an acoustic model. The acoustic model uses parameters like, voicing, fundamental frequency, noise levels, etc that varied over time. Formant-based systems can control all aspects of the output speech, producing a wide variety of emotions and different tone voices with the help of some prosodic and intonation modeling techniques.

Advantages:

• Highly intelligible synthesized speech, even at high speeds, avoiding the acoustic glitches;
• Less dependant on a speech corpus to produce the output speech;
• Well-suited for embedded systems, where memory and microprocessor power are limited.

Disadvantages:

• Low naturalness: the technique produces artificial, robotic-sounding speech that is far from the natural speech spoken by a human.
• Difficult to design rules that specify the timing of the source and the dynamic values of all filter parameters for even simple words

3. Statistical Parametric Synthesis

HMM based Speech Synthesis

To overcome the limitations inherent in concatenative text-to-speech (TTS) systems, a statistical parametric approach was developed. This method operates on the premise that by approximating the parameters constituting speech, a model can be trained to generate diverse speech outputs. It integrates parameters such as fundamental frequency and magnitude spectrum, processing them to synthesize speech.​

The process begins with text analysis to extract linguistic features, including phonemes and duration. Subsequently, vocoder features—such as cepstra, spectrogram, and fundamental frequency—that encapsulate intrinsic characteristics of human speech are extracted for audio processing. These hand-engineered features, along with the linguistic attributes, are input into a mathematical model known as a vocoder. During waveform generation, the vocoder transforms these features and estimates speech parameters like phase, speech rate, and intonation. This technique employs Hidden Semi-Markov Models (HSMMs), which, while maintaining state transitions characteristic of Markov models, incorporate explicit duration modeling within each state.

Advantages:

• Enhanced Naturalness: This approach yields audio with improved naturalness. Although the technology for generating emotive voices is still evolving, parametric TTS shows promise in areas such as speaker adaptation and interpolation.​

• Flexibility: It facilitates easier modification of pitch to convey emotions and allows for adjustments to voice characteristics through techniques like Maximum Likelihood Linear Regression (MLLR) adaptation.​
• Reduced Development Costs: Requiring only 2–3 hours of voice actor recordings, this method necessitates fewer recordings, a smaller database, and less data processing.​


Disadvantages:

• Diminished Audio Quality: The synthesized speech may contain artifacts leading to muffled sounds and a persistent buzzing, resulting in noisy audio.​
• Robotic Tone: Due to the muffled quality inherent in statistical models, the synthesized voice can appear stable yet unnatural and robotic.

4. Neural Network-Based Synthesis

Deep Neural Network (DNN)-based speech synthesis represents a significant advancement in statistical synthesis methods, addressing the limitations of Hidden Markov Models (HMMs) that utilize decision trees for modeling complex contextual dependencies. A notable innovation in this approach is the automation of feature design, allowing machines to learn feature representations without human intervention. Unlike manually crafted features based on human understanding of speech—which may not always be accurate—DNN techniques establish relationships between input text and their acoustic realizations through deep learning models. These models generate acoustic features using maximum likelihood parameter generation with trajectory smoothing.

Advantages:

• Improved Intelligibility and Naturalness: DNN-based approaches have markedly enhanced the clarity and naturalness of synthesized speech compared to earlier methods.
• Reduced Need for Manual Feature Engineering: By automating feature learning, these methods diminish the reliance on extensive human preprocessing and the manual development of features, streamlining the synthesis process. ​


Disadvantages:

• Ongoing Research and Development: As a relatively recent innovation, deep learning-based speech synthesis techniques continue to evolve. Further research is necessary to fully address challenges such as modeling acoustic feature parameters and enhancing the expressiveness of synthesized speech. ​

WaveNet, the foundation of Google Cloud Text-to-Speech, is a fully convolutional neural network that accepts raw audio waveforms as input. These waveforms traverse multiple convolutional layers, producing output waveforms. Despite achieving near-perfect intelligibility and naturalness, WaveNet's initial implementation was notably slow, reportedly requiring approximately four minutes to generate one second of audio.​

Advancements in end-to-end training led to Google's development of the Tacotron model, which learns to synthesize speech directly from text-audio pairs. Tacotron processes text characters through various neural network submodules to generate corresponding audio spectrograms. This approach streamlines the speech synthesis process by integrating text analysis and audio generation into a unified framework.

In summary, the human voice remains the benchmark for speech synthesis technology. Continuous advancements in understanding and replicating human speech patterns are bringing us closer to creating synthesized voices that authentically capture the expressiveness and diversity of natural human communication.