As opposed to most of the existing hybrid methods that are focused on improving the quality of a unit selection system, here, we propose a hybrid SSS/unit selection algorithm to boost the quality of our Turkish SSS system.
Because a complete unit selection synthesis system is not needed, the expensive processes of building a unit selection system or additional data collection are avoided.
(2) A unit selection algorithm is used to determine the closest stored unit to each segment in the target utterance.
Depending on the phoneme information, the unit selection selects mouth images from the database and assembles them in an optimal way to produce the desired animation.
Towards this goal, we propose two deep neural network (DNN) architectures for Video-realistic Expressive Audio-Visual Text-To-Speech synthesis (EAVTTS) and evaluate them by comparing them directly both to traditional hidden Markov model (HMM) based EAVTTS, as well as a concatenative unit selection EAVTTS approach, both on the realism and the expressiveness of the generated talking head.
In a typical unit selection based TTS system, target cost and concatenation cost are used in selecting the units.
Our audiovisual synthesis system is designed as an extension of our unit selection auditory TTS system, which uses a Viterbi search on cost functions to select the optimal sequence of long nonuniform units from the database [20].
Furthermore, we show some implications of unit-cost improvement: in a short run, a firm is better off concentrating on the improvement of the unit selection cost rather than the unit customizing cost.
Given a query Q with n frames (and equivalently, an utterance U with m frames), three speech representations that result in a set Q={q1,…,q n } of n vectors of dimension D (and equivalently, a set of U={u1,…,u m } of m vectors of dimension D) are based on: Phoneme posteriorgram + phoneme unit selection: This speech representation relies on phoneme posteriorgrams [34].
The quality of synthesized animations depends mainly on the database and the unit selection.
