We propose and evaluate a method for significantly compressing modal sound models, thereby making them far more practical for audiovisual applications.
In the audiovisual industry, applications are very data-intensive, time-constrained and computationally demanding, and design of a Service Oriented Architecture in this domain is no straightforward task.
However, because psychologists and cognitive scientists are generally interested in how complex perception unfolds whether it is face or word recognition this chapter delves into an application involving audiovisual speech perception.
It can be applied both for audiovisual and audio-only applications to significantly reduce the requirements of transmission bandwidth and data storage with low distortion.
Today's world is characterized by the increasing availability of audiovisual content, created in many application domains and, thus, with rather different semantic characteristics.
Visiting Researcher at CNMAT, Thor Magnusson will discuss work in composition, performance and the development of audiovisual software - from small instrumental applications to live coding systems.
From the results obtained we believe it is important to further investigate the importance of the coherence between audio and video modes for the perceived quality and naturalness of audiovisual speech synthesis and other applications such as audiovisual speech recognition and audio-to-audiovisual speech mapping techniques.
Finally, we propose four general audiovisual integration functions for specific applications jointly based on the results presented in this analysis and on the results from the literature.
In addition, Ries et al. [10] and Winkler and Faller [11] have recently developed audiovisual quality models for mobile applications, but the reported model versions do not yet cover the effect of transmission errors.
Further experiments were conducted in order to derive audiovisual integration functions fitted for specific applications like mobile television [35, 36] or high-definition television [37].
