The audio-visual system is also expected to record the visual and audio interactions between the presenter and each audience.
For the developer, the framework should provide a set of concepts that simplify the process of thinking about and programming audio interactions without overly restricting their scope.
As game developers know, situations like the above may be simple to grasp but grow much more complicated when you're dealing with dozens of these audio interactions taking place simultaneously.
In particular, use of a browser-based audio-video conferencing tool, in which each individual is represented by a "real time" (web-cam) image, and by audio interactions through a microphone headset, provides a strong semblance of face-to-face interactions which allow participants to present themselves as "real people".
So in Toca Dance, there's dubstep, and parents might hear that coming from the tablet their kid is playing with and wonder, 'What is that?' So we're creating openings – visually, through audio, or interaction wise – for parents, so they can have this interest in what their kids are playing, and feel more engaged themselves".
Within the small group of scientists interested in multisensory integration, the paper heralded a new direction for the field, a shift from teasing out the mechanics of audio-visual interaction to what Paul Breslin, an experimental psychologist at Rutgers University, described as "the new frontier" of oral perception.
Micro-analytic observational methods were used to analyse the audio-recorded interaction of the children as they played.
There is a real-time processor that is listening to and "reading" all the audio from interactions as they take place.
Engineers have been interested in the audio-visual interaction from the perspective of optimizing the perception of quality offered by technologies [4, 5].
This research represents a first step in understanding the effects of audio-visual interaction on the perception of tranquillity and identifies how the interpretation of acoustic information is an integral part of this process.
The SDK is now available to all via the Daydream developer website, and supports "integrated asynchronous reprojection, high fidelity spatialized audio, and interactions using the Daydream controller," writes Nathan Martz, Product Manager, Google VR.
