Material Detail

Multi-stream modeling with applications in speech and multimodal processing

Multi-stream modeling with applications in speech and multimodal processing

This video was recorded at Machine Learning Workshop, Sheffield 2004. After a brief discussion of the problem arising from the processing and modeling of multiple stream (multi-channel, multi-sensor) signals, we will discuss a few statistical structures (such as multi-stream HMM and asynchronous HMM) that can accommodate multiple (asynchnonous) observation streams (possibly exhibiting different frame rates). Indeed, it will be shown on different speech recognition and multimodal fusion tasks that it might sometimes be a good idea to be able to ``desynchronize'' the streams in order to maximize their joint likelihood. Different applications in speech recognition, such as multi-band and multi-stream speech processing, will be discussed. Finally, multimodal applications significantly benefiting from this multi-stream paradigm will also be discussed, including audio-visual speech recognition and modeling of human interaction in meetings (by modeling the joint behaviours of participants through multiple audio and visual features).

Quality

  • User Rating
  • Comments
  • Learning Exercises
  • Bookmark Collections
  • Course ePortfolios
  • Accessibility Info

More about this material

Comments

Log in to participate in the discussions or sign up if you are not already a MERLOT member.