Björn Schuller

The Handbook of Multimodal-Multisensor Interfaces provides the first authoritative resource on what has become the dominant paradigm for new computer interfaces— user input involving new media (speech, multi-touch, gestures, writing) embedded in multimodal-multisensor interfaces. These interfaces support smart phones, wearables, in-vehicle and robotic applications, and many other areas that are now highly competitive commercially. This edited collection is written by international experts and pioneers in the field. It provides a textbook, reference, and technology roadmap for professionals working in this and related areas. This first volume of the handbook presents relevant theory and neuroscience foundations for guiding the development of high-performance systems. Additional chapters discuss approaches to user modeling and interface designs that support user choice, that synergistically combine modalities with sensors, and that blend multimodal input and output. This volume also highlights an in-depth look at the most common multimodal-multisensor combinations—for example, touch and pen input, haptic and non-speech audio output, and speech-centric systems that co-process either gestures, pen input, gaze, or visible lip movements. A common theme throughout these chapters is supporting mobility and individual differences among users. These handbook chapters provide walk-through examples of system design and processing, information on tools and practical resources for developing and evaluating new systems, and terminology and tutorial support for mastering this emerging field. In the final section of this volume, experts exchange views on a timely and controversial challenge topic, and how they believe multimodal-multisensor interfaces should be designed in the future to most effectively advance human performance.

This book presents the methods, tools and techniques that are currently being used to recognise (automatically) the affect, emotion, personality and everything else beyond linguistics (‘paralinguistics’) expressed by or embedded in human speech and language. It is the first book to provide such a systematic survey of paralinguistics in speech and language processing. The technology described has evolved mainly from automatic speech and speaker recognition and processing, but also takes into account recent developments within speech signal processing, machine intelligence and data mining. Moreover, the book offers a hands-on approach by integrating actual data sets, software, and open-source utilities which will make the book invaluable as a teaching tool and similarly useful for those professionals already in the field. Key features: Provides an integrated presentation of basic research (in phonetics/linguistics and humanities) with state-of-the-art engineering approaches for speech signal processing and machine intelligence.Explains the history and state of the art of all of the sub-fields which contribute to the topic of computational paralinguistics.C overs the signal processing and machine learning aspects of the actual computational modelling of emotion and personality and explains the detection process from corpus collection to feature extraction and from model testing to system integration.Details aspects of real-world system integration including distribution, weakly supervised learning and confidence measures.Outlines machine learning approaches including static, dynamic and context-sensitive algorithms for classification and regression.Includes a tutorial on freely available toolkits, such as the open-source ‘openEAR’ toolkit for emotion and affect recognition co-developed by one of the authors, and a listing of standard databases and feature sets used in the field to allow for immediate experimentation enabling the reader to build an emotion detection model on an existing corpus.

Inhaltlich unver nderte Neuauflage. Integration emotionaler Aspekte ist Basis nat rlicher und zukunftsweisender Mensch-Maschine-Kommunikation. Vor diesem Hintergrund werden in-novative Verfahren zur robusten maschinellen Erkennung menschlicher Ge-f hle aus Sprache und manueller Interaktion vorgestellt. Die Besonderheit der Sch tzung aus dem akustischen Sprachsignal liegt in der automatischen Ge-nerierung und Selektion neuartiger Merkmale sowie einem extensiven Ver-gleich diverser Lernverfahren. Eine inhaltliche Interpretation unterst tzt die Erkennung und erm glicht diese auch aus geschriebener Eingabe. Als Novum wird auch konventionelle Bedienung ber eine Computermaus oder einen Touchscreen ohne erg nzende Hardware betrachtet. ber singul re Ansatz-punkte hinaus werden Methoden der synergetischen Integration vorgestellt. Den Abschluss bilden der praxistaugliche Einsatz und Transfer erarbeiteter Methodik. Als Ergebnis kann Emotion unter idealen Bedingungen maschinell vergleichbar einem menschlichen Entscheider klassifiziert werden. Das Buch richtet sich somit vorrangig an Ingenieure und Informatiker der Fachbereiche Signalverarbeitung, Mustererkennung und Ergonomie, aber auch Psycho-logen, Phonetiker und Linguisten.