MIN Faculty
Department of Informatics
Knowledge Technology


This webpage is outdated and has moved. Please find the official Knowledge Technology page at:



CINACS Lecture Series: Cross-modal Interaction in Natural and Artificial Cognitive Systems

Winter Semester 2010/2011

Information/Allgemeine Informationen

LV-Nummer: 64-631
Lecturer: Christopher Habel, Wolfgang Menzel, Stefan Wermter, Jianwei Zhang
Period: Mo. 14:15-15:45h, weekly
Room: Informatikum C-101
Credit Hours 2 SWS
Language: English



Natural cognitive systems - like humans - profit from combining the input of the different sensory systems not only because each modality provides information about different aspects of the world but also because the different senses can jointly encode particular aspects of events, e.g. the location or meaning of an event. However, the gains of cross-modal integration come at a cost: since each modality uses very specific representations, information needs to be transferred into a code that allows the different senses to interact. Corresponding problems arise in human communication when information about one topic is expressed using combinations of different formats such as written or spoken language and graphics.

In this lecture, we will focus on models and methods suitable to realize processes and representations for cross-modal interactions in artificial cognitive systems, i.e. computational systems. After introducing the core phenomena of cross-modal interaction we exemplify the mono- modal basis of cross-modal interaction and the current development of informatics-oriented research in this field with four topics: