Neuro-inspired Human-Robot Interaction
General Information |
|
| Humans use different types of tools, or modalities, to communicate in their everyday lives, including spoken and written language, sign language, body gestures, facial expressions and also computational interfaces. For humans, the means to interact with each other is of vital importance in a social environment, but, communication does not occur exclusively amongst humans since animals and robots are also able to communicate using various modalities. The aim of the research in our group is to contribute to fundamental research in offering functional models for testing neuro-cognitive hypotheses about aspects of human communication, and in providing efficient bio-inspired methods to produce robust controllers for a communicative robot that successfully engages in human-robot interaction. |
|
| Leading Investigator: | Prof. Dr. S. Wermter, Dr. C. Weber |
| Associates: | S. Heinrich, D. Jirak, Dr. S. Magg |
 |
|
Natural Language Learning on a Neural Cognitive Robot - Stefan Heinrich
 |
In the last decade the debate has been intensified if humans have either an amodel system for language and higher cognition or whether everything we think or talk about is grounded in the modalities of our sensory, sensori-motor or motor systems. The view that language is likely to be distributed over the brain is supported by recent neuro-scientific evidence based on fMRI and EEG measurements and valuable theoretical hypotheses based on psycho-linguistic long-term studies. Previous neuro-cognitive models have contributed in testing hypotheses about how functional webs for words or morphemes could occur and raised a common question: What are the functional dynamics of these distributed neural representations? |
| This research project aims to approach this general question from a computational perspective and to investigate: What are the functional dynamics in a plausible neural model, which also takes into account the temporal nature of morpheme or word sequences, namely sentences. The research investigate different recent continuous time recurrent neural networks (CTRNNs) in processing sentences with respect to cross-modal communication, integrating visual context information. At a later stage of this project, it is intended to integrate the recurrent architecture on a robotic platform to evaluate the capabilities in learning semantic mappings of language and the context, and to generalise to unheard sentences in a teacher-learner scenario. |  |
|
|
|
Vision-Based Gesture Recognition - Doreen Jirak
|
|
|
Evolving Neural Agents - Sven Magg
| Neural networks are often used to control autonomous agents or form part of a controller in a hybrid approach. With growing complexity of such agents and their tasks, the corresponding controller inevitably has to become more complex as well. Instead of only increasing network sizes to allow for higher complexity, different mechanisms, which lead to more complex time dependent behaviour can be incorporated. | |
 |
Neuromodulation is one mechanism that can be found in mammalian brains and is involved in higher-level cognitive functions like decision-making, attention, and emotion. Understanding its influence on biological networks might lead to models, which can be used to improve the function of artifial neural network controllers. In complex networks with a large parameter space, defining the necessary topology and finding a working parameter set, becomes increasingly difficult. One way to solve this problem is artificial evolution, which uses biologically inspired mechanisms like selection, reproduction, and mutation. Each individual in this process is evaluated using a fitness criterion that defines how well a network solves the given problem. By repeatedly selecting promising candidates from a population and changing them though recombination and mutation, the evolutionary process constitutes a parallel search through the space of possible solutions. |
| In this project, we want to investigate computational models for neuromodulation in neural networks, to identify mechanisms, which can be efficiently used in robot controllers. In addition to this bottom-up approach, we also want to find working controllers through an evolutionary process that utilise neuromodulatory effects. By analysing successful controllers (network dynamics as well as their development through evolution), we hope to gain essential knowledge to improve models found in the first step. Since the speed in which successful controllers evolve depends on the combination of evolutionary operators, network features, and task class, analysis of the process will also enable us to identify efficient combinations of operators and network types for given tasks. The aim is to arrive at "optimal" combinations, to be able to create efficient robot controllers quickly for different problems. |  |
|
|
|
Related Publications
Heinrich, S., Natural Language Acquisition in Recurrent Neural Architectures, Dissertation, Department of Computer Science, University of Hamburg, Germany, March/June 2016.
|
|
Heinrich, S., Magg, S., Wermter, S. Analysing the Multiple Timescale Recurrent Neural Network for Embodied Language Understanding. In Koprinkova-Hristova, P.D., et al., editors, Artificial Neural Networks - Methods and Applications, SSBN 4, pp. 149-174, Springer International Publishing, 2015.
|
|
Heinrich, S., Wermter, S. Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. In Wermter, S., et al., editors, Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN 2014), pp. 193-200, Springer Heidelberg. Hamburg, DE, September 2014.
|
|
Heinrich, S., Weber, C., Wermter, S. Embodied Language Understanding with a Multiple Timescale Recurrent Neural Network. Proceedings of the 23rd International Conference on Artificial Neural Networks (ICANN 2013), LNCS 8131, pp. 216-223, Springer Heidelberg. Sofia, BG, September 2013.
|
|
Heinrich, S., Folleher, P., Springstübe, P., Strahl, E., Twiefel, J., Weber, C., Wermter, S. Object Learning with Natural Language in a Distributed Intelligent System - A Case Study of Human-Robot Interaction. Proceedings of the IEEE First International Conference on Cognitive Systems and Information Processing (CSIP 2012), AISC 215, 8 p., Springer Berlin. Beijing, CN, December 2012.
|
|
Heinrich, S., Weber, C., Wermter, S. Adaptive Learning of Linguistic Hierarchy in a Multiple Timescale Recurrent Neural Network. Proceedings of the 22nd International Conference on Artificial Neural Networks (ICANN 2012), LNCS 7552, pp. 555-562, Springer. Lausanne, CH, September 2012.
|
|
Heinrich, S., Wermter, S. Towards Robust Speech Recognition for Human-Robot Interaction. Proceedings of the IROS2011 Workshop on Cognitive Neuroscience Robotics (CNR), pp. 29-34, San Francisco, CA, USA, September 2011.
|
|
Magg, S., Philippides, A. GasNets and CTRNNs - A Comparison in Terms of Evolvability. In Proceedings of From Animals to Animats 9: 9th International Conference on Simulation of Adaptive Behavior (SAB'06), LNCS, Vol. 4095, pp. 461-472, Springer Berlin/Heidelberg. Rome, IT, September 2006. |
|
|
|
|
Further Information |
|
|
The research in the Knowledge Technology Group is closely related with the teaching we offer. Periodically we offer lectures, seminars, practical courses, and projects concerning with:
|
